2024-05-09T06:25:02.166 INFO:root:teuthology version: 0.0.1.dev271+g1ae7ad8 2024-05-09T06:25:02.166 DEBUG:teuthology.run:Teuthology command: teuthology --name teuthology-2024-05-08_21:24:04-fs-squid-distro-default-smithi --owner scheduled_teuthology@teuthology --description fs/volumes/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/volumes/{overrides test/finisher_per_module}} --verbose --archive /home/teuthworker/archive/teuthology-2024-05-08_21:24:04-fs-squid-distro-default-smithi/7698541 -- /home/teuthworker/archive/teuthology-2024-05-08_21:24:04-fs-squid-distro-default-smithi/7698541/orig.config.yaml 2024-05-09T06:25:02.214 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-05-09T06:25:02.298 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-05-08_21:24:04-fs-squid-distro-default-smithi/7698541 branch: squid description: fs/volumes/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/volumes/{overrides test/finisher_per_module}} email: ceph-qa@ceph.com first_in_suite: false job_id: '7698541' kernel: client: sha1: distro kdb: 1 sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: teuthology-2024-05-08_21:24:04-fs-squid-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: squid ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug client: 20 debug finisher: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - MON_DOWN - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - OSD full dropping all updates - OSD near full - pausewr flag - failsafe engaged, dropping updates - failsafe disengaged, no longer dropping - is full \(reached quota - POOL_FULL - POOL_BACKFILLFULL sha1: 96805dcf909541c4ca9a1e90258de8ae5b6fa1bc ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 96805dcf909541c4ca9a1e90258de8ae5b6fa1bc kclient: syntax: v2 selinux: allowlist: - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: squid sha1: be85c202969d25af6e130678e7d73a20e9c4ca11 owner: scheduled_teuthology@teuthology priority: 100 repo: https://git.ceph.com/ceph.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 2334 sha1: 96805dcf909541c4ca9a1e90258de8ae5b6fa1bc sleep_before_teardown: 0 subset: 359/512 suite: fs suite_branch: squid suite_path: /home/teuthworker/src/git.ceph.com_ceph_be85c202969d25af6e130678e7d73a20e9c4ca11/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: be85c202969d25af6e130678e7d73a20e9c4ca11 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi003.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHDTU59s3KInmJs3dd/8Ydtk7rpoIaz7d+fqqmrBTda2032gaHXoylm0yOE65o6JjhGJ7B/rmm+wl2vgamxnxr8= smithi190.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOd0Ny4wJnHAMIg19Z/g3f4ECf1K802dz6GWEEPGbzfBK6w5EeLqpVO0/i8FObMdi5eYZEtQxGdnQq5AcppZ2cs= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - sequential: - mgrmodules - kclient: null - check-counter: counters: mgr: - min: 4 name: finisher-volumes.complete_latency.avgcount - expected_val: 0 name: finisher-volumes.queue_len - cephfs_test_runner: fail_on_skip: false modules: - tasks.cephfs.test_volumes.TestPerModuleFinsherThread teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 1ae7ad82388e92a475afff437d49054826c019a1 timestamp: 2024-05-08_21:24:04 tube: smithi user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-05-09T06:25:02.299 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_be85c202969d25af6e130678e7d73a20e9c4ca11/qa; will attempt to use it 2024-05-09T06:25:02.299 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_be85c202969d25af6e130678e7d73a20e9c4ca11/qa/tasks 2024-05-09T06:25:02.300 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-05-09T06:25:02.301 INFO:teuthology.task.internal:Checking packages... 2024-05-09T06:25:02.325 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '96805dcf909541c4ca9a1e90258de8ae5b6fa1bc' 2024-05-09T06:25:02.325 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-05-09T06:25:02.325 INFO:teuthology.packaging:ref: None 2024-05-09T06:25:02.325 INFO:teuthology.packaging:tag: None 2024-05-09T06:25:02.325 INFO:teuthology.packaging:branch: squid 2024-05-09T06:25:02.325 INFO:teuthology.packaging:sha1: 96805dcf909541c4ca9a1e90258de8ae5b6fa1bc 2024-05-09T06:25:02.325 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=squid 2024-05-09T06:25:02.537 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-2529.gbf77c77e 2024-05-09T06:25:02.538 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-05-09T06:25:02.544 INFO:teuthology.task.internal:no buildpackages task found 2024-05-09T06:25:02.544 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-05-09T06:25:02.550 INFO:teuthology.task.internal:Saving configuration 2024-05-09T06:25:02.568 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-05-09T06:25:02.574 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-05-09T06:25:02.600 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi003.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-05-08_21:24:04-fs-squid-distro-default-smithi/7698541', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-09 06:20:05.066849', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHDTU59s3KInmJs3dd/8Ydtk7rpoIaz7d+fqqmrBTda2032gaHXoylm0yOE65o6JjhGJ7B/rmm+wl2vgamxnxr8='} 2024-05-09T06:25:02.621 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi190.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-05-08_21:24:04-fs-squid-distro-default-smithi/7698541', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-09 06:20:05.065360', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOd0Ny4wJnHAMIg19Z/g3f4ECf1K802dz6GWEEPGbzfBK6w5EeLqpVO0/i8FObMdi5eYZEtQxGdnQq5AcppZ2cs='} 2024-05-09T06:25:02.621 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-05-09T06:25:02.627 INFO:teuthology.task.internal:roles: ubuntu@smithi003.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2024-05-09T06:25:02.627 INFO:teuthology.task.internal:roles: ubuntu@smithi190.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2024-05-09T06:25:02.627 INFO:teuthology.run_tasks:Running task console_log... 2024-05-09T06:25:02.699 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f7c93e4e0d0>, signals=[15]) 2024-05-09T06:25:02.700 INFO:teuthology.run_tasks:Running task internal.connect... 2024-05-09T06:25:02.706 INFO:teuthology.task.internal:Opening connections... 2024-05-09T06:25:02.706 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi003.front.sepia.ceph.com 2024-05-09T06:25:02.708 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi003.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-09T06:25:02.790 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi190.front.sepia.ceph.com 2024-05-09T06:25:02.791 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi190.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-09T06:25:02.868 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-05-09T06:25:02.874 DEBUG:teuthology.orchestra.run.smithi003:> uname -m 2024-05-09T06:25:02.892 INFO:teuthology.orchestra.run.smithi003.stdout:x86_64 2024-05-09T06:25:02.892 DEBUG:teuthology.orchestra.run.smithi003:> cat /etc/os-release 2024-05-09T06:25:02.949 INFO:teuthology.orchestra.run.smithi003.stdout:NAME="CentOS Stream" 2024-05-09T06:25:02.949 INFO:teuthology.orchestra.run.smithi003.stdout:VERSION="9" 2024-05-09T06:25:02.949 INFO:teuthology.orchestra.run.smithi003.stdout:ID="centos" 2024-05-09T06:25:02.949 INFO:teuthology.orchestra.run.smithi003.stdout:ID_LIKE="rhel fedora" 2024-05-09T06:25:02.949 INFO:teuthology.orchestra.run.smithi003.stdout:VERSION_ID="9" 2024-05-09T06:25:02.949 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM_ID="platform:el9" 2024-05-09T06:25:02.950 INFO:teuthology.orchestra.run.smithi003.stdout:PRETTY_NAME="CentOS Stream 9" 2024-05-09T06:25:02.950 INFO:teuthology.orchestra.run.smithi003.stdout:ANSI_COLOR="0;31" 2024-05-09T06:25:02.950 INFO:teuthology.orchestra.run.smithi003.stdout:LOGO="fedora-logo-icon" 2024-05-09T06:25:02.950 INFO:teuthology.orchestra.run.smithi003.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-05-09T06:25:02.950 INFO:teuthology.orchestra.run.smithi003.stdout:HOME_URL="https://centos.org/" 2024-05-09T06:25:02.950 INFO:teuthology.orchestra.run.smithi003.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-09T06:25:02.950 INFO:teuthology.orchestra.run.smithi003.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-05-09T06:25:02.950 INFO:teuthology.orchestra.run.smithi003.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-05-09T06:25:02.951 INFO:teuthology.lock.ops:Updating smithi003.front.sepia.ceph.com on lock server 2024-05-09T06:25:02.970 DEBUG:teuthology.orchestra.run.smithi190:> uname -m 2024-05-09T06:25:02.987 INFO:teuthology.orchestra.run.smithi190.stdout:x86_64 2024-05-09T06:25:02.988 DEBUG:teuthology.orchestra.run.smithi190:> cat /etc/os-release 2024-05-09T06:25:03.044 INFO:teuthology.orchestra.run.smithi190.stdout:NAME="CentOS Stream" 2024-05-09T06:25:03.045 INFO:teuthology.orchestra.run.smithi190.stdout:VERSION="9" 2024-05-09T06:25:03.045 INFO:teuthology.orchestra.run.smithi190.stdout:ID="centos" 2024-05-09T06:25:03.045 INFO:teuthology.orchestra.run.smithi190.stdout:ID_LIKE="rhel fedora" 2024-05-09T06:25:03.045 INFO:teuthology.orchestra.run.smithi190.stdout:VERSION_ID="9" 2024-05-09T06:25:03.045 INFO:teuthology.orchestra.run.smithi190.stdout:PLATFORM_ID="platform:el9" 2024-05-09T06:25:03.045 INFO:teuthology.orchestra.run.smithi190.stdout:PRETTY_NAME="CentOS Stream 9" 2024-05-09T06:25:03.045 INFO:teuthology.orchestra.run.smithi190.stdout:ANSI_COLOR="0;31" 2024-05-09T06:25:03.045 INFO:teuthology.orchestra.run.smithi190.stdout:LOGO="fedora-logo-icon" 2024-05-09T06:25:03.045 INFO:teuthology.orchestra.run.smithi190.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-05-09T06:25:03.046 INFO:teuthology.orchestra.run.smithi190.stdout:HOME_URL="https://centos.org/" 2024-05-09T06:25:03.046 INFO:teuthology.orchestra.run.smithi190.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-09T06:25:03.046 INFO:teuthology.orchestra.run.smithi190.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-05-09T06:25:03.046 INFO:teuthology.orchestra.run.smithi190.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-05-09T06:25:03.046 INFO:teuthology.lock.ops:Updating smithi190.front.sepia.ceph.com on lock server 2024-05-09T06:25:03.069 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-05-09T06:25:03.081 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-05-09T06:25:03.087 INFO:teuthology.task.internal:Checking for old test directory... 2024-05-09T06:25:03.088 DEBUG:teuthology.orchestra.run.smithi003:> test '!' -e /home/ubuntu/cephtest 2024-05-09T06:25:03.090 DEBUG:teuthology.orchestra.run.smithi190:> test '!' -e /home/ubuntu/cephtest 2024-05-09T06:25:03.106 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-05-09T06:25:03.112 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-05-09T06:25:03.113 DEBUG:teuthology.orchestra.run.smithi003:> test -z $(ls -A /var/lib/ceph) 2024-05-09T06:25:03.148 DEBUG:teuthology.orchestra.run.smithi190:> test -z $(ls -A /var/lib/ceph) 2024-05-09T06:25:03.265 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-05-09T06:25:03.328 INFO:teuthology.run_tasks:Running task kernel... 2024-05-09T06:25:03.344 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': 1, 'sha1': 'distro'} 2024-05-09T06:25:03.344 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}, 'client.2': {'sha1': 'distro'}, 'client.1': {'sha1': 'distro'}, 'client.3': {'sha1': 'distro'}} 2024-05-09T06:25:03.345 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}, 'client.1': {'sha1': 'distro'}}, timeout 300 2024-05-09T06:25:03.345 DEBUG:teuthology.orchestra.run.smithi003:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-09T06:25:03.345 DEBUG:teuthology.orchestra.run.smithi190:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-09T06:25:03.361 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T06:25:03.362 DEBUG:teuthology.orchestra.run.smithi190:> uname -r 2024-05-09T06:25:03.363 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T06:25:03.364 DEBUG:teuthology.orchestra.run.smithi003:> uname -r 2024-05-09T06:25:03.419 INFO:teuthology.orchestra.run.smithi190.stdout:5.14.0-437.el9.x86_64 2024-05-09T06:25:03.420 INFO:teuthology.task.kernel:Running kernel on smithi190: 5.14.0-437.el9.x86_64 2024-05-09T06:25:03.420 DEBUG:teuthology.orchestra.run.smithi190:> sudo yum install -y kernel 2024-05-09T06:25:03.423 INFO:teuthology.orchestra.run.smithi003.stdout:5.14.0-437.el9.x86_64 2024-05-09T06:25:03.423 INFO:teuthology.task.kernel:Running kernel on smithi003: 5.14.0-437.el9.x86_64 2024-05-09T06:25:03.423 DEBUG:teuthology.orchestra.run.smithi003:> sudo yum install -y kernel 2024-05-09T06:25:06.061 INFO:teuthology.orchestra.run.smithi003.stdout:CentOS Stream 9 - BaseOS 7.8 MB/s | 8.1 MB 00:01 2024-05-09T06:25:06.593 INFO:teuthology.orchestra.run.smithi190.stdout:CentOS Stream 9 - BaseOS 5.5 MB/s | 8.1 MB 00:01 2024-05-09T06:25:09.487 INFO:teuthology.orchestra.run.smithi003.stdout:CentOS Stream 9 - AppStream 8.5 MB/s | 19 MB 00:02 2024-05-09T06:25:09.920 INFO:teuthology.orchestra.run.smithi190.stdout:CentOS Stream 9 - AppStream 8.5 MB/s | 19 MB 00:02 2024-05-09T06:25:13.846 INFO:teuthology.orchestra.run.smithi003.stdout:CentOS Stream 9 - CRB 9.2 MB/s | 6.2 MB 00:00 2024-05-09T06:25:15.037 INFO:teuthology.orchestra.run.smithi190.stdout:CentOS Stream 9 - CRB 4.1 MB/s | 6.2 MB 00:01 2024-05-09T06:25:15.305 INFO:teuthology.orchestra.run.smithi003.stdout:CentOS Stream 9 - Extras packages 68 kB/s | 16 kB 00:00 2024-05-09T06:25:16.590 INFO:teuthology.orchestra.run.smithi003.stdout:Extra Packages for Enterprise Linux 18 MB/s | 21 MB 00:01 2024-05-09T06:25:21.136 INFO:teuthology.orchestra.run.smithi190.stdout:CentOS Stream 9 - Extras packages 3.3 kB/s | 16 kB 00:04 2024-05-09T06:25:21.998 INFO:teuthology.orchestra.run.smithi003.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-05-09T06:25:23.623 INFO:teuthology.orchestra.run.smithi190.stdout:Extra Packages for Enterprise Linux 9.0 MB/s | 21 MB 00:02 2024-05-09T06:25:24.104 INFO:teuthology.orchestra.run.smithi003.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-05-09T06:25:24.104 INFO:teuthology.orchestra.run.smithi003.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-05-09T06:25:24.200 INFO:teuthology.orchestra.run.smithi003.stdout:Dependencies resolved. 2024-05-09T06:25:24.205 INFO:teuthology.orchestra.run.smithi003.stdout:================================================================================ 2024-05-09T06:25:24.206 INFO:teuthology.orchestra.run.smithi003.stdout: Package Architecture Version Repository Size 2024-05-09T06:25:24.206 INFO:teuthology.orchestra.run.smithi003.stdout:================================================================================ 2024-05-09T06:25:24.206 INFO:teuthology.orchestra.run.smithi003.stdout:Installing: 2024-05-09T06:25:24.206 INFO:teuthology.orchestra.run.smithi003.stdout: kernel x86_64 5.14.0-444.el9 baseos 484 k 2024-05-09T06:25:24.206 INFO:teuthology.orchestra.run.smithi003.stdout:Installing dependencies: 2024-05-09T06:25:24.206 INFO:teuthology.orchestra.run.smithi003.stdout: kernel-core x86_64 5.14.0-444.el9 baseos 15 M 2024-05-09T06:25:24.206 INFO:teuthology.orchestra.run.smithi003.stdout: kernel-modules x86_64 5.14.0-444.el9 baseos 34 M 2024-05-09T06:25:24.206 INFO:teuthology.orchestra.run.smithi003.stdout: kernel-modules-core x86_64 5.14.0-444.el9 baseos 28 M 2024-05-09T06:25:24.206 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-09T06:25:24.206 INFO:teuthology.orchestra.run.smithi003.stdout:Transaction Summary 2024-05-09T06:25:24.206 INFO:teuthology.orchestra.run.smithi003.stdout:================================================================================ 2024-05-09T06:25:24.206 INFO:teuthology.orchestra.run.smithi003.stdout:Install 4 Packages 2024-05-09T06:25:24.206 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-09T06:25:24.207 INFO:teuthology.orchestra.run.smithi003.stdout:Total download size: 78 M 2024-05-09T06:25:24.207 INFO:teuthology.orchestra.run.smithi003.stdout:Installed size: 123 M 2024-05-09T06:25:24.207 INFO:teuthology.orchestra.run.smithi003.stdout:Downloading Packages: 2024-05-09T06:25:24.507 INFO:teuthology.orchestra.run.smithi003.stdout:(1/4): kernel-5.14.0-444.el9.x86_64.rpm 2.0 MB/s | 484 kB 00:00 2024-05-09T06:25:25.141 INFO:teuthology.orchestra.run.smithi003.stdout:(2/4): kernel-core-5.14.0-444.el9.x86_64.rpm 18 MB/s | 15 MB 00:00 2024-05-09T06:25:25.566 INFO:teuthology.orchestra.run.smithi003.stdout:(3/4): kernel-modules-core-5.14.0-444.el9.x86_6 26 MB/s | 28 MB 00:01 2024-05-09T06:25:26.083 INFO:teuthology.orchestra.run.smithi003.stdout:(4/4): kernel-modules-5.14.0-444.el9.x86_64.rpm 19 MB/s | 34 MB 00:01 2024-05-09T06:25:26.083 INFO:teuthology.orchestra.run.smithi003.stdout:-------------------------------------------------------------------------------- 2024-05-09T06:25:26.083 INFO:teuthology.orchestra.run.smithi003.stdout:Total 41 MB/s | 78 MB 00:01 2024-05-09T06:25:26.791 INFO:teuthology.orchestra.run.smithi003.stdout:Running transaction check 2024-05-09T06:25:27.199 INFO:teuthology.orchestra.run.smithi003.stdout:Transaction check succeeded. 2024-05-09T06:25:27.199 INFO:teuthology.orchestra.run.smithi003.stdout:Running transaction test 2024-05-09T06:25:28.134 INFO:teuthology.orchestra.run.smithi003.stdout:Transaction test succeeded. 2024-05-09T06:25:28.134 INFO:teuthology.orchestra.run.smithi003.stdout:Running transaction 2024-05-09T06:25:28.829 INFO:teuthology.orchestra.run.smithi190.stdout:lab-extras 14 kB/s | 1.7 kB 00:00 2024-05-09T06:25:30.709 INFO:teuthology.orchestra.run.smithi003.stdout: Preparing : 1/1 2024-05-09T06:25:30.843 INFO:teuthology.orchestra.run.smithi190.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-05-09T06:25:30.844 INFO:teuthology.orchestra.run.smithi190.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-05-09T06:25:30.941 INFO:teuthology.orchestra.run.smithi190.stdout:Dependencies resolved. 2024-05-09T06:25:30.952 INFO:teuthology.orchestra.run.smithi190.stdout:================================================================================ 2024-05-09T06:25:30.952 INFO:teuthology.orchestra.run.smithi190.stdout: Package Architecture Version Repository Size 2024-05-09T06:25:30.952 INFO:teuthology.orchestra.run.smithi190.stdout:================================================================================ 2024-05-09T06:25:30.952 INFO:teuthology.orchestra.run.smithi190.stdout:Installing: 2024-05-09T06:25:30.952 INFO:teuthology.orchestra.run.smithi190.stdout: kernel x86_64 5.14.0-444.el9 baseos 484 k 2024-05-09T06:25:30.952 INFO:teuthology.orchestra.run.smithi190.stdout:Installing dependencies: 2024-05-09T06:25:30.952 INFO:teuthology.orchestra.run.smithi190.stdout: kernel-core x86_64 5.14.0-444.el9 baseos 15 M 2024-05-09T06:25:30.953 INFO:teuthology.orchestra.run.smithi190.stdout: kernel-modules x86_64 5.14.0-444.el9 baseos 34 M 2024-05-09T06:25:30.953 INFO:teuthology.orchestra.run.smithi190.stdout: kernel-modules-core x86_64 5.14.0-444.el9 baseos 28 M 2024-05-09T06:25:30.953 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-05-09T06:25:30.953 INFO:teuthology.orchestra.run.smithi190.stdout:Transaction Summary 2024-05-09T06:25:30.953 INFO:teuthology.orchestra.run.smithi190.stdout:================================================================================ 2024-05-09T06:25:30.953 INFO:teuthology.orchestra.run.smithi190.stdout:Install 4 Packages 2024-05-09T06:25:30.953 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-05-09T06:25:30.953 INFO:teuthology.orchestra.run.smithi190.stdout:Total download size: 78 M 2024-05-09T06:25:30.953 INFO:teuthology.orchestra.run.smithi190.stdout:Installed size: 123 M 2024-05-09T06:25:30.953 INFO:teuthology.orchestra.run.smithi190.stdout:Downloading Packages: 2024-05-09T06:25:31.365 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : kernel-modules-core-5.14.0-444.el9.x86_64 1/4 2024-05-09T06:25:31.450 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : kernel-core-5.14.0-444.el9.x86_64 2/4 2024-05-09T06:25:31.747 INFO:teuthology.orchestra.run.smithi190.stdout:(1/4): kernel-5.14.0-444.el9.x86_64.rpm 710 kB/s | 484 kB 00:00 2024-05-09T06:25:34.271 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: kernel-core-5.14.0-444.el9.x86_64 2/4 2024-05-09T06:25:34.394 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : kernel-modules-5.14.0-444.el9.x86_64 3/4 2024-05-09T06:25:38.755 INFO:teuthology.orchestra.run.smithi190.stdout:(2/4): kernel-modules-5.14.0-444.el9.x86_64.rpm 4.4 MB/s | 34 MB 00:07 2024-05-09T06:25:39.467 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: kernel-modules-5.14.0-444.el9.x86_64 3/4 2024-05-09T06:25:39.506 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : kernel-5.14.0-444.el9.x86_64 4/4 2024-05-09T06:25:44.606 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: kernel-modules-core-5.14.0-444.el9.x86_64 4/4 2024-05-09T06:25:47.854 INFO:teuthology.orchestra.run.smithi190.stdout:(3/4): kernel-core-5.14.0-444.el9.x86_64.rpm 941 kB/s | 15 MB 00:16 2024-05-09T06:26:05.769 INFO:teuthology.orchestra.run.smithi190.stdout:(4/4): kernel-modules-core-5.14.0-444.el9.x86_6 842 kB/s | 28 MB 00:34 2024-05-09T06:26:05.769 INFO:teuthology.orchestra.run.smithi190.stdout:-------------------------------------------------------------------------------- 2024-05-09T06:26:05.769 INFO:teuthology.orchestra.run.smithi190.stdout:Total 2.2 MB/s | 78 MB 00:34 2024-05-09T06:26:06.448 INFO:teuthology.orchestra.run.smithi190.stdout:Running transaction check 2024-05-09T06:26:06.788 INFO:teuthology.orchestra.run.smithi190.stdout:Transaction check succeeded. 2024-05-09T06:26:06.788 INFO:teuthology.orchestra.run.smithi190.stdout:Running transaction test 2024-05-09T06:26:07.720 INFO:teuthology.orchestra.run.smithi190.stdout:Transaction test succeeded. 2024-05-09T06:26:07.720 INFO:teuthology.orchestra.run.smithi190.stdout:Running transaction 2024-05-09T06:26:10.114 INFO:teuthology.orchestra.run.smithi190.stdout: Preparing : 1/1 2024-05-09T06:26:10.779 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : kernel-modules-core-5.14.0-444.el9.x86_64 1/4 2024-05-09T06:26:10.860 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : kernel-core-5.14.0-444.el9.x86_64 2/4 2024-05-09T06:26:13.543 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: kernel-core-5.14.0-444.el9.x86_64 2/4 2024-05-09T06:26:13.678 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : kernel-modules-5.14.0-444.el9.x86_64 3/4 2024-05-09T06:26:15.682 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: kernel-core-5.14.0-444.el9.x86_64 4/4 2024-05-09T06:26:15.723 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: kernel-modules-5.14.0-444.el9.x86_64 4/4 2024-05-09T06:26:16.019 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: kernel-5.14.0-444.el9.x86_64 4/4 2024-05-09T06:26:16.019 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : kernel-5.14.0-444.el9.x86_64 1/4 2024-05-09T06:26:16.019 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : kernel-core-5.14.0-444.el9.x86_64 2/4 2024-05-09T06:26:16.019 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : kernel-modules-5.14.0-444.el9.x86_64 3/4 2024-05-09T06:26:16.829 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : kernel-modules-core-5.14.0-444.el9.x86_64 4/4 2024-05-09T06:26:16.830 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-09T06:26:16.830 INFO:teuthology.orchestra.run.smithi003.stdout:Installed: 2024-05-09T06:26:16.830 INFO:teuthology.orchestra.run.smithi003.stdout: kernel-5.14.0-444.el9.x86_64 2024-05-09T06:26:16.830 INFO:teuthology.orchestra.run.smithi003.stdout: kernel-core-5.14.0-444.el9.x86_64 2024-05-09T06:26:16.830 INFO:teuthology.orchestra.run.smithi003.stdout: kernel-modules-5.14.0-444.el9.x86_64 2024-05-09T06:26:16.830 INFO:teuthology.orchestra.run.smithi003.stdout: kernel-modules-core-5.14.0-444.el9.x86_64 2024-05-09T06:26:16.830 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-09T06:26:16.830 INFO:teuthology.orchestra.run.smithi003.stdout:Complete! 2024-05-09T06:26:17.138 DEBUG:teuthology.orchestra.run.smithi003:> rpm -q kernel | sort -rV | head -n 1 2024-05-09T06:26:17.263 INFO:teuthology.orchestra.run.smithi003.stdout:kernel-5.14.0-444.el9.x86_64 2024-05-09T06:26:17.263 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-444.el9.x86_64 2024-05-09T06:26:17.263 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-437.el9.x86_64 Expected: 5.14.0-444.el9.x86_64 2024-05-09T06:26:17.263 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-05-09T06:26:17.264 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-05-09T06:26:17.264 INFO:teuthology.task.kernel:Installing distro kernel on client.0... 2024-05-09T06:26:17.264 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi003.front.sepia.ceph.com, path=None, version=distro) 2024-05-09T06:26:17.264 DEBUG:teuthology.orchestra.run.smithi003:> rpm -q kernel | sort -rV | head -n 1 2024-05-09T06:26:17.314 INFO:teuthology.orchestra.run.smithi003.stdout:kernel-5.14.0-444.el9.x86_64 2024-05-09T06:26:17.314 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-444.el9.x86_64 2024-05-09T06:26:17.315 DEBUG:teuthology.orchestra.run.smithi003:> sudo rpm -qi grub2-tools 2024-05-09T06:26:17.347 INFO:teuthology.orchestra.run.smithi003.stdout:Name : grub2-tools 2024-05-09T06:26:17.347 INFO:teuthology.orchestra.run.smithi003.stdout:Epoch : 1 2024-05-09T06:26:17.347 INFO:teuthology.orchestra.run.smithi003.stdout:Version : 2.06 2024-05-09T06:26:17.347 INFO:teuthology.orchestra.run.smithi003.stdout:Release : 68.el9 2024-05-09T06:26:17.347 INFO:teuthology.orchestra.run.smithi003.stdout:Architecture: x86_64 2024-05-09T06:26:17.347 INFO:teuthology.orchestra.run.smithi003.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-05-09T06:26:17.347 INFO:teuthology.orchestra.run.smithi003.stdout:Group : Unspecified 2024-05-09T06:26:17.347 INFO:teuthology.orchestra.run.smithi003.stdout:Size : 8264254 2024-05-09T06:26:17.347 INFO:teuthology.orchestra.run.smithi003.stdout:License : GPLv3+ 2024-05-09T06:26:17.347 INFO:teuthology.orchestra.run.smithi003.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-05-09T06:26:17.347 INFO:teuthology.orchestra.run.smithi003.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-05-09T06:26:17.347 INFO:teuthology.orchestra.run.smithi003.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-05-09T06:26:17.348 INFO:teuthology.orchestra.run.smithi003.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-05-09T06:26:17.348 INFO:teuthology.orchestra.run.smithi003.stdout:Packager : builder@centos.org 2024-05-09T06:26:17.348 INFO:teuthology.orchestra.run.smithi003.stdout:Vendor : CentOS 2024-05-09T06:26:17.348 INFO:teuthology.orchestra.run.smithi003.stdout:URL : http://www.gnu.org/software/grub/ 2024-05-09T06:26:17.348 INFO:teuthology.orchestra.run.smithi003.stdout:Summary : Support tools for GRUB. 2024-05-09T06:26:17.348 INFO:teuthology.orchestra.run.smithi003.stdout:Description : 2024-05-09T06:26:17.348 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-09T06:26:17.348 INFO:teuthology.orchestra.run.smithi003.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-05-09T06:26:17.348 INFO:teuthology.orchestra.run.smithi003.stdout:customizable bootloader with modular architecture. It supports a rich 2024-05-09T06:26:17.348 INFO:teuthology.orchestra.run.smithi003.stdout:variety of kernel formats, file systems, computer architectures and 2024-05-09T06:26:17.348 INFO:teuthology.orchestra.run.smithi003.stdout:hardware devices. 2024-05-09T06:26:17.348 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-09T06:26:17.349 INFO:teuthology.orchestra.run.smithi003.stdout:This subpackage provides tools for support of all platforms. 2024-05-09T06:26:17.350 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-05-09T06:26:17.350 INFO:teuthology.task.kernel:Updating grub on smithi003 to boot 5.14.0-444.el9.x86_64 2024-05-09T06:26:17.350 DEBUG:teuthology.orchestra.run.smithi003:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-05-09T06:26:17.990 INFO:teuthology.orchestra.run.smithi003.stderr:Generating grub configuration file ... 2024-05-09T06:26:18.411 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: kernel-modules-5.14.0-444.el9.x86_64 3/4 2024-05-09T06:26:18.514 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : kernel-5.14.0-444.el9.x86_64 4/4 2024-05-09T06:26:19.383 INFO:teuthology.orchestra.run.smithi003.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-05-09T06:26:19.411 INFO:teuthology.orchestra.run.smithi003.stderr:done 2024-05-09T06:26:19.413 DEBUG:teuthology.orchestra.run.smithi003:> mktemp 2024-05-09T06:26:19.433 INFO:teuthology.orchestra.run.smithi003.stdout:/tmp/tmp.5Qemo2KcC6 2024-05-09T06:26:19.433 DEBUG:teuthology.orchestra.run.smithi003:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.5Qemo2KcC6 2024-05-09T06:26:19.501 DEBUG:teuthology.orchestra.run.smithi003:> sudo chmod 0666 /tmp/tmp.5Qemo2KcC6 2024-05-09T06:26:19.637 DEBUG:teuthology.orchestra.remote:smithi003:/tmp/tmp.5Qemo2KcC6 is 6KB 2024-05-09T06:26:19.686 DEBUG:teuthology.orchestra.run.smithi003:> rm -fr /tmp/tmp.5Qemo2KcC6 2024-05-09T06:26:19.700 DEBUG:teuthology.orchestra.run.smithi003:> sudo /bin/ls /boot/loader/entries || true 2024-05-09T06:26:19.762 INFO:teuthology.orchestra.run.smithi003.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-05-09T06:26:19.762 INFO:teuthology.orchestra.run.smithi003.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-05-09T06:26:19.762 INFO:teuthology.orchestra.run.smithi003.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-05-09T06:26:19.762 INFO:teuthology.orchestra.run.smithi003.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-444.el9.x86_64.conf 2024-05-09T06:26:19.764 DEBUG:teuthology.orchestra.run.smithi003:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-5.14.0-444.el9.x86_64 2024-05-09T06:26:19.909 DEBUG:teuthology.orchestra.run.smithi003:> sudo shutdown -r now 2024-05-09T06:26:23.244 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: kernel-modules-core-5.14.0-444.el9.x86_64 4/4 2024-05-09T06:26:49.911 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-05-09T06:26:49.912 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi003.front.sepia.ceph.com' 2024-05-09T06:26:49.913 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi003.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-09T06:26:52.352 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: kernel-core-5.14.0-444.el9.x86_64 4/4 2024-05-09T06:26:52.390 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: kernel-modules-5.14.0-444.el9.x86_64 4/4 2024-05-09T06:26:52.655 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: kernel-5.14.0-444.el9.x86_64 4/4 2024-05-09T06:26:52.655 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : kernel-5.14.0-444.el9.x86_64 1/4 2024-05-09T06:26:52.655 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : kernel-core-5.14.0-444.el9.x86_64 2/4 2024-05-09T06:26:52.655 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : kernel-modules-5.14.0-444.el9.x86_64 3/4 2024-05-09T06:26:53.471 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : kernel-modules-core-5.14.0-444.el9.x86_64 4/4 2024-05-09T06:26:53.471 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-05-09T06:26:53.471 INFO:teuthology.orchestra.run.smithi190.stdout:Installed: 2024-05-09T06:26:53.471 INFO:teuthology.orchestra.run.smithi190.stdout: kernel-5.14.0-444.el9.x86_64 2024-05-09T06:26:53.471 INFO:teuthology.orchestra.run.smithi190.stdout: kernel-core-5.14.0-444.el9.x86_64 2024-05-09T06:26:53.471 INFO:teuthology.orchestra.run.smithi190.stdout: kernel-modules-5.14.0-444.el9.x86_64 2024-05-09T06:26:53.472 INFO:teuthology.orchestra.run.smithi190.stdout: kernel-modules-core-5.14.0-444.el9.x86_64 2024-05-09T06:26:53.472 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-05-09T06:26:53.472 INFO:teuthology.orchestra.run.smithi190.stdout:Complete! 2024-05-09T06:26:53.836 DEBUG:teuthology.orchestra.run.smithi190:> rpm -q kernel | sort -rV | head -n 1 2024-05-09T06:26:53.917 INFO:teuthology.orchestra.run.smithi190.stdout:kernel-5.14.0-444.el9.x86_64 2024-05-09T06:26:53.918 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-444.el9.x86_64 2024-05-09T06:26:53.918 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-437.el9.x86_64 Expected: 5.14.0-444.el9.x86_64 2024-05-09T06:26:53.918 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-05-09T06:26:53.918 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-05-09T06:26:53.918 INFO:teuthology.task.kernel:Installing distro kernel on client.1... 2024-05-09T06:26:53.918 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi190.front.sepia.ceph.com, path=None, version=distro) 2024-05-09T06:26:53.918 DEBUG:teuthology.orchestra.run.smithi190:> rpm -q kernel | sort -rV | head -n 1 2024-05-09T06:26:53.968 INFO:teuthology.orchestra.run.smithi190.stdout:kernel-5.14.0-444.el9.x86_64 2024-05-09T06:26:53.968 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-444.el9.x86_64 2024-05-09T06:26:53.968 DEBUG:teuthology.orchestra.run.smithi190:> sudo rpm -qi grub2-tools 2024-05-09T06:26:53.998 INFO:teuthology.orchestra.run.smithi190.stdout:Name : grub2-tools 2024-05-09T06:26:53.999 INFO:teuthology.orchestra.run.smithi190.stdout:Epoch : 1 2024-05-09T06:26:53.999 INFO:teuthology.orchestra.run.smithi190.stdout:Version : 2.06 2024-05-09T06:26:53.999 INFO:teuthology.orchestra.run.smithi190.stdout:Release : 68.el9 2024-05-09T06:26:53.999 INFO:teuthology.orchestra.run.smithi190.stdout:Architecture: x86_64 2024-05-09T06:26:53.999 INFO:teuthology.orchestra.run.smithi190.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-05-09T06:26:53.999 INFO:teuthology.orchestra.run.smithi190.stdout:Group : Unspecified 2024-05-09T06:26:53.999 INFO:teuthology.orchestra.run.smithi190.stdout:Size : 8264254 2024-05-09T06:26:53.999 INFO:teuthology.orchestra.run.smithi190.stdout:License : GPLv3+ 2024-05-09T06:26:53.999 INFO:teuthology.orchestra.run.smithi190.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-05-09T06:26:53.999 INFO:teuthology.orchestra.run.smithi190.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-05-09T06:26:53.999 INFO:teuthology.orchestra.run.smithi190.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-05-09T06:26:53.999 INFO:teuthology.orchestra.run.smithi190.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-05-09T06:26:53.999 INFO:teuthology.orchestra.run.smithi190.stdout:Packager : builder@centos.org 2024-05-09T06:26:53.999 INFO:teuthology.orchestra.run.smithi190.stdout:Vendor : CentOS 2024-05-09T06:26:53.999 INFO:teuthology.orchestra.run.smithi190.stdout:URL : http://www.gnu.org/software/grub/ 2024-05-09T06:26:54.000 INFO:teuthology.orchestra.run.smithi190.stdout:Summary : Support tools for GRUB. 2024-05-09T06:26:54.000 INFO:teuthology.orchestra.run.smithi190.stdout:Description : 2024-05-09T06:26:54.000 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-05-09T06:26:54.000 INFO:teuthology.orchestra.run.smithi190.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-05-09T06:26:54.000 INFO:teuthology.orchestra.run.smithi190.stdout:customizable bootloader with modular architecture. It supports a rich 2024-05-09T06:26:54.000 INFO:teuthology.orchestra.run.smithi190.stdout:variety of kernel formats, file systems, computer architectures and 2024-05-09T06:26:54.000 INFO:teuthology.orchestra.run.smithi190.stdout:hardware devices. 2024-05-09T06:26:54.000 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-05-09T06:26:54.000 INFO:teuthology.orchestra.run.smithi190.stdout:This subpackage provides tools for support of all platforms. 2024-05-09T06:26:54.001 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-05-09T06:26:54.001 INFO:teuthology.task.kernel:Updating grub on smithi190 to boot 5.14.0-444.el9.x86_64 2024-05-09T06:26:54.001 DEBUG:teuthology.orchestra.run.smithi190:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-05-09T06:26:54.552 INFO:teuthology.orchestra.run.smithi190.stderr:Generating grub configuration file ... 2024-05-09T06:26:55.955 INFO:teuthology.orchestra.run.smithi190.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-05-09T06:26:55.987 INFO:teuthology.orchestra.run.smithi190.stderr:done 2024-05-09T06:26:55.997 DEBUG:teuthology.orchestra.run.smithi190:> mktemp 2024-05-09T06:26:56.013 INFO:teuthology.orchestra.run.smithi190.stdout:/tmp/tmp.IUXsVotmMW 2024-05-09T06:26:56.013 DEBUG:teuthology.orchestra.run.smithi190:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.IUXsVotmMW 2024-05-09T06:26:56.078 DEBUG:teuthology.orchestra.run.smithi190:> sudo chmod 0666 /tmp/tmp.IUXsVotmMW 2024-05-09T06:26:56.217 DEBUG:teuthology.orchestra.remote:smithi190:/tmp/tmp.IUXsVotmMW is 6KB 2024-05-09T06:26:56.264 DEBUG:teuthology.orchestra.run.smithi190:> rm -fr /tmp/tmp.IUXsVotmMW 2024-05-09T06:26:56.278 DEBUG:teuthology.orchestra.run.smithi190:> sudo /bin/ls /boot/loader/entries || true 2024-05-09T06:26:56.344 INFO:teuthology.orchestra.run.smithi190.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-05-09T06:26:56.344 INFO:teuthology.orchestra.run.smithi190.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-05-09T06:26:56.344 INFO:teuthology.orchestra.run.smithi190.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-05-09T06:26:56.345 INFO:teuthology.orchestra.run.smithi190.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-444.el9.x86_64.conf 2024-05-09T06:26:56.346 DEBUG:teuthology.orchestra.run.smithi190:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-5.14.0-444.el9.x86_64 2024-05-09T06:26:56.492 DEBUG:teuthology.orchestra.run.smithi190:> sudo shutdown -r now 2024-05-09T06:27:08.446 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.3 2024-05-09T06:27:17.454 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi003.front.sepia.ceph.com' 2024-05-09T06:27:17.455 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi003.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-09T06:27:26.498 INFO:teuthology.task.kernel:Checking client client.1 for new kernel version... 2024-05-09T06:27:26.499 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi190.front.sepia.ceph.com' 2024-05-09T06:27:26.499 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi190.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-09T06:27:45.054 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.190 2024-05-09T06:27:54.062 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi190.front.sepia.ceph.com' 2024-05-09T06:27:54.063 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi190.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-09T06:28:17.465 DEBUG:teuthology.orchestra.remote:timed out 2024-05-09T06:28:29.476 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi003.front.sepia.ceph.com' 2024-05-09T06:28:29.477 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi003.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-09T06:28:29.826 DEBUG:teuthology.orchestra.run.smithi003:> true 2024-05-09T06:28:30.193 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi003.front.sepia.ceph.com' 2024-05-09T06:28:30.194 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "5.14.0-444.el9.x86_64"... 2024-05-09T06:28:30.194 DEBUG:teuthology.orchestra.run.smithi003:> uname -r 2024-05-09T06:28:30.211 INFO:teuthology.orchestra.run.smithi003.stdout:5.14.0-444.el9.x86_64 2024-05-09T06:28:30.211 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-444.el9.x86_64 vs 5.14.0-444.el9.x86_64 2024-05-09T06:28:30.211 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-05-09T06:28:30.211 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-05-09T06:28:31.212 DEBUG:teuthology.parallel:result is None 2024-05-09T06:28:54.065 DEBUG:teuthology.orchestra.remote:timed out 2024-05-09T06:29:06.065 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi190.front.sepia.ceph.com' 2024-05-09T06:29:06.066 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi190.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-09T06:29:06.450 DEBUG:teuthology.orchestra.run.smithi190:> true 2024-05-09T06:29:06.836 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi190.front.sepia.ceph.com' 2024-05-09T06:29:06.836 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "5.14.0-444.el9.x86_64"... 2024-05-09T06:29:06.837 DEBUG:teuthology.orchestra.run.smithi190:> uname -r 2024-05-09T06:29:06.855 INFO:teuthology.orchestra.run.smithi190.stdout:5.14.0-444.el9.x86_64 2024-05-09T06:29:06.855 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-444.el9.x86_64 vs 5.14.0-444.el9.x86_64 2024-05-09T06:29:06.855 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-05-09T06:29:06.855 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-05-09T06:29:07.857 DEBUG:teuthology.parallel:result is None 2024-05-09T06:29:07.857 INFO:teuthology.run_tasks:Running task internal.base... 2024-05-09T06:29:07.865 INFO:teuthology.task.internal:Creating test directory... 2024-05-09T06:29:07.865 DEBUG:teuthology.orchestra.run.smithi003:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-09T06:29:07.868 DEBUG:teuthology.orchestra.run.smithi190:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-09T06:29:07.886 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-05-09T06:29:07.893 INFO:teuthology.run_tasks:Running task internal.archive... 2024-05-09T06:29:07.901 INFO:teuthology.task.internal:Creating archive directory... 2024-05-09T06:29:07.901 DEBUG:teuthology.orchestra.run.smithi003:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-09T06:29:07.928 DEBUG:teuthology.orchestra.run.smithi190:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-09T06:29:07.966 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-05-09T06:29:07.973 INFO:teuthology.task.internal:Enabling coredump saving... 2024-05-09T06:29:07.973 DEBUG:teuthology.orchestra.run.smithi003:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-09T06:29:07.998 DEBUG:teuthology.orchestra.run.smithi190:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-09T06:29:08.072 INFO:teuthology.orchestra.run.smithi003.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-09T06:29:08.082 INFO:teuthology.orchestra.run.smithi190.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-09T06:29:08.116 INFO:teuthology.orchestra.run.smithi190.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-09T06:29:08.121 INFO:teuthology.orchestra.run.smithi003.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-09T06:29:08.124 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-05-09T06:29:08.131 INFO:teuthology.task.internal:Configuring sudo... 2024-05-09T06:29:08.131 DEBUG:teuthology.orchestra.run.smithi003:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-09T06:29:08.134 DEBUG:teuthology.orchestra.run.smithi190:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-09T06:29:08.161 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-05-09T06:29:08.169 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-05-09T06:29:08.170 DEBUG:teuthology.orchestra.run.smithi003:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-09T06:29:08.202 DEBUG:teuthology.orchestra.run.smithi190:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-09T06:29:08.219 DEBUG:teuthology.orchestra.run.smithi003:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-09T06:29:08.301 DEBUG:teuthology.orchestra.run.smithi003:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-09T06:29:08.346 DEBUG:teuthology.orchestra.run.smithi003:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-09T06:29:08.432 DEBUG:teuthology.orchestra.run.smithi003:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-09T06:29:08.473 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-05-09T06:29:08.473 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-09T06:29:08.539 DEBUG:teuthology.orchestra.run.smithi190:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-09T06:29:08.584 DEBUG:teuthology.orchestra.run.smithi190:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-09T06:29:08.635 DEBUG:teuthology.orchestra.run.smithi190:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-09T06:29:08.720 DEBUG:teuthology.orchestra.run.smithi190:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-09T06:29:08.763 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-05-09T06:29:08.763 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-09T06:29:08.830 DEBUG:teuthology.orchestra.run.smithi003:> sudo service rsyslog restart 2024-05-09T06:29:08.833 DEBUG:teuthology.orchestra.run.smithi190:> sudo service rsyslog restart 2024-05-09T06:29:08.875 INFO:teuthology.orchestra.run.smithi003.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-09T06:29:08.917 INFO:teuthology.orchestra.run.smithi190.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-09T06:29:09.399 INFO:teuthology.run_tasks:Running task internal.timer... 2024-05-09T06:29:09.407 INFO:teuthology.task.internal:Starting timer... 2024-05-09T06:29:09.407 INFO:teuthology.run_tasks:Running task pcp... 2024-05-09T06:29:09.422 INFO:teuthology.run_tasks:Running task selinux... 2024-05-09T06:29:09.430 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:getty_t:s0']} 2024-05-09T06:29:09.431 DEBUG:teuthology.orchestra.run.smithi003:> sudo service auditd rotate 2024-05-09T06:29:09.492 INFO:teuthology.orchestra.run.smithi003.stdout:Rotating logs: 2024-05-09T06:29:09.494 DEBUG:teuthology.orchestra.run.smithi190:> sudo service auditd rotate 2024-05-09T06:29:09.561 INFO:teuthology.orchestra.run.smithi190.stdout:Rotating logs: 2024-05-09T06:29:09.563 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-05-09T06:29:09.563 DEBUG:teuthology.orchestra.run.smithi003:> /usr/sbin/getenforce 2024-05-09T06:29:09.586 INFO:teuthology.orchestra.run.smithi003.stdout:Permissive 2024-05-09T06:29:09.586 DEBUG:teuthology.orchestra.run.smithi190:> /usr/sbin/getenforce 2024-05-09T06:29:09.610 INFO:teuthology.orchestra.run.smithi190.stdout:Permissive 2024-05-09T06:29:09.610 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi003.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi190.front.sepia.ceph.com': 'permissive'} 2024-05-09T06:29:09.611 DEBUG:teuthology.orchestra.run.smithi003:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-09T06:29:09.654 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T06:29:09.654 DEBUG:teuthology.orchestra.run.smithi190:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-09T06:29:09.682 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T06:29:09.683 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-05-09T06:29:09.683 DEBUG:teuthology.orchestra.run.smithi003:> sudo /usr/sbin/setenforce permissive 2024-05-09T06:29:09.721 DEBUG:teuthology.orchestra.run.smithi190:> sudo /usr/sbin/setenforce permissive 2024-05-09T06:29:09.752 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-05-09T06:29:09.761 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-05-09T06:29:09.767 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-05-09T06:29:09.898 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-05-09T06:29:09.913 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-05-09T06:29:09.914 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi003.front.sepia.ceph.com,smithi190.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-05-09T06:33:12.806 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi003.front.sepia.ceph.com'), Remote(name='ubuntu@smithi190.front.sepia.ceph.com')] 2024-05-09T06:33:12.808 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi003.front.sepia.ceph.com' 2024-05-09T06:33:12.809 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi003.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-09T06:33:12.887 DEBUG:teuthology.orchestra.run.smithi003:> true 2024-05-09T06:33:12.970 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi003.front.sepia.ceph.com' 2024-05-09T06:33:12.971 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi190.front.sepia.ceph.com' 2024-05-09T06:33:12.971 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi190.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-09T06:33:13.046 DEBUG:teuthology.orchestra.run.smithi190:> true 2024-05-09T06:33:13.125 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi190.front.sepia.ceph.com' 2024-05-09T06:33:13.126 INFO:teuthology.run_tasks:Running task clock... 2024-05-09T06:33:13.136 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-05-09T06:33:13.136 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-09T06:33:13.136 DEBUG:teuthology.orchestra.run.smithi003:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-09T06:33:13.138 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-09T06:33:13.139 DEBUG:teuthology.orchestra.run.smithi190:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-09T06:33:13.167 INFO:teuthology.orchestra.run.smithi003.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-09T06:33:13.183 INFO:teuthology.orchestra.run.smithi003.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-09T06:33:13.195 INFO:teuthology.orchestra.run.smithi190.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-09T06:33:13.209 INFO:teuthology.orchestra.run.smithi003.stderr:sudo: ntpd: command not found 2024-05-09T06:33:13.209 INFO:teuthology.orchestra.run.smithi190.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-09T06:33:13.220 INFO:teuthology.orchestra.run.smithi003.stdout:506 Cannot talk to daemon 2024-05-09T06:33:13.233 INFO:teuthology.orchestra.run.smithi190.stderr:sudo: ntpd: command not found 2024-05-09T06:33:13.235 INFO:teuthology.orchestra.run.smithi003.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-09T06:33:13.244 INFO:teuthology.orchestra.run.smithi190.stdout:506 Cannot talk to daemon 2024-05-09T06:33:13.250 INFO:teuthology.orchestra.run.smithi003.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-09T06:33:13.258 INFO:teuthology.orchestra.run.smithi190.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-09T06:33:13.271 INFO:teuthology.orchestra.run.smithi190.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-09T06:33:13.306 INFO:teuthology.orchestra.run.smithi003.stderr:bash: line 1: ntpq: command not found 2024-05-09T06:33:13.311 INFO:teuthology.orchestra.run.smithi003.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-09T06:33:13.311 INFO:teuthology.orchestra.run.smithi003.stdout:=============================================================================== 2024-05-09T06:33:13.311 INFO:teuthology.orchestra.run.smithi003.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-09T06:33:13.311 INFO:teuthology.orchestra.run.smithi003.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-09T06:33:13.311 INFO:teuthology.orchestra.run.smithi003.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-09T06:33:13.311 INFO:teuthology.orchestra.run.smithi003.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-09T06:33:13.325 INFO:teuthology.orchestra.run.smithi190.stderr:bash: line 1: ntpq: command not found 2024-05-09T06:33:13.329 INFO:teuthology.orchestra.run.smithi190.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-09T06:33:13.329 INFO:teuthology.orchestra.run.smithi190.stdout:=============================================================================== 2024-05-09T06:33:13.329 INFO:teuthology.orchestra.run.smithi190.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-09T06:33:13.329 INFO:teuthology.orchestra.run.smithi190.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-09T06:33:13.329 INFO:teuthology.orchestra.run.smithi190.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-09T06:33:13.330 INFO:teuthology.orchestra.run.smithi190.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-09T06:33:13.330 INFO:teuthology.run_tasks:Running task install... 2024-05-09T06:33:13.338 DEBUG:teuthology.task.install:project ceph 2024-05-09T06:33:13.338 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '96805dcf909541c4ca9a1e90258de8ae5b6fa1bc'}} 2024-05-09T06:33:13.338 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '96805dcf909541c4ca9a1e90258de8ae5b6fa1bc'} 2024-05-09T06:33:13.338 INFO:teuthology.task.install:Using flavor: default 2024-05-09T06:33:13.344 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-05-09T06:33:13.344 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-05-09T06:33:13.344 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '96805dcf909541c4ca9a1e90258de8ae5b6fa1bc', 'tag': None, 'wait_for_package': False} 2024-05-09T06:33:13.344 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=96805dcf909541c4ca9a1e90258de8ae5b6fa1bc 2024-05-09T06:33:13.346 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '96805dcf909541c4ca9a1e90258de8ae5b6fa1bc', 'tag': None, 'wait_for_package': False} 2024-05-09T06:33:13.347 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=96805dcf909541c4ca9a1e90258de8ae5b6fa1bc 2024-05-09T06:33:13.561 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/squid/96805dcf909541c4ca9a1e90258de8ae5b6fa1bc/centos/9/flavors/default/ 2024-05-09T06:33:13.562 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2515.g96805dcf 2024-05-09T06:33:13.580 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/squid/96805dcf909541c4ca9a1e90258de8ae5b6fa1bc/centos/9/flavors/default/ 2024-05-09T06:33:13.580 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2515.g96805dcf 2024-05-09T06:33:13.670 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/squid/96805dcf909541c4ca9a1e90258de8ae5b6fa1bc/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/96805dcf909541c4ca9a1e90258de8ae5b6fa1bc/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/96805dcf909541c4ca9a1e90258de8ae5b6fa1bc/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-09T06:33:13.671 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-05-09T06:33:13.671 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-09T06:33:13.685 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/squid/96805dcf909541c4ca9a1e90258de8ae5b6fa1bc/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/96805dcf909541c4ca9a1e90258de8ae5b6fa1bc/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/96805dcf909541c4ca9a1e90258de8ae5b6fa1bc/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-09T06:33:13.685 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-05-09T06:33:13.685 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-09T06:33:13.698 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-05-09T06:33:13.698 DEBUG:teuthology.orchestra.run.smithi003:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/96805dcf909541c4ca9a1e90258de8ae5b6fa1bc/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-09T06:33:13.711 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-05-09T06:33:13.711 DEBUG:teuthology.orchestra.run.smithi190:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/96805dcf909541c4ca9a1e90258de8ae5b6fa1bc/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-09T06:33:13.765 DEBUG:teuthology.orchestra.run.smithi003:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-09T06:33:13.777 DEBUG:teuthology.orchestra.run.smithi190:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-09T06:33:13.841 DEBUG:teuthology.orchestra.run.smithi003:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-09T06:33:13.852 DEBUG:teuthology.orchestra.run.smithi190:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-09T06:33:13.906 INFO:teuthology.orchestra.run.smithi003.stdout:check_obsoletes = 1 2024-05-09T06:33:13.908 DEBUG:teuthology.orchestra.run.smithi003:> sudo yum clean all 2024-05-09T06:33:13.915 INFO:teuthology.orchestra.run.smithi190.stdout:check_obsoletes = 1 2024-05-09T06:33:13.917 DEBUG:teuthology.orchestra.run.smithi190:> sudo yum clean all 2024-05-09T06:33:14.144 INFO:teuthology.orchestra.run.smithi003.stdout:54 files removed 2024-05-09T06:33:14.157 INFO:teuthology.orchestra.run.smithi190.stdout:54 files removed 2024-05-09T06:33:14.171 DEBUG:teuthology.orchestra.run.smithi003:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-05-09T06:33:14.184 DEBUG:teuthology.orchestra.run.smithi190:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-05-09T06:33:14.882 INFO:teuthology.orchestra.run.smithi003.stdout:ceph packages for x86_64 197 kB/s | 83 kB 00:00 2024-05-09T06:33:15.017 INFO:teuthology.orchestra.run.smithi190.stdout:ceph packages for x86_64 148 kB/s | 83 kB 00:00 2024-05-09T06:33:15.433 INFO:teuthology.orchestra.run.smithi003.stdout:ceph noarch packages 23 kB/s | 11 kB 00:00 2024-05-09T06:33:15.618 INFO:teuthology.orchestra.run.smithi190.stdout:ceph noarch packages 21 kB/s | 11 kB 00:00 2024-05-09T06:33:15.883 INFO:teuthology.orchestra.run.smithi003.stdout:ceph source packages 4.7 kB/s | 1.9 kB 00:00 2024-05-09T06:33:16.069 INFO:teuthology.orchestra.run.smithi190.stdout:ceph source packages 4.6 kB/s | 1.9 kB 00:00 2024-05-09T06:33:18.100 INFO:teuthology.orchestra.run.smithi003.stdout:CentOS Stream 9 - BaseOS 3.7 MB/s | 8.1 MB 00:02 2024-05-09T06:33:22.801 INFO:teuthology.orchestra.run.smithi003.stdout:CentOS Stream 9 - AppStream 5.4 MB/s | 19 MB 00:03 2024-05-09T06:33:25.211 INFO:teuthology.orchestra.run.smithi190.stdout:CentOS Stream 9 - BaseOS 907 kB/s | 8.1 MB 00:09 2024-05-09T06:33:27.127 INFO:teuthology.orchestra.run.smithi003.stdout:CentOS Stream 9 - CRB 10 MB/s | 6.2 MB 00:00 2024-05-09T06:33:28.611 INFO:teuthology.orchestra.run.smithi003.stdout:CentOS Stream 9 - Extras packages 64 kB/s | 16 kB 00:00 2024-05-09T06:33:29.807 INFO:teuthology.orchestra.run.smithi003.stdout:Extra Packages for Enterprise Linux 20 MB/s | 21 MB 00:01 2024-05-09T06:33:30.078 INFO:teuthology.orchestra.run.smithi190.stdout:CentOS Stream 9 - AppStream 5.1 MB/s | 19 MB 00:03 2024-05-09T06:33:35.204 INFO:teuthology.orchestra.run.smithi003.stdout:lab-extras 33 kB/s | 1.7 kB 00:00 2024-05-09T06:33:36.637 INFO:teuthology.orchestra.run.smithi190.stdout:CentOS Stream 9 - CRB 2.1 MB/s | 6.2 MB 00:02 2024-05-09T06:33:37.648 INFO:teuthology.orchestra.run.smithi003.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-09T06:33:37.649 INFO:teuthology.orchestra.run.smithi003.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-09T06:33:37.661 INFO:teuthology.orchestra.run.smithi003.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-05-09T06:33:37.661 INFO:teuthology.orchestra.run.smithi003.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-05-09T06:33:37.662 INFO:teuthology.orchestra.run.smithi003.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-05-09T06:33:37.663 INFO:teuthology.orchestra.run.smithi003.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-05-09T06:33:37.665 INFO:teuthology.orchestra.run.smithi003.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-05-09T06:33:37.667 INFO:teuthology.orchestra.run.smithi003.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-05-09T06:33:37.668 INFO:teuthology.orchestra.run.smithi003.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-05-09T06:33:37.734 INFO:teuthology.orchestra.run.smithi003.stdout:Dependencies resolved. 2024-05-09T06:33:37.747 INFO:teuthology.orchestra.run.smithi003.stdout:======================================================================================= 2024-05-09T06:33:37.747 INFO:teuthology.orchestra.run.smithi003.stdout: Package Arch Version Repository Size 2024-05-09T06:33:37.747 INFO:teuthology.orchestra.run.smithi003.stdout:======================================================================================= 2024-05-09T06:33:37.747 INFO:teuthology.orchestra.run.smithi003.stdout:Installing: 2024-05-09T06:33:37.747 INFO:teuthology.orchestra.run.smithi003.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-05-09T06:33:37.747 INFO:teuthology.orchestra.run.smithi003.stdout: ceph x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 6.6 k 2024-05-09T06:33:37.747 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-base x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 5.4 M 2024-05-09T06:33:37.747 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-fuse x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 889 k 2024-05-09T06:33:37.747 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 145 k 2024-05-09T06:33:37.748 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mgr x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 1.5 M 2024-05-09T06:33:37.748 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 142 k 2024-05-09T06:33:37.748 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 3.5 M 2024-05-09T06:33:37.748 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 7.4 M 2024-05-09T06:33:37.748 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mgr-rook noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 49 k 2024-05-09T06:33:37.748 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-radosgw x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 10 M 2024-05-09T06:33:37.748 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-test x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 48 M 2024-05-09T06:33:37.748 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-volume noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 264 k 2024-05-09T06:33:37.748 INFO:teuthology.orchestra.run.smithi003.stdout: cephadm noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 763 k 2024-05-09T06:33:37.748 INFO:teuthology.orchestra.run.smithi003.stdout: cephfs-mirror x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 222 k 2024-05-09T06:33:37.748 INFO:teuthology.orchestra.run.smithi003.stdout: cephfs-shell x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 27 k 2024-05-09T06:33:37.748 INFO:teuthology.orchestra.run.smithi003.stdout: cephfs-top noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 18 k 2024-05-09T06:33:37.748 INFO:teuthology.orchestra.run.smithi003.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-05-09T06:33:37.749 INFO:teuthology.orchestra.run.smithi003.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-05-09T06:33:37.749 INFO:teuthology.orchestra.run.smithi003.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-05-09T06:33:37.749 INFO:teuthology.orchestra.run.smithi003.stdout: libcephfs-devel x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 32 k 2024-05-09T06:33:37.749 INFO:teuthology.orchestra.run.smithi003.stdout: libcephfs2 x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 746 k 2024-05-09T06:33:37.749 INFO:teuthology.orchestra.run.smithi003.stdout: librados-devel x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 127 k 2024-05-09T06:33:37.749 INFO:teuthology.orchestra.run.smithi003.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-05-09T06:33:37.749 INFO:teuthology.orchestra.run.smithi003.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-05-09T06:33:37.749 INFO:teuthology.orchestra.run.smithi003.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-05-09T06:33:37.749 INFO:teuthology.orchestra.run.smithi003.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-05-09T06:33:37.749 INFO:teuthology.orchestra.run.smithi003.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-05-09T06:33:37.749 INFO:teuthology.orchestra.run.smithi003.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-05-09T06:33:37.749 INFO:teuthology.orchestra.run.smithi003.stdout: python3-cephfs x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 162 k 2024-05-09T06:33:37.749 INFO:teuthology.orchestra.run.smithi003.stdout: python3-rados x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 321 k 2024-05-09T06:33:37.750 INFO:teuthology.orchestra.run.smithi003.stdout: python3-rbd x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 298 k 2024-05-09T06:33:37.750 INFO:teuthology.orchestra.run.smithi003.stdout: python3-rgw x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 99 k 2024-05-09T06:33:37.750 INFO:teuthology.orchestra.run.smithi003.stdout: rbd-fuse x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 85 k 2024-05-09T06:33:37.750 INFO:teuthology.orchestra.run.smithi003.stdout: rbd-mirror x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 3.1 M 2024-05-09T06:33:37.750 INFO:teuthology.orchestra.run.smithi003.stdout: rbd-nbd x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 171 k 2024-05-09T06:33:37.750 INFO:teuthology.orchestra.run.smithi003.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-05-09T06:33:37.750 INFO:teuthology.orchestra.run.smithi003.stdout:Upgrading: 2024-05-09T06:33:37.750 INFO:teuthology.orchestra.run.smithi003.stdout: NetworkManager x86_64 1:1.47.5-1.el9 baseos 2.3 M 2024-05-09T06:33:37.750 INFO:teuthology.orchestra.run.smithi003.stdout: NetworkManager-libnm x86_64 1:1.47.5-1.el9 baseos 1.8 M 2024-05-09T06:33:37.750 INFO:teuthology.orchestra.run.smithi003.stdout: NetworkManager-team x86_64 1:1.47.5-1.el9 baseos 42 k 2024-05-09T06:33:37.750 INFO:teuthology.orchestra.run.smithi003.stdout: NetworkManager-tui x86_64 1:1.47.5-1.el9 baseos 248 k 2024-05-09T06:33:37.750 INFO:teuthology.orchestra.run.smithi003.stdout: librados2 x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 3.4 M 2024-05-09T06:33:37.750 INFO:teuthology.orchestra.run.smithi003.stdout: librbd1 x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 3.1 M 2024-05-09T06:33:37.750 INFO:teuthology.orchestra.run.smithi003.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-05-09T06:33:37.751 INFO:teuthology.orchestra.run.smithi003.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-05-09T06:33:37.751 INFO:teuthology.orchestra.run.smithi003.stdout:Installing dependencies: 2024-05-09T06:33:37.751 INFO:teuthology.orchestra.run.smithi003.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-05-09T06:33:37.751 INFO:teuthology.orchestra.run.smithi003.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-05-09T06:33:37.751 INFO:teuthology.orchestra.run.smithi003.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-05-09T06:33:37.751 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-common x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 21 M 2024-05-09T06:33:37.751 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 32 k 2024-05-09T06:33:37.751 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mds x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 2.4 M 2024-05-09T06:33:37.751 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 249 k 2024-05-09T06:33:37.751 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mon x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 4.7 M 2024-05-09T06:33:37.751 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-osd x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 17 M 2024-05-09T06:33:37.751 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 16 k 2024-05-09T06:33:37.751 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-selinux x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 25 k 2024-05-09T06:33:37.751 INFO:teuthology.orchestra.run.smithi003.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-05-09T06:33:37.752 INFO:teuthology.orchestra.run.smithi003.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-05-09T06:33:37.752 INFO:teuthology.orchestra.run.smithi003.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-05-09T06:33:37.752 INFO:teuthology.orchestra.run.smithi003.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-05-09T06:33:37.752 INFO:teuthology.orchestra.run.smithi003.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-05-09T06:33:37.752 INFO:teuthology.orchestra.run.smithi003.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-05-09T06:33:37.752 INFO:teuthology.orchestra.run.smithi003.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-05-09T06:33:37.752 INFO:teuthology.orchestra.run.smithi003.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-05-09T06:33:37.752 INFO:teuthology.orchestra.run.smithi003.stdout: libcephsqlite x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 163 k 2024-05-09T06:33:37.752 INFO:teuthology.orchestra.run.smithi003.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-05-09T06:33:37.752 INFO:teuthology.orchestra.run.smithi003.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-05-09T06:33:37.752 INFO:teuthology.orchestra.run.smithi003.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-05-09T06:33:37.752 INFO:teuthology.orchestra.run.smithi003.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-05-09T06:33:37.752 INFO:teuthology.orchestra.run.smithi003.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-05-09T06:33:37.752 INFO:teuthology.orchestra.run.smithi003.stdout: libradosstriper1 x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 500 k 2024-05-09T06:33:37.752 INFO:teuthology.orchestra.run.smithi003.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-05-09T06:33:37.753 INFO:teuthology.orchestra.run.smithi003.stdout: librgw2 x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 5.3 M 2024-05-09T06:33:37.753 INFO:teuthology.orchestra.run.smithi003.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-05-09T06:33:37.753 INFO:teuthology.orchestra.run.smithi003.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-05-09T06:33:37.753 INFO:teuthology.orchestra.run.smithi003.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-05-09T06:33:37.753 INFO:teuthology.orchestra.run.smithi003.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-05-09T06:33:37.753 INFO:teuthology.orchestra.run.smithi003.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-05-09T06:33:37.753 INFO:teuthology.orchestra.run.smithi003.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-05-09T06:33:37.753 INFO:teuthology.orchestra.run.smithi003.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-05-09T06:33:37.753 INFO:teuthology.orchestra.run.smithi003.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-05-09T06:33:37.753 INFO:teuthology.orchestra.run.smithi003.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-05-09T06:33:37.753 INFO:teuthology.orchestra.run.smithi003.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-05-09T06:33:37.753 INFO:teuthology.orchestra.run.smithi003.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-05-09T06:33:37.753 INFO:teuthology.orchestra.run.smithi003.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-05-09T06:33:37.753 INFO:teuthology.orchestra.run.smithi003.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-05-09T06:33:37.753 INFO:teuthology.orchestra.run.smithi003.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-05-09T06:33:37.754 INFO:teuthology.orchestra.run.smithi003.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-05-09T06:33:37.754 INFO:teuthology.orchestra.run.smithi003.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-05-09T06:33:37.754 INFO:teuthology.orchestra.run.smithi003.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-05-09T06:33:37.754 INFO:teuthology.orchestra.run.smithi003.stdout: python3-ceph-argparse x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 45 k 2024-05-09T06:33:37.754 INFO:teuthology.orchestra.run.smithi003.stdout: python3-ceph-common x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 135 k 2024-05-09T06:33:37.754 INFO:teuthology.orchestra.run.smithi003.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-05-09T06:33:37.754 INFO:teuthology.orchestra.run.smithi003.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-05-09T06:33:37.754 INFO:teuthology.orchestra.run.smithi003.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-05-09T06:33:37.754 INFO:teuthology.orchestra.run.smithi003.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-05-09T06:33:37.754 INFO:teuthology.orchestra.run.smithi003.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-05-09T06:33:37.754 INFO:teuthology.orchestra.run.smithi003.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-05-09T06:33:37.754 INFO:teuthology.orchestra.run.smithi003.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-05-09T06:33:37.754 INFO:teuthology.orchestra.run.smithi003.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-05-09T06:33:37.754 INFO:teuthology.orchestra.run.smithi003.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-05-09T06:33:37.755 INFO:teuthology.orchestra.run.smithi003.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-05-09T06:33:37.755 INFO:teuthology.orchestra.run.smithi003.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-05-09T06:33:37.755 INFO:teuthology.orchestra.run.smithi003.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-05-09T06:33:37.755 INFO:teuthology.orchestra.run.smithi003.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-05-09T06:33:37.755 INFO:teuthology.orchestra.run.smithi003.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-05-09T06:33:37.755 INFO:teuthology.orchestra.run.smithi003.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-05-09T06:33:37.755 INFO:teuthology.orchestra.run.smithi003.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-05-09T06:33:37.755 INFO:teuthology.orchestra.run.smithi003.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-05-09T06:33:37.755 INFO:teuthology.orchestra.run.smithi003.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-05-09T06:33:37.755 INFO:teuthology.orchestra.run.smithi003.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-05-09T06:33:37.755 INFO:teuthology.orchestra.run.smithi003.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-05-09T06:33:37.755 INFO:teuthology.orchestra.run.smithi003.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-05-09T06:33:37.755 INFO:teuthology.orchestra.run.smithi003.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-05-09T06:33:37.755 INFO:teuthology.orchestra.run.smithi003.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-05-09T06:33:37.755 INFO:teuthology.orchestra.run.smithi003.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-05-09T06:33:37.756 INFO:teuthology.orchestra.run.smithi003.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-05-09T06:33:37.756 INFO:teuthology.orchestra.run.smithi003.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-05-09T06:33:37.756 INFO:teuthology.orchestra.run.smithi003.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-05-09T06:33:37.756 INFO:teuthology.orchestra.run.smithi003.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-05-09T06:33:37.756 INFO:teuthology.orchestra.run.smithi003.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-05-09T06:33:37.756 INFO:teuthology.orchestra.run.smithi003.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-05-09T06:33:37.756 INFO:teuthology.orchestra.run.smithi003.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-05-09T06:33:37.756 INFO:teuthology.orchestra.run.smithi003.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-05-09T06:33:37.756 INFO:teuthology.orchestra.run.smithi003.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-05-09T06:33:37.756 INFO:teuthology.orchestra.run.smithi003.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-05-09T06:33:37.756 INFO:teuthology.orchestra.run.smithi003.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-05-09T06:33:37.756 INFO:teuthology.orchestra.run.smithi003.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-05-09T06:33:37.756 INFO:teuthology.orchestra.run.smithi003.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-05-09T06:33:37.756 INFO:teuthology.orchestra.run.smithi003.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-05-09T06:33:37.756 INFO:teuthology.orchestra.run.smithi003.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-05-09T06:33:37.756 INFO:teuthology.orchestra.run.smithi003.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-05-09T06:33:37.757 INFO:teuthology.orchestra.run.smithi003.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-05-09T06:33:37.757 INFO:teuthology.orchestra.run.smithi003.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-05-09T06:33:37.757 INFO:teuthology.orchestra.run.smithi003.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-05-09T06:33:37.757 INFO:teuthology.orchestra.run.smithi003.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-05-09T06:33:37.757 INFO:teuthology.orchestra.run.smithi003.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-05-09T06:33:37.757 INFO:teuthology.orchestra.run.smithi003.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-05-09T06:33:37.757 INFO:teuthology.orchestra.run.smithi003.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-05-09T06:33:37.757 INFO:teuthology.orchestra.run.smithi003.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-05-09T06:33:37.757 INFO:teuthology.orchestra.run.smithi003.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-05-09T06:33:37.757 INFO:teuthology.orchestra.run.smithi003.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-05-09T06:33:37.757 INFO:teuthology.orchestra.run.smithi003.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-05-09T06:33:37.757 INFO:teuthology.orchestra.run.smithi003.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-05-09T06:33:37.757 INFO:teuthology.orchestra.run.smithi003.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-05-09T06:33:37.757 INFO:teuthology.orchestra.run.smithi003.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-05-09T06:33:37.757 INFO:teuthology.orchestra.run.smithi003.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-05-09T06:33:37.758 INFO:teuthology.orchestra.run.smithi003.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-05-09T06:33:37.758 INFO:teuthology.orchestra.run.smithi003.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-05-09T06:33:37.758 INFO:teuthology.orchestra.run.smithi003.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-05-09T06:33:37.758 INFO:teuthology.orchestra.run.smithi003.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-05-09T06:33:37.758 INFO:teuthology.orchestra.run.smithi003.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-05-09T06:33:37.758 INFO:teuthology.orchestra.run.smithi003.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-05-09T06:33:37.758 INFO:teuthology.orchestra.run.smithi003.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-05-09T06:33:37.758 INFO:teuthology.orchestra.run.smithi003.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-05-09T06:33:37.758 INFO:teuthology.orchestra.run.smithi003.stdout:Installing weak dependencies: 2024-05-09T06:33:37.758 INFO:teuthology.orchestra.run.smithi003.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-05-09T06:33:37.758 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-09T06:33:37.758 INFO:teuthology.orchestra.run.smithi003.stdout:Transaction Summary 2024-05-09T06:33:37.758 INFO:teuthology.orchestra.run.smithi003.stdout:======================================================================================= 2024-05-09T06:33:37.758 INFO:teuthology.orchestra.run.smithi003.stdout:Install 148 Packages 2024-05-09T06:33:37.759 INFO:teuthology.orchestra.run.smithi003.stdout:Upgrade 8 Packages 2024-05-09T06:33:37.759 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-09T06:33:37.759 INFO:teuthology.orchestra.run.smithi003.stdout:Total download size: 225 M 2024-05-09T06:33:37.759 INFO:teuthology.orchestra.run.smithi003.stdout:Downloading Packages: 2024-05-09T06:33:38.437 INFO:teuthology.orchestra.run.smithi190.stdout:CentOS Stream 9 - Extras packages 38 kB/s | 16 kB 00:00 2024-05-09T06:33:38.631 INFO:teuthology.orchestra.run.smithi003.stdout:(1/156): ceph-19.0.0-2515.g96805dcf.el9.x86_64. 37 kB/s | 6.6 kB 00:00 2024-05-09T06:33:38.831 INFO:teuthology.orchestra.run.smithi003.stdout:(2/156): ceph-fuse-19.0.0-2515.g96805dcf.el9.x8 4.3 MB/s | 889 kB 00:00 2024-05-09T06:33:38.898 INFO:teuthology.orchestra.run.smithi003.stdout:(3/156): ceph-immutable-object-cache-19.0.0-251 2.1 MB/s | 145 kB 00:00 2024-05-09T06:33:39.157 INFO:teuthology.orchestra.run.smithi003.stdout:(4/156): ceph-mds-19.0.0-2515.g96805dcf.el9.x86 9.1 MB/s | 2.4 MB 00:00 2024-05-09T06:33:39.365 INFO:teuthology.orchestra.run.smithi003.stdout:(5/156): ceph-mgr-19.0.0-2515.g96805dcf.el9.x86 7.2 MB/s | 1.5 MB 00:00 2024-05-09T06:33:39.899 INFO:teuthology.orchestra.run.smithi003.stdout:(6/156): ceph-mon-19.0.0-2515.g96805dcf.el9.x86 8.8 MB/s | 4.7 MB 00:00 2024-05-09T06:33:40.099 INFO:teuthology.orchestra.run.smithi003.stdout:(7/156): ceph-common-19.0.0-2515.g96805dcf.el9. 13 MB/s | 21 MB 00:01 2024-05-09T06:33:40.874 INFO:teuthology.orchestra.run.smithi003.stdout:(8/156): ceph-radosgw-19.0.0-2515.g96805dcf.el9 13 MB/s | 10 MB 00:00 2024-05-09T06:33:40.941 INFO:teuthology.orchestra.run.smithi003.stdout:(9/156): ceph-base-19.0.0-2515.g96805dcf.el9.x8 2.2 MB/s | 5.4 MB 00:02 2024-05-09T06:33:40.958 INFO:teuthology.orchestra.run.smithi003.stdout:(10/156): ceph-selinux-19.0.0-2515.g96805dcf.el 301 kB/s | 25 kB 00:00 2024-05-09T06:33:41.034 INFO:teuthology.orchestra.run.smithi003.stdout:(11/156): cephfs-mirror-19.0.0-2515.g96805dcf.e 2.9 MB/s | 222 kB 00:00 2024-05-09T06:33:41.092 INFO:teuthology.orchestra.run.smithi003.stdout:(12/156): cephfs-shell-19.0.0-2515.g96805dcf.el 465 kB/s | 27 kB 00:00 2024-05-09T06:33:41.151 INFO:teuthology.orchestra.run.smithi003.stdout:(13/156): libcephfs-devel-19.0.0-2515.g96805dcf 545 kB/s | 32 kB 00:00 2024-05-09T06:33:41.226 INFO:teuthology.orchestra.run.smithi003.stdout:(14/156): libcephfs2-19.0.0-2515.g96805dcf.el9. 9.7 MB/s | 746 kB 00:00 2024-05-09T06:33:41.385 INFO:teuthology.orchestra.run.smithi003.stdout:(15/156): ceph-osd-19.0.0-2515.g96805dcf.el9.x8 11 MB/s | 17 MB 00:01 2024-05-09T06:33:41.408 INFO:teuthology.orchestra.run.smithi190.stdout:Extra Packages for Enterprise Linux 7.5 MB/s | 21 MB 00:02 2024-05-09T06:33:41.410 INFO:teuthology.orchestra.run.smithi003.stdout:(16/156): libcephsqlite-19.0.0-2515.g96805dcf.e 889 kB/s | 163 kB 00:00 2024-05-09T06:33:41.443 INFO:teuthology.orchestra.run.smithi003.stdout:(17/156): librados-devel-19.0.0-2515.g96805dcf. 2.1 MB/s | 127 kB 00:00 2024-05-09T06:33:41.486 INFO:teuthology.orchestra.run.smithi003.stdout:(18/156): libradosstriper1-19.0.0-2515.g96805dc 6.5 MB/s | 500 kB 00:00 2024-05-09T06:33:41.536 INFO:teuthology.orchestra.run.smithi003.stdout:(19/156): python3-ceph-argparse-19.0.0-2515.g96 900 kB/s | 45 kB 00:00 2024-05-09T06:33:41.595 INFO:teuthology.orchestra.run.smithi003.stdout:(20/156): python3-ceph-common-19.0.0-2515.g9680 2.3 MB/s | 135 kB 00:00 2024-05-09T06:33:41.662 INFO:teuthology.orchestra.run.smithi003.stdout:(21/156): python3-cephfs-19.0.0-2515.g96805dcf. 2.4 MB/s | 162 kB 00:00 2024-05-09T06:33:41.737 INFO:teuthology.orchestra.run.smithi003.stdout:(22/156): python3-rados-19.0.0-2515.g96805dcf.e 4.2 MB/s | 321 kB 00:00 2024-05-09T06:33:41.820 INFO:teuthology.orchestra.run.smithi003.stdout:(23/156): librgw2-19.0.0-2515.g96805dcf.el9.x86 14 MB/s | 5.3 MB 00:00 2024-05-09T06:33:41.846 INFO:teuthology.orchestra.run.smithi003.stdout:(24/156): python3-rbd-19.0.0-2515.g96805dcf.el9 2.7 MB/s | 298 kB 00:00 2024-05-09T06:33:41.888 INFO:teuthology.orchestra.run.smithi003.stdout:(25/156): python3-rgw-19.0.0-2515.g96805dcf.el9 1.4 MB/s | 99 kB 00:00 2024-05-09T06:33:41.905 INFO:teuthology.orchestra.run.smithi003.stdout:(26/156): rbd-fuse-19.0.0-2515.g96805dcf.el9.x8 1.4 MB/s | 85 kB 00:00 2024-05-09T06:33:41.980 INFO:teuthology.orchestra.run.smithi003.stdout:(27/156): rbd-nbd-19.0.0-2515.g96805dcf.el9.x86 2.2 MB/s | 171 kB 00:00 2024-05-09T06:33:42.047 INFO:teuthology.orchestra.run.smithi003.stdout:(28/156): ceph-grafana-dashboards-19.0.0-2515.g 478 kB/s | 32 kB 00:00 2024-05-09T06:33:42.114 INFO:teuthology.orchestra.run.smithi003.stdout:(29/156): ceph-mgr-cephadm-19.0.0-2515.g96805dc 2.1 MB/s | 142 kB 00:00 2024-05-09T06:33:42.165 INFO:teuthology.orchestra.run.smithi003.stdout:(30/156): rbd-mirror-19.0.0-2515.g96805dcf.el9. 11 MB/s | 3.1 MB 00:00 2024-05-09T06:33:42.548 INFO:teuthology.orchestra.run.smithi003.stdout:(31/156): ceph-mgr-dashboard-19.0.0-2515.g96805 8.1 MB/s | 3.5 MB 00:00 2024-05-09T06:33:42.631 INFO:teuthology.orchestra.run.smithi003.stdout:(32/156): ceph-mgr-modules-core-19.0.0-2515.g96 2.9 MB/s | 249 kB 00:00 2024-05-09T06:33:42.715 INFO:teuthology.orchestra.run.smithi003.stdout:(33/156): ceph-mgr-rook-19.0.0-2515.g96805dcf.e 592 kB/s | 49 kB 00:00 2024-05-09T06:33:42.799 INFO:teuthology.orchestra.run.smithi003.stdout:(34/156): ceph-prometheus-alerts-19.0.0-2515.g9 197 kB/s | 16 kB 00:00 2024-05-09T06:33:42.924 INFO:teuthology.orchestra.run.smithi003.stdout:(35/156): ceph-mgr-diskprediction-local-19.0.0- 9.8 MB/s | 7.4 MB 00:00 2024-05-09T06:33:42.949 INFO:teuthology.orchestra.run.smithi003.stdout:(36/156): ceph-volume-19.0.0-2515.g96805dcf.el9 1.7 MB/s | 264 kB 00:00 2024-05-09T06:33:43.016 INFO:teuthology.orchestra.run.smithi003.stdout:(37/156): cephadm-19.0.0-2515.g96805dcf.el9.noa 8.1 MB/s | 763 kB 00:00 2024-05-09T06:33:43.033 INFO:teuthology.orchestra.run.smithi003.stdout:(38/156): cephfs-top-19.0.0-2515.g96805dcf.el9. 218 kB/s | 18 kB 00:00 2024-05-09T06:33:43.100 INFO:teuthology.orchestra.run.smithi003.stdout:(39/156): fuse-2.9.9-15.el9.x86_64.rpm 962 kB/s | 80 kB 00:00 2024-05-09T06:33:43.142 INFO:teuthology.orchestra.run.smithi003.stdout:(40/156): libgfortran-11.4.1-3.el9.x86_64.rpm 7.2 MB/s | 803 kB 00:00 2024-05-09T06:33:43.184 INFO:teuthology.orchestra.run.smithi003.stdout:(41/156): libquadmath-11.4.1-3.el9.x86_64.rpm 2.3 MB/s | 195 kB 00:00 2024-05-09T06:33:43.234 INFO:teuthology.orchestra.run.smithi003.stdout:(42/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.7 MB/s | 253 kB 00:00 2024-05-09T06:33:43.301 INFO:teuthology.orchestra.run.smithi003.stdout:(43/156): python3-ply-3.11-14.el9.noarch.rpm 1.6 MB/s | 106 kB 00:00 2024-05-09T06:33:43.343 INFO:teuthology.orchestra.run.smithi003.stdout:(44/156): python3-cryptography-36.0.1-4.el9.x86 7.7 MB/s | 1.2 MB 00:00 2024-05-09T06:33:43.394 INFO:teuthology.orchestra.run.smithi003.stdout:(45/156): python3-pycparser-2.20-6.el9.noarch.r 1.4 MB/s | 135 kB 00:00 2024-05-09T06:33:43.419 INFO:teuthology.orchestra.run.smithi003.stdout:(46/156): python3-requests-2.25.1-8.el9.noarch. 1.6 MB/s | 125 kB 00:00 2024-05-09T06:33:43.503 INFO:teuthology.orchestra.run.smithi003.stdout:(47/156): python3-urllib3-1.26.5-5.el9.noarch.r 1.9 MB/s | 215 kB 00:00 2024-05-09T06:33:44.353 INFO:teuthology.orchestra.run.smithi003.stdout:(48/156): ceph-test-19.0.0-2515.g96805dcf.el9.x 14 MB/s | 48 MB 00:03 2024-05-09T06:33:45.070 INFO:teuthology.orchestra.run.smithi003.stdout:(49/156): boost-program-options-1.75.0-8.el9.x8 149 kB/s | 107 kB 00:00 2024-05-09T06:33:45.103 INFO:teuthology.orchestra.run.smithi003.stdout:(50/156): automake-1.16.2-8.el9.noarch.rpm 409 kB/s | 689 kB 00:01 2024-05-09T06:33:45.654 INFO:teuthology.orchestra.run.smithi003.stdout:(51/156): elfutils-libelf-devel-0.190-2.el9.x86 37 kB/s | 21 kB 00:00 2024-05-09T06:33:45.712 INFO:teuthology.orchestra.run.smithi003.stdout:(52/156): bison-3.7.4-5.el9.x86_64.rpm 427 kB/s | 943 kB 00:02 2024-05-09T06:33:45.937 INFO:teuthology.orchestra.run.smithi003.stdout:(53/156): flex-2.6.4-9.el9.x86_64.rpm 376 kB/s | 313 kB 00:00 2024-05-09T06:33:46.246 INFO:teuthology.orchestra.run.smithi003.stdout:(54/156): flexiblas-3.0.4-8.el9.x86_64.rpm 55 kB/s | 33 kB 00:00 2024-05-09T06:33:46.488 INFO:teuthology.orchestra.run.smithi003.stdout:(55/156): flexiblas-openblas-openmp-3.0.4-8.el9 35 kB/s | 18 kB 00:00 2024-05-09T06:33:46.631 INFO:teuthology.orchestra.run.smithi190.stdout:lab-extras 16 kB/s | 1.7 kB 00:00 2024-05-09T06:33:46.788 INFO:teuthology.orchestra.run.smithi003.stdout:(56/156): libaio-devel-0.3.111-13.el9.x86_64.rp 21 kB/s | 11 kB 00:00 2024-05-09T06:33:46.855 INFO:teuthology.orchestra.run.smithi003.stdout:(57/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 2.6 MB/s | 3.0 MB 00:01 2024-05-09T06:33:47.306 INFO:teuthology.orchestra.run.smithi003.stdout:(58/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 203 kB/s | 160 kB 00:00 2024-05-09T06:33:47.414 INFO:teuthology.orchestra.run.smithi003.stdout:(59/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 72 kB/s | 45 kB 00:00 2024-05-09T06:33:47.789 INFO:teuthology.orchestra.run.smithi003.stdout:(60/156): librdkafka-1.6.1-102.el9.x86_64.rpm 710 kB/s | 662 kB 00:00 2024-05-09T06:33:48.015 INFO:teuthology.orchestra.run.smithi003.stdout:(61/156): libuuid-devel-2.37.4-18.el9.x86_64.rp 42 kB/s | 25 kB 00:00 2024-05-09T06:33:48.223 INFO:teuthology.orchestra.run.smithi003.stdout:(62/156): libtool-2.4.6-45.el9.x86_64.rpm 635 kB/s | 582 kB 00:00 2024-05-09T06:33:48.424 INFO:teuthology.orchestra.run.smithi003.stdout:(63/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 75 kB/s | 47 kB 00:00 2024-05-09T06:33:48.865 INFO:teuthology.orchestra.run.smithi003.stdout:(64/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 356 kB/s | 292 kB 00:00 2024-05-09T06:33:48.990 INFO:teuthology.orchestra.run.smithi003.stdout:(65/156): lua-5.4.4-4.el9.x86_64.rpm 246 kB/s | 188 kB 00:00 2024-05-09T06:33:48.991 INFO:teuthology.orchestra.run.smithi190.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-09T06:33:48.992 INFO:teuthology.orchestra.run.smithi190.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-09T06:33:49.003 INFO:teuthology.orchestra.run.smithi190.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-05-09T06:33:49.004 INFO:teuthology.orchestra.run.smithi190.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-05-09T06:33:49.005 INFO:teuthology.orchestra.run.smithi190.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-05-09T06:33:49.005 INFO:teuthology.orchestra.run.smithi190.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-05-09T06:33:49.007 INFO:teuthology.orchestra.run.smithi190.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-05-09T06:33:49.009 INFO:teuthology.orchestra.run.smithi190.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-05-09T06:33:49.010 INFO:teuthology.orchestra.run.smithi190.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-05-09T06:33:49.024 INFO:teuthology.orchestra.run.smithi003.stdout:(66/156): openblas-0.3.21-2.el9.x86_64.rpm 59 kB/s | 35 kB 00:00 2024-05-09T06:33:49.070 INFO:teuthology.orchestra.run.smithi190.stdout:Dependencies resolved. 2024-05-09T06:33:49.085 INFO:teuthology.orchestra.run.smithi190.stdout:======================================================================================= 2024-05-09T06:33:49.085 INFO:teuthology.orchestra.run.smithi190.stdout: Package Arch Version Repository Size 2024-05-09T06:33:49.085 INFO:teuthology.orchestra.run.smithi190.stdout:======================================================================================= 2024-05-09T06:33:49.085 INFO:teuthology.orchestra.run.smithi190.stdout:Installing: 2024-05-09T06:33:49.085 INFO:teuthology.orchestra.run.smithi190.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-05-09T06:33:49.085 INFO:teuthology.orchestra.run.smithi190.stdout: ceph x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 6.6 k 2024-05-09T06:33:49.085 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-base x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 5.4 M 2024-05-09T06:33:49.085 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-fuse x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 889 k 2024-05-09T06:33:49.085 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 145 k 2024-05-09T06:33:49.085 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mgr x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 1.5 M 2024-05-09T06:33:49.085 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 142 k 2024-05-09T06:33:49.085 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 3.5 M 2024-05-09T06:33:49.085 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 7.4 M 2024-05-09T06:33:49.085 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mgr-rook noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 49 k 2024-05-09T06:33:49.086 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-radosgw x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 10 M 2024-05-09T06:33:49.086 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-test x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 48 M 2024-05-09T06:33:49.086 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-volume noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 264 k 2024-05-09T06:33:49.086 INFO:teuthology.orchestra.run.smithi190.stdout: cephadm noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 763 k 2024-05-09T06:33:49.086 INFO:teuthology.orchestra.run.smithi190.stdout: cephfs-mirror x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 222 k 2024-05-09T06:33:49.086 INFO:teuthology.orchestra.run.smithi190.stdout: cephfs-shell x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 27 k 2024-05-09T06:33:49.086 INFO:teuthology.orchestra.run.smithi190.stdout: cephfs-top noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 18 k 2024-05-09T06:33:49.086 INFO:teuthology.orchestra.run.smithi190.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-05-09T06:33:49.086 INFO:teuthology.orchestra.run.smithi190.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-05-09T06:33:49.086 INFO:teuthology.orchestra.run.smithi190.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-05-09T06:33:49.086 INFO:teuthology.orchestra.run.smithi190.stdout: libcephfs-devel x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 32 k 2024-05-09T06:33:49.086 INFO:teuthology.orchestra.run.smithi190.stdout: libcephfs2 x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 746 k 2024-05-09T06:33:49.086 INFO:teuthology.orchestra.run.smithi190.stdout: librados-devel x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 127 k 2024-05-09T06:33:49.087 INFO:teuthology.orchestra.run.smithi190.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-05-09T06:33:49.087 INFO:teuthology.orchestra.run.smithi190.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-05-09T06:33:49.087 INFO:teuthology.orchestra.run.smithi190.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-05-09T06:33:49.087 INFO:teuthology.orchestra.run.smithi190.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-05-09T06:33:49.087 INFO:teuthology.orchestra.run.smithi190.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-05-09T06:33:49.087 INFO:teuthology.orchestra.run.smithi190.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-05-09T06:33:49.087 INFO:teuthology.orchestra.run.smithi190.stdout: python3-cephfs x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 162 k 2024-05-09T06:33:49.087 INFO:teuthology.orchestra.run.smithi190.stdout: python3-rados x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 321 k 2024-05-09T06:33:49.087 INFO:teuthology.orchestra.run.smithi190.stdout: python3-rbd x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 298 k 2024-05-09T06:33:49.087 INFO:teuthology.orchestra.run.smithi190.stdout: python3-rgw x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 99 k 2024-05-09T06:33:49.087 INFO:teuthology.orchestra.run.smithi190.stdout: rbd-fuse x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 85 k 2024-05-09T06:33:49.087 INFO:teuthology.orchestra.run.smithi190.stdout: rbd-mirror x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 3.1 M 2024-05-09T06:33:49.087 INFO:teuthology.orchestra.run.smithi190.stdout: rbd-nbd x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 171 k 2024-05-09T06:33:49.087 INFO:teuthology.orchestra.run.smithi190.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-05-09T06:33:49.087 INFO:teuthology.orchestra.run.smithi190.stdout:Upgrading: 2024-05-09T06:33:49.088 INFO:teuthology.orchestra.run.smithi190.stdout: NetworkManager x86_64 1:1.47.5-1.el9 baseos 2.3 M 2024-05-09T06:33:49.088 INFO:teuthology.orchestra.run.smithi190.stdout: NetworkManager-libnm x86_64 1:1.47.5-1.el9 baseos 1.8 M 2024-05-09T06:33:49.088 INFO:teuthology.orchestra.run.smithi190.stdout: NetworkManager-team x86_64 1:1.47.5-1.el9 baseos 42 k 2024-05-09T06:33:49.088 INFO:teuthology.orchestra.run.smithi190.stdout: NetworkManager-tui x86_64 1:1.47.5-1.el9 baseos 248 k 2024-05-09T06:33:49.088 INFO:teuthology.orchestra.run.smithi190.stdout: librados2 x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 3.4 M 2024-05-09T06:33:49.088 INFO:teuthology.orchestra.run.smithi190.stdout: librbd1 x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 3.1 M 2024-05-09T06:33:49.088 INFO:teuthology.orchestra.run.smithi190.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-05-09T06:33:49.088 INFO:teuthology.orchestra.run.smithi190.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-05-09T06:33:49.088 INFO:teuthology.orchestra.run.smithi190.stdout:Installing dependencies: 2024-05-09T06:33:49.088 INFO:teuthology.orchestra.run.smithi190.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-05-09T06:33:49.088 INFO:teuthology.orchestra.run.smithi190.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-05-09T06:33:49.088 INFO:teuthology.orchestra.run.smithi190.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-05-09T06:33:49.088 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-common x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 21 M 2024-05-09T06:33:49.089 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 32 k 2024-05-09T06:33:49.089 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mds x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 2.4 M 2024-05-09T06:33:49.089 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 249 k 2024-05-09T06:33:49.089 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mon x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 4.7 M 2024-05-09T06:33:49.089 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-osd x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 17 M 2024-05-09T06:33:49.089 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 16 k 2024-05-09T06:33:49.089 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-selinux x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 25 k 2024-05-09T06:33:49.089 INFO:teuthology.orchestra.run.smithi190.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-05-09T06:33:49.089 INFO:teuthology.orchestra.run.smithi190.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-05-09T06:33:49.089 INFO:teuthology.orchestra.run.smithi190.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-05-09T06:33:49.089 INFO:teuthology.orchestra.run.smithi190.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-05-09T06:33:49.089 INFO:teuthology.orchestra.run.smithi190.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-05-09T06:33:49.089 INFO:teuthology.orchestra.run.smithi190.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-05-09T06:33:49.089 INFO:teuthology.orchestra.run.smithi190.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-05-09T06:33:49.090 INFO:teuthology.orchestra.run.smithi190.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-05-09T06:33:49.090 INFO:teuthology.orchestra.run.smithi190.stdout: libcephsqlite x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 163 k 2024-05-09T06:33:49.090 INFO:teuthology.orchestra.run.smithi190.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-05-09T06:33:49.090 INFO:teuthology.orchestra.run.smithi190.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-05-09T06:33:49.090 INFO:teuthology.orchestra.run.smithi190.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-05-09T06:33:49.090 INFO:teuthology.orchestra.run.smithi190.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-05-09T06:33:49.090 INFO:teuthology.orchestra.run.smithi190.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-05-09T06:33:49.090 INFO:teuthology.orchestra.run.smithi190.stdout: libradosstriper1 x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 500 k 2024-05-09T06:33:49.090 INFO:teuthology.orchestra.run.smithi190.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-05-09T06:33:49.090 INFO:teuthology.orchestra.run.smithi190.stdout: librgw2 x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 5.3 M 2024-05-09T06:33:49.090 INFO:teuthology.orchestra.run.smithi190.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-05-09T06:33:49.090 INFO:teuthology.orchestra.run.smithi190.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-05-09T06:33:49.090 INFO:teuthology.orchestra.run.smithi190.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-05-09T06:33:49.090 INFO:teuthology.orchestra.run.smithi190.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-05-09T06:33:49.091 INFO:teuthology.orchestra.run.smithi190.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-05-09T06:33:49.091 INFO:teuthology.orchestra.run.smithi190.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-05-09T06:33:49.091 INFO:teuthology.orchestra.run.smithi190.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-05-09T06:33:49.091 INFO:teuthology.orchestra.run.smithi190.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-05-09T06:33:49.091 INFO:teuthology.orchestra.run.smithi190.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-05-09T06:33:49.091 INFO:teuthology.orchestra.run.smithi190.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-05-09T06:33:49.091 INFO:teuthology.orchestra.run.smithi190.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-05-09T06:33:49.091 INFO:teuthology.orchestra.run.smithi190.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-05-09T06:33:49.091 INFO:teuthology.orchestra.run.smithi190.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-05-09T06:33:49.091 INFO:teuthology.orchestra.run.smithi190.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-05-09T06:33:49.091 INFO:teuthology.orchestra.run.smithi190.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-05-09T06:33:49.091 INFO:teuthology.orchestra.run.smithi190.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-05-09T06:33:49.091 INFO:teuthology.orchestra.run.smithi190.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-05-09T06:33:49.091 INFO:teuthology.orchestra.run.smithi190.stdout: python3-ceph-argparse x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 45 k 2024-05-09T06:33:49.092 INFO:teuthology.orchestra.run.smithi190.stdout: python3-ceph-common x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 135 k 2024-05-09T06:33:49.092 INFO:teuthology.orchestra.run.smithi190.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-05-09T06:33:49.092 INFO:teuthology.orchestra.run.smithi190.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-05-09T06:33:49.092 INFO:teuthology.orchestra.run.smithi190.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-05-09T06:33:49.092 INFO:teuthology.orchestra.run.smithi190.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-05-09T06:33:49.092 INFO:teuthology.orchestra.run.smithi190.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-05-09T06:33:49.092 INFO:teuthology.orchestra.run.smithi190.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-05-09T06:33:49.092 INFO:teuthology.orchestra.run.smithi190.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-05-09T06:33:49.092 INFO:teuthology.orchestra.run.smithi190.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-05-09T06:33:49.092 INFO:teuthology.orchestra.run.smithi190.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-05-09T06:33:49.092 INFO:teuthology.orchestra.run.smithi190.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-05-09T06:33:49.092 INFO:teuthology.orchestra.run.smithi190.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-05-09T06:33:49.092 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-05-09T06:33:49.092 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-05-09T06:33:49.093 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-05-09T06:33:49.093 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-05-09T06:33:49.093 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-05-09T06:33:49.093 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-05-09T06:33:49.093 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-05-09T06:33:49.093 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-05-09T06:33:49.093 INFO:teuthology.orchestra.run.smithi190.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-05-09T06:33:49.093 INFO:teuthology.orchestra.run.smithi190.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-05-09T06:33:49.093 INFO:teuthology.orchestra.run.smithi190.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-05-09T06:33:49.093 INFO:teuthology.orchestra.run.smithi190.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-05-09T06:33:49.093 INFO:teuthology.orchestra.run.smithi190.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-05-09T06:33:49.093 INFO:teuthology.orchestra.run.smithi190.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-05-09T06:33:49.093 INFO:teuthology.orchestra.run.smithi190.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-05-09T06:33:49.093 INFO:teuthology.orchestra.run.smithi190.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-05-09T06:33:49.093 INFO:teuthology.orchestra.run.smithi190.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-05-09T06:33:49.094 INFO:teuthology.orchestra.run.smithi190.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-05-09T06:33:49.094 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-05-09T06:33:49.094 INFO:teuthology.orchestra.run.smithi190.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-05-09T06:33:49.094 INFO:teuthology.orchestra.run.smithi190.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-05-09T06:33:49.094 INFO:teuthology.orchestra.run.smithi190.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-05-09T06:33:49.094 INFO:teuthology.orchestra.run.smithi190.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-05-09T06:33:49.094 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-05-09T06:33:49.094 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-05-09T06:33:49.094 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-05-09T06:33:49.094 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-05-09T06:33:49.094 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-05-09T06:33:49.094 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-05-09T06:33:49.094 INFO:teuthology.orchestra.run.smithi190.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-05-09T06:33:49.094 INFO:teuthology.orchestra.run.smithi190.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-05-09T06:33:49.095 INFO:teuthology.orchestra.run.smithi190.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-05-09T06:33:49.095 INFO:teuthology.orchestra.run.smithi190.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-05-09T06:33:49.095 INFO:teuthology.orchestra.run.smithi190.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-05-09T06:33:49.095 INFO:teuthology.orchestra.run.smithi190.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-05-09T06:33:49.095 INFO:teuthology.orchestra.run.smithi190.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-05-09T06:33:49.095 INFO:teuthology.orchestra.run.smithi190.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-05-09T06:33:49.095 INFO:teuthology.orchestra.run.smithi190.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-05-09T06:33:49.095 INFO:teuthology.orchestra.run.smithi190.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-05-09T06:33:49.095 INFO:teuthology.orchestra.run.smithi190.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-05-09T06:33:49.095 INFO:teuthology.orchestra.run.smithi190.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-05-09T06:33:49.095 INFO:teuthology.orchestra.run.smithi190.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-05-09T06:33:49.095 INFO:teuthology.orchestra.run.smithi190.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-05-09T06:33:49.095 INFO:teuthology.orchestra.run.smithi190.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-05-09T06:33:49.095 INFO:teuthology.orchestra.run.smithi190.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-05-09T06:33:49.095 INFO:teuthology.orchestra.run.smithi190.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-05-09T06:33:49.096 INFO:teuthology.orchestra.run.smithi190.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-05-09T06:33:49.096 INFO:teuthology.orchestra.run.smithi190.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-05-09T06:33:49.096 INFO:teuthology.orchestra.run.smithi190.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-05-09T06:33:49.096 INFO:teuthology.orchestra.run.smithi190.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-05-09T06:33:49.096 INFO:teuthology.orchestra.run.smithi190.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-05-09T06:33:49.096 INFO:teuthology.orchestra.run.smithi190.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-05-09T06:33:49.096 INFO:teuthology.orchestra.run.smithi190.stdout:Installing weak dependencies: 2024-05-09T06:33:49.096 INFO:teuthology.orchestra.run.smithi190.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-05-09T06:33:49.096 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-05-09T06:33:49.096 INFO:teuthology.orchestra.run.smithi190.stdout:Transaction Summary 2024-05-09T06:33:49.096 INFO:teuthology.orchestra.run.smithi190.stdout:======================================================================================= 2024-05-09T06:33:49.096 INFO:teuthology.orchestra.run.smithi190.stdout:Install 148 Packages 2024-05-09T06:33:49.096 INFO:teuthology.orchestra.run.smithi190.stdout:Upgrade 8 Packages 2024-05-09T06:33:49.096 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-05-09T06:33:49.097 INFO:teuthology.orchestra.run.smithi190.stdout:Total download size: 225 M 2024-05-09T06:33:49.097 INFO:teuthology.orchestra.run.smithi190.stdout:Downloading Packages: 2024-05-09T06:33:50.033 INFO:teuthology.orchestra.run.smithi003.stdout:(67/156): openblas-openmp-0.3.21-2.el9.x86_64.r 4.1 MB/s | 4.7 MB 00:01 2024-05-09T06:33:50.075 INFO:teuthology.orchestra.run.smithi003.stdout:(68/156): postgresql-13.14-1.el9.x86_64.rpm 1.5 MB/s | 1.6 MB 00:01 2024-05-09T06:33:50.124 INFO:teuthology.orchestra.run.smithi190.stdout:(1/156): ceph-19.0.0-2515.g96805dcf.el9.x86_64. 27 kB/s | 6.6 kB 00:00 2024-05-09T06:33:50.167 INFO:teuthology.orchestra.run.smithi003.stdout:(69/156): openssl-devel-3.2.1-1.el9.x86_64.rpm 3.8 MB/s | 4.4 MB 00:01 2024-05-09T06:33:50.566 INFO:teuthology.orchestra.run.smithi190.stdout:(2/156): ceph-base-19.0.0-2515.g96805dcf.el9.x8 7.8 MB/s | 5.4 MB 00:00 2024-05-09T06:33:50.633 INFO:teuthology.orchestra.run.smithi190.stdout:(3/156): ceph-immutable-object-cache-19.0.0-251 2.1 MB/s | 145 kB 00:00 2024-05-09T06:33:50.859 INFO:teuthology.orchestra.run.smithi003.stdout:(70/156): postgresql-private-libs-13.14-1.el9.x 170 kB/s | 133 kB 00:00 2024-05-09T06:33:50.899 INFO:teuthology.orchestra.run.smithi190.stdout:(4/156): ceph-mds-19.0.0-2515.g96805dcf.el9.x86 8.9 MB/s | 2.4 MB 00:00 2024-05-09T06:33:50.991 INFO:teuthology.orchestra.run.smithi190.stdout:(5/156): ceph-fuse-19.0.0-2515.g96805dcf.el9.x8 1.0 MB/s | 889 kB 00:00 2024-05-09T06:33:51.051 INFO:teuthology.orchestra.run.smithi003.stdout:(71/156): postgresql-contrib-13.14-1.el9.x86_64 896 kB/s | 911 kB 00:01 2024-05-09T06:33:51.092 INFO:teuthology.orchestra.run.smithi190.stdout:(6/156): ceph-mgr-19.0.0-2515.g96805dcf.el9.x86 7.8 MB/s | 1.5 MB 00:00 2024-05-09T06:33:51.409 INFO:teuthology.orchestra.run.smithi003.stdout:(72/156): postgresql-server-13.14-1.el9.x86_64. 4.7 MB/s | 5.8 MB 00:01 2024-05-09T06:33:51.509 INFO:teuthology.orchestra.run.smithi190.stdout:(7/156): ceph-common-19.0.0-2515.g96805dcf.el9. 13 MB/s | 21 MB 00:01 2024-05-09T06:33:51.810 INFO:teuthology.orchestra.run.smithi003.stdout:(73/156): python3-attrs-20.3.0-7.el9.noarch.rpm 115 kB/s | 87 kB 00:00 2024-05-09T06:33:51.868 INFO:teuthology.orchestra.run.smithi003.stdout:(74/156): protobuf-3.14.0-13.el9.x86_64.rpm 1.0 MB/s | 1.0 MB 00:01 2024-05-09T06:33:52.309 INFO:teuthology.orchestra.run.smithi190.stdout:(8/156): ceph-osd-19.0.0-2515.g96805dcf.el9.x86 14 MB/s | 17 MB 00:01 2024-05-09T06:33:52.442 INFO:teuthology.orchestra.run.smithi190.stdout:(9/156): ceph-radosgw-19.0.0-2515.g96805dcf.el9 11 MB/s | 10 MB 00:00 2024-05-09T06:33:52.468 INFO:teuthology.orchestra.run.smithi190.stdout:(10/156): ceph-selinux-19.0.0-2515.g96805dcf.el 159 kB/s | 25 kB 00:00 2024-05-09T06:33:52.526 INFO:teuthology.orchestra.run.smithi190.stdout:(11/156): cephfs-mirror-19.0.0-2515.g96805dcf.e 3.7 MB/s | 222 kB 00:00 2024-05-09T06:33:52.585 INFO:teuthology.orchestra.run.smithi190.stdout:(12/156): cephfs-shell-19.0.0-2515.g96805dcf.el 465 kB/s | 27 kB 00:00 2024-05-09T06:33:52.643 INFO:teuthology.orchestra.run.smithi190.stdout:(13/156): libcephfs-devel-19.0.0-2515.g96805dcf 542 kB/s | 32 kB 00:00 2024-05-09T06:33:52.660 INFO:teuthology.orchestra.run.smithi003.stdout:(75/156): python3-babel-2.9.1-2.el9.noarch.rpm 4.8 MB/s | 6.0 MB 00:01 2024-05-09T06:33:52.686 INFO:teuthology.orchestra.run.smithi003.stdout:(76/156): python3-devel-3.9.18-3.el9.x86_64.rpm 279 kB/s | 244 kB 00:00 2024-05-09T06:33:52.719 INFO:teuthology.orchestra.run.smithi003.stdout:(77/156): python3-jinja2-2.11.3-5.el9.noarch.rp 291 kB/s | 248 kB 00:00 2024-05-09T06:33:52.735 INFO:teuthology.orchestra.run.smithi190.stdout:(14/156): libcephfs2-19.0.0-2515.g96805dcf.el9. 8.0 MB/s | 746 kB 00:00 2024-05-09T06:33:52.811 INFO:teuthology.orchestra.run.smithi190.stdout:(15/156): ceph-mon-19.0.0-2515.g96805dcf.el9.x8 2.6 MB/s | 4.7 MB 00:01 2024-05-09T06:33:52.836 INFO:teuthology.orchestra.run.smithi190.stdout:(16/156): libcephsqlite-19.0.0-2515.g96805dcf.e 1.6 MB/s | 163 kB 00:00 2024-05-09T06:33:52.878 INFO:teuthology.orchestra.run.smithi190.stdout:(17/156): librados-devel-19.0.0-2515.g96805dcf. 1.9 MB/s | 127 kB 00:00 2024-05-09T06:33:52.920 INFO:teuthology.orchestra.run.smithi190.stdout:(18/156): libradosstriper1-19.0.0-2515.g96805dc 5.8 MB/s | 500 kB 00:00 2024-05-09T06:33:52.978 INFO:teuthology.orchestra.run.smithi190.stdout:(19/156): python3-ceph-argparse-19.0.0-2515.g96 773 kB/s | 45 kB 00:00 2024-05-09T06:33:53.037 INFO:teuthology.orchestra.run.smithi190.stdout:(20/156): python3-ceph-common-19.0.0-2515.g9680 2.3 MB/s | 135 kB 00:00 2024-05-09T06:33:53.104 INFO:teuthology.orchestra.run.smithi190.stdout:(21/156): python3-cephfs-19.0.0-2515.g96805dcf. 2.4 MB/s | 162 kB 00:00 2024-05-09T06:33:53.188 INFO:teuthology.orchestra.run.smithi190.stdout:(22/156): python3-rados-19.0.0-2515.g96805dcf.e 3.8 MB/s | 321 kB 00:00 2024-05-09T06:33:53.228 INFO:teuthology.orchestra.run.smithi003.stdout:(78/156): python3-jsonpointer-2.0-4.el9.noarch. 35 kB/s | 19 kB 00:00 2024-05-09T06:33:53.253 INFO:teuthology.orchestra.run.smithi003.stdout:(79/156): python3-jsonpatch-1.21-16.el9.noarch. 43 kB/s | 26 kB 00:00 2024-05-09T06:33:53.263 INFO:teuthology.orchestra.run.smithi190.stdout:(23/156): python3-rbd-19.0.0-2515.g96805dcf.el9 3.9 MB/s | 298 kB 00:00 2024-05-09T06:33:53.321 INFO:teuthology.orchestra.run.smithi190.stdout:(24/156): python3-rgw-19.0.0-2515.g96805dcf.el9 1.7 MB/s | 99 kB 00:00 2024-05-09T06:33:53.380 INFO:teuthology.orchestra.run.smithi190.stdout:(25/156): rbd-fuse-19.0.0-2515.g96805dcf.el9.x8 1.4 MB/s | 85 kB 00:00 2024-05-09T06:33:53.536 INFO:teuthology.orchestra.run.smithi003.stdout:(80/156): python3-mako-1.1.4-6.el9.noarch.rpm 211 kB/s | 172 kB 00:00 2024-05-09T06:33:53.705 INFO:teuthology.orchestra.run.smithi190.stdout:(26/156): rbd-mirror-19.0.0-2515.g96805dcf.el9. 9.5 MB/s | 3.1 MB 00:00 2024-05-09T06:33:53.790 INFO:teuthology.orchestra.run.smithi190.stdout:(27/156): librgw2-19.0.0-2515.g96805dcf.el9.x86 5.8 MB/s | 5.3 MB 00:00 2024-05-09T06:33:53.814 INFO:teuthology.orchestra.run.smithi190.stdout:(28/156): rbd-nbd-19.0.0-2515.g96805dcf.el9.x86 1.5 MB/s | 171 kB 00:00 2024-05-09T06:33:53.853 INFO:teuthology.orchestra.run.smithi003.stdout:(81/156): python3-markupsafe-1.1.1-12.el9.x86_6 56 kB/s | 35 kB 00:00 2024-05-09T06:33:53.865 INFO:teuthology.orchestra.run.smithi190.stdout:(29/156): ceph-grafana-dashboards-19.0.0-2515.g 422 kB/s | 32 kB 00:00 2024-05-09T06:33:53.898 INFO:teuthology.orchestra.run.smithi190.stdout:(30/156): ceph-mgr-cephadm-19.0.0-2515.g96805dc 1.7 MB/s | 142 kB 00:00 2024-05-09T06:33:54.362 INFO:teuthology.orchestra.run.smithi003.stdout:(82/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 304 kB/s | 250 kB 00:00 2024-05-09T06:33:54.462 INFO:teuthology.orchestra.run.smithi003.stdout:(83/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 4.2 MB/s | 5.1 MB 00:01 2024-05-09T06:33:54.507 INFO:teuthology.orchestra.run.smithi190.stdout:(31/156): ceph-mgr-dashboard-19.0.0-2515.g96805 5.5 MB/s | 3.5 MB 00:00 2024-05-09T06:33:54.607 INFO:teuthology.orchestra.run.smithi190.stdout:(32/156): ceph-mgr-modules-core-19.0.0-2515.g96 2.4 MB/s | 249 kB 00:00 2024-05-09T06:33:54.699 INFO:teuthology.orchestra.run.smithi190.stdout:(33/156): ceph-mgr-rook-19.0.0-2515.g96805dcf.e 539 kB/s | 49 kB 00:00 2024-05-09T06:33:54.738 INFO:teuthology.orchestra.run.smithi003.stdout:(84/156): python3-oauthlib-3.1.1-5.el9.noarch.r 251 kB/s | 222 kB 00:00 2024-05-09T06:33:54.808 INFO:teuthology.orchestra.run.smithi190.stdout:(34/156): ceph-mgr-diskprediction-local-19.0.0- 8.2 MB/s | 7.4 MB 00:00 2024-05-09T06:33:54.825 INFO:teuthology.orchestra.run.smithi190.stdout:(35/156): ceph-prometheus-alerts-19.0.0-2515.g9 131 kB/s | 16 kB 00:00 2024-05-09T06:33:54.875 INFO:teuthology.orchestra.run.smithi190.stdout:(36/156): ceph-volume-19.0.0-2515.g96805dcf.el9 3.9 MB/s | 264 kB 00:00 2024-05-09T06:33:54.917 INFO:teuthology.orchestra.run.smithi190.stdout:(37/156): cephfs-top-19.0.0-2515.g96805dcf.el9. 440 kB/s | 18 kB 00:00 2024-05-09T06:33:54.976 INFO:teuthology.orchestra.run.smithi190.stdout:(38/156): cephadm-19.0.0-2515.g96805dcf.el9.noa 4.9 MB/s | 763 kB 00:00 2024-05-09T06:33:55.121 INFO:teuthology.orchestra.run.smithi003.stdout:(85/156): python3-packaging-20.9-5.el9.noarch.r 102 kB/s | 77 kB 00:00 2024-05-09T06:33:55.155 INFO:teuthology.orchestra.run.smithi003.stdout:(86/156): python3-prettytable-0.7.2-27.el9.noar 61 kB/s | 42 kB 00:00 2024-05-09T06:33:55.618 INFO:teuthology.orchestra.run.smithi190.stdout:(39/156): ceph-test-19.0.0-2515.g96805dcf.el9.x 15 MB/s | 48 MB 00:03 2024-05-09T06:33:55.638 INFO:teuthology.orchestra.run.smithi003.stdout:(87/156): python3-protobuf-3.14.0-13.el9.noarch 309 kB/s | 269 kB 00:00 2024-05-09T06:33:55.880 INFO:teuthology.orchestra.run.smithi003.stdout:(88/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 209 kB/s | 159 kB 00:00 2024-05-09T06:33:55.976 INFO:teuthology.orchestra.run.smithi190.stdout:(40/156): fuse-2.9.9-15.el9.x86_64.rpm 76 kB/s | 80 kB 00:01 2024-05-09T06:33:55.980 INFO:teuthology.orchestra.run.smithi003.stdout:(89/156): python3-pyasn1-modules-0.4.8-6.el9.no 338 kB/s | 279 kB 00:00 2024-05-09T06:33:56.297 INFO:teuthology.orchestra.run.smithi003.stdout:(90/156): python3-pytz-2021.1-5.el9.noarch.rpm 77 kB/s | 51 kB 00:00 2024-05-09T06:33:56.427 INFO:teuthology.orchestra.run.smithi190.stdout:(41/156): libgfortran-11.4.1-3.el9.x86_64.rpm 554 kB/s | 803 kB 00:01 2024-05-09T06:33:56.443 INFO:teuthology.orchestra.run.smithi190.stdout:(42/156): libquadmath-11.4.1-3.el9.x86_64.rpm 245 kB/s | 195 kB 00:00 2024-05-09T06:33:56.598 INFO:teuthology.orchestra.run.smithi003.stdout:(91/156): python3-requests-oauthlib-1.3.0-12.el 78 kB/s | 54 kB 00:00 2024-05-09T06:33:56.852 INFO:teuthology.orchestra.run.smithi190.stdout:(43/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 290 kB/s | 253 kB 00:00 2024-05-09T06:33:56.948 INFO:teuthology.orchestra.run.smithi003.stdout:(92/156): python3-toml-0.10.2-6.el9.noarch.rpm 64 kB/s | 42 kB 00:00 2024-05-09T06:33:57.236 INFO:teuthology.orchestra.run.smithi190.stdout:(44/156): python3-ply-3.11-14.el9.noarch.rpm 135 kB/s | 106 kB 00:00 2024-05-09T06:33:57.282 INFO:teuthology.orchestra.run.smithi003.stdout:(93/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 65 kB/s | 44 kB 00:00 2024-05-09T06:33:57.561 INFO:teuthology.orchestra.run.smithi190.stdout:(45/156): python3-cryptography-36.0.1-4.el9.x86 1.1 MB/s | 1.2 MB 00:01 2024-05-09T06:33:57.623 INFO:teuthology.orchestra.run.smithi003.stdout:(94/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 9.9 MB/s | 16 MB 00:01 2024-05-09T06:33:57.694 INFO:teuthology.orchestra.run.smithi190.stdout:(46/156): python3-pycparser-2.20-6.el9.noarch.r 161 kB/s | 135 kB 00:00 2024-05-09T06:33:57.803 INFO:teuthology.orchestra.run.smithi190.stdout:(47/156): automake-1.16.2-8.el9.noarch.rpm 6.2 MB/s | 689 kB 00:00 2024-05-09T06:33:57.923 INFO:teuthology.orchestra.run.smithi003.stdout:(95/156): qatlib-24.02.0-1.el9.x86_64.rpm 226 kB/s | 221 kB 00:00 2024-05-09T06:33:57.936 INFO:teuthology.orchestra.run.smithi190.stdout:(48/156): bison-3.7.4-5.el9.x86_64.rpm 8.7 MB/s | 943 kB 00:00 2024-05-09T06:33:57.987 INFO:teuthology.orchestra.run.smithi190.stdout:(49/156): python3-requests-2.25.1-8.el9.noarch. 166 kB/s | 125 kB 00:00 2024-05-09T06:33:58.020 INFO:teuthology.orchestra.run.smithi190.stdout:(50/156): boost-program-options-1.75.0-8.el9.x8 1.3 MB/s | 107 kB 00:00 2024-05-09T06:33:58.024 INFO:teuthology.orchestra.run.smithi003.stdout:(96/156): qatlib-service-24.02.0-1.el9.x86_64.r 48 kB/s | 36 kB 00:00 2024-05-09T06:33:58.062 INFO:teuthology.orchestra.run.smithi190.stdout:(51/156): elfutils-libelf-devel-0.190-2.el9.x86 287 kB/s | 21 kB 00:00 2024-05-09T06:33:58.113 INFO:teuthology.orchestra.run.smithi190.stdout:(52/156): flexiblas-3.0.4-8.el9.x86_64.rpm 654 kB/s | 33 kB 00:00 2024-05-09T06:33:58.138 INFO:teuthology.orchestra.run.smithi190.stdout:(53/156): flex-2.6.4-9.el9.x86_64.rpm 2.6 MB/s | 313 kB 00:00 2024-05-09T06:33:58.188 INFO:teuthology.orchestra.run.smithi190.stdout:(54/156): flexiblas-openblas-openmp-3.0.4-8.el9 365 kB/s | 18 kB 00:00 2024-05-09T06:33:58.230 INFO:teuthology.orchestra.run.smithi190.stdout:(55/156): libaio-devel-0.3.111-13.el9.x86_64.rp 266 kB/s | 11 kB 00:00 2024-05-09T06:33:58.291 INFO:teuthology.orchestra.run.smithi003.stdout:(97/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 74 kB/s | 47 kB 00:00 2024-05-09T06:33:58.297 INFO:teuthology.orchestra.run.smithi190.stdout:(56/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.3 MB/s | 160 kB 00:00 2024-05-09T06:33:58.356 INFO:teuthology.orchestra.run.smithi190.stdout:(57/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 12 MB/s | 3.0 MB 00:00 2024-05-09T06:33:58.406 INFO:teuthology.orchestra.run.smithi190.stdout:(58/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 417 kB/s | 45 kB 00:00 2024-05-09T06:33:58.431 INFO:teuthology.orchestra.run.smithi190.stdout:(59/156): python3-urllib3-1.26.5-5.el9.noarch.r 247 kB/s | 215 kB 00:00 2024-05-09T06:33:58.482 INFO:teuthology.orchestra.run.smithi190.stdout:(60/156): librdkafka-1.6.1-102.el9.x86_64.rpm 5.2 MB/s | 662 kB 00:00 2024-05-09T06:33:58.507 INFO:teuthology.orchestra.run.smithi190.stdout:(61/156): libuuid-devel-2.37.4-18.el9.x86_64.rp 338 kB/s | 25 kB 00:00 2024-05-09T06:33:58.549 INFO:teuthology.orchestra.run.smithi190.stdout:(62/156): libtool-2.4.6-45.el9.x86_64.rpm 4.0 MB/s | 582 kB 00:00 2024-05-09T06:33:58.574 INFO:teuthology.orchestra.run.smithi190.stdout:(63/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 515 kB/s | 47 kB 00:00 2024-05-09T06:33:58.608 INFO:teuthology.orchestra.run.smithi190.stdout:(64/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.8 MB/s | 292 kB 00:00 2024-05-09T06:33:58.650 INFO:teuthology.orchestra.run.smithi190.stdout:(65/156): openblas-0.3.21-2.el9.x86_64.rpm 469 kB/s | 35 kB 00:00 2024-05-09T06:33:58.674 INFO:teuthology.orchestra.run.smithi003.stdout:(98/156): uuid-1.6.2-55.el9.x86_64.rpm 89 kB/s | 58 kB 00:00 2024-05-09T06:33:58.675 INFO:teuthology.orchestra.run.smithi190.stdout:(66/156): lua-5.4.4-4.el9.x86_64.rpm 1.5 MB/s | 188 kB 00:00 2024-05-09T06:33:58.783 INFO:teuthology.orchestra.run.smithi003.stdout:(99/156): socat-1.7.4.1-5.el9.x86_64.rpm 355 kB/s | 305 kB 00:00 2024-05-09T06:33:58.808 INFO:teuthology.orchestra.run.smithi190.stdout:(67/156): postgresql-13.14-1.el9.x86_64.rpm 12 MB/s | 1.6 MB 00:00 2024-05-09T06:33:58.901 INFO:teuthology.orchestra.run.smithi190.stdout:(68/156): openblas-openmp-0.3.21-2.el9.x86_64.r 16 MB/s | 4.7 MB 00:00 2024-05-09T06:33:58.959 INFO:teuthology.orchestra.run.smithi190.stdout:(69/156): openssl-devel-3.2.1-1.el9.x86_64.rpm 14 MB/s | 4.4 MB 00:00 2024-05-09T06:33:59.010 INFO:teuthology.orchestra.run.smithi190.stdout:(70/156): postgresql-contrib-13.14-1.el9.x86_64 4.4 MB/s | 911 kB 00:00 2024-05-09T06:33:59.016 INFO:teuthology.orchestra.run.smithi003.stdout:(100/156): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 129 kB/s | 94 kB 00:00 2024-05-09T06:33:59.043 INFO:teuthology.orchestra.run.smithi190.stdout:(71/156): postgresql-private-libs-13.14-1.el9.x 937 kB/s | 133 kB 00:00 2024-05-09T06:33:59.110 INFO:teuthology.orchestra.run.smithi190.stdout:(72/156): python3-attrs-20.3.0-7.el9.noarch.rpm 1.3 MB/s | 87 kB 00:00 2024-05-09T06:33:59.152 INFO:teuthology.orchestra.run.smithi190.stdout:(73/156): protobuf-3.14.0-13.el9.x86_64.rpm 7.1 MB/s | 1.0 MB 00:00 2024-05-09T06:33:59.261 INFO:teuthology.orchestra.run.smithi190.stdout:(74/156): postgresql-server-13.14-1.el9.x86_64. 19 MB/s | 5.8 MB 00:00 2024-05-09T06:33:59.302 INFO:teuthology.orchestra.run.smithi190.stdout:(75/156): python3-devel-3.9.18-3.el9.x86_64.rpm 1.6 MB/s | 244 kB 00:00 2024-05-09T06:33:59.353 INFO:teuthology.orchestra.run.smithi190.stdout:(76/156): python3-jinja2-2.11.3-5.el9.noarch.rp 2.6 MB/s | 248 kB 00:00 2024-05-09T06:33:59.358 INFO:teuthology.orchestra.run.smithi003.stdout:(101/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 93 kB/s | 64 kB 00:00 2024-05-09T06:33:59.378 INFO:teuthology.orchestra.run.smithi190.stdout:(77/156): python3-jsonpatch-1.21-16.el9.noarch. 341 kB/s | 26 kB 00:00 2024-05-09T06:33:59.417 INFO:teuthology.orchestra.run.smithi003.stdout:(102/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 71 kB/s | 45 kB 00:00 2024-05-09T06:33:59.420 INFO:teuthology.orchestra.run.smithi190.stdout:(78/156): python3-jsonpointer-2.0-4.el9.noarch. 286 kB/s | 19 kB 00:00 2024-05-09T06:33:59.504 INFO:teuthology.orchestra.run.smithi190.stdout:(79/156): python3-babel-2.9.1-2.el9.noarch.rpm 15 MB/s | 6.0 MB 00:00 2024-05-09T06:33:59.529 INFO:teuthology.orchestra.run.smithi190.stdout:(80/156): python3-mako-1.1.4-6.el9.noarch.rpm 1.1 MB/s | 172 kB 00:00 2024-05-09T06:33:59.554 INFO:teuthology.orchestra.run.smithi190.stdout:(81/156): python3-markupsafe-1.1.1-12.el9.x86_6 259 kB/s | 35 kB 00:00 2024-05-09T06:33:59.621 INFO:teuthology.orchestra.run.smithi190.stdout:(82/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.7 MB/s | 250 kB 00:00 2024-05-09T06:33:59.643 INFO:teuthology.orchestra.run.smithi003.stdout:(103/156): lua-devel-5.4.4-4.el9.x86_64.rpm 37 kB/s | 22 kB 00:00 2024-05-09T06:33:59.654 INFO:teuthology.orchestra.run.smithi190.stdout:(83/156): python3-oauthlib-3.1.1-5.el9.noarch.r 2.2 MB/s | 222 kB 00:00 2024-05-09T06:33:59.705 INFO:teuthology.orchestra.run.smithi190.stdout:(84/156): python3-packaging-20.9-5.el9.noarch.r 924 kB/s | 77 kB 00:00 2024-05-09T06:33:59.730 INFO:teuthology.orchestra.run.smithi190.stdout:(85/156): python3-prettytable-0.7.2-27.el9.noar 559 kB/s | 42 kB 00:00 2024-05-09T06:33:59.759 INFO:teuthology.orchestra.run.smithi003.stdout:(104/156): abseil-cpp-20211102.0-3.el9.x86_64.r 1.6 MB/s | 548 kB 00:00 2024-05-09T06:33:59.797 INFO:teuthology.orchestra.run.smithi190.stdout:(86/156): python3-protobuf-3.14.0-13.el9.noarch 2.9 MB/s | 269 kB 00:00 2024-05-09T06:33:59.822 INFO:teuthology.orchestra.run.smithi190.stdout:(87/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.7 MB/s | 159 kB 00:00 2024-05-09T06:33:59.898 INFO:teuthology.orchestra.run.smithi190.stdout:(88/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 13 MB/s | 5.1 MB 00:00 2024-05-09T06:33:59.948 INFO:teuthology.orchestra.run.smithi190.stdout:(89/156): python3-pyasn1-modules-0.4.8-6.el9.no 1.8 MB/s | 279 kB 00:00 2024-05-09T06:33:59.960 INFO:teuthology.orchestra.run.smithi003.stdout:(105/156): grpc-data-1.46.7-10.el9.noarch.rpm 97 kB/s | 19 kB 00:00 2024-05-09T06:33:59.990 INFO:teuthology.orchestra.run.smithi190.stdout:(90/156): python3-pytz-2021.1-5.el9.noarch.rpm 305 kB/s | 51 kB 00:00 2024-05-09T06:34:00.015 INFO:teuthology.orchestra.run.smithi190.stdout:(91/156): python3-requests-oauthlib-1.3.0-12.el 457 kB/s | 54 kB 00:00 2024-05-09T06:34:00.066 INFO:teuthology.orchestra.run.smithi190.stdout:(92/156): python3-toml-0.10.2-6.el9.noarch.rpm 552 kB/s | 42 kB 00:00 2024-05-09T06:34:00.091 INFO:teuthology.orchestra.run.smithi190.stdout:(93/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 586 kB/s | 44 kB 00:00 2024-05-09T06:34:00.141 INFO:teuthology.orchestra.run.smithi190.stdout:(94/156): qatlib-service-24.02.0-1.el9.x86_64.r 721 kB/s | 36 kB 00:00 2024-05-09T06:34:00.167 INFO:teuthology.orchestra.run.smithi190.stdout:(95/156): qatlib-24.02.0-1.el9.x86_64.rpm 2.1 MB/s | 221 kB 00:00 2024-05-09T06:34:00.194 INFO:teuthology.orchestra.run.smithi003.stdout:(106/156): libarrow-9.0.0-11.el9.x86_64.rpm 19 MB/s | 4.4 MB 00:00 2024-05-09T06:34:00.217 INFO:teuthology.orchestra.run.smithi190.stdout:(96/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 629 kB/s | 47 kB 00:00 2024-05-09T06:34:00.219 INFO:teuthology.orchestra.run.smithi003.stdout:(107/156): gperftools-libs-2.9.1-2.el9.x86_64.r 536 kB/s | 309 kB 00:00 2024-05-09T06:34:00.253 INFO:teuthology.orchestra.run.smithi003.stdout:(108/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 446 kB/s | 26 kB 00:00 2024-05-09T06:34:00.276 INFO:teuthology.orchestra.run.smithi190.stdout:(97/156): socat-1.7.4.1-5.el9.x86_64.rpm 2.7 MB/s | 305 kB 00:00 2024-05-09T06:34:00.301 INFO:teuthology.orchestra.run.smithi190.stdout:(98/156): uuid-1.6.2-55.el9.x86_64.rpm 692 kB/s | 58 kB 00:00 2024-05-09T06:34:00.311 INFO:teuthology.orchestra.run.smithi003.stdout:(109/156): libunwind-1.6.2-1.el9.x86_64.rpm 1.1 MB/s | 67 kB 00:00 2024-05-09T06:34:00.336 INFO:teuthology.orchestra.run.smithi003.stdout:(110/156): liboath-2.6.7-2.el9.x86_64.rpm 414 kB/s | 49 kB 00:00 2024-05-09T06:34:00.359 INFO:teuthology.orchestra.run.smithi190.stdout:(99/156): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.1 MB/s | 94 kB 00:00 2024-05-09T06:34:00.378 INFO:teuthology.orchestra.run.smithi003.stdout:(111/156): luarocks-3.9.2-1.el9.noarch.rpm 2.2 MB/s | 151 kB 00:00 2024-05-09T06:34:00.384 INFO:teuthology.orchestra.run.smithi190.stdout:(100/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 762 kB/s | 64 kB 00:00 2024-05-09T06:34:00.412 INFO:teuthology.orchestra.run.smithi003.stdout:(112/156): protobuf-compiler-3.14.0-13.el9.x86_ 820 kB/s | 863 kB 00:01 2024-05-09T06:34:00.443 INFO:teuthology.orchestra.run.smithi190.stdout:(101/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 536 kB/s | 45 kB 00:00 2024-05-09T06:34:00.454 INFO:teuthology.orchestra.run.smithi003.stdout:(113/156): python3-asyncssh-2.13.2-5.el9.noarch 7.1 MB/s | 548 kB 00:00 2024-05-09T06:34:00.512 INFO:teuthology.orchestra.run.smithi003.stdout:(114/156): python3-cachetools-4.2.4-1.el9.noarc 552 kB/s | 32 kB 00:00 2024-05-09T06:34:00.571 INFO:teuthology.orchestra.run.smithi003.stdout:(115/156): python3-certifi-2021.10.8-2.el9.noar 257 kB/s | 15 kB 00:00 2024-05-09T06:34:00.597 INFO:teuthology.orchestra.run.smithi003.stdout:(116/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 236 kB/s | 43 kB 00:00 2024-05-09T06:34:00.619 INFO:teuthology.orchestra.run.smithi190.stdout:(102/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 24 MB/s | 16 MB 00:00 2024-05-09T06:34:00.630 INFO:teuthology.orchestra.run.smithi003.stdout:(117/156): python3-cheroot-8.6.0-4.el9.noarch.r 2.9 MB/s | 172 kB 00:00 2024-05-09T06:34:00.644 INFO:teuthology.orchestra.run.smithi190.stdout:(103/156): lua-devel-5.4.4-4.el9.x86_64.rpm 86 kB/s | 22 kB 00:00 2024-05-09T06:34:00.689 INFO:teuthology.orchestra.run.smithi003.stdout:(118/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 5.1 MB/s | 303 kB 00:00 2024-05-09T06:34:00.747 INFO:teuthology.orchestra.run.smithi003.stdout:(119/156): python3-colorama-0.4.4-12.el9.noarch 606 kB/s | 35 kB 00:00 2024-05-09T06:34:00.773 INFO:teuthology.orchestra.run.smithi003.stdout:(120/156): python3-cherrypy-18.6.1-2.el9.noarch 2.0 MB/s | 358 kB 00:00 2024-05-09T06:34:00.806 INFO:teuthology.orchestra.run.smithi003.stdout:(121/156): python3-google-auth-2.29.0-1.el9.noa 3.4 MB/s | 206 kB 00:00 2024-05-09T06:34:00.865 INFO:teuthology.orchestra.run.smithi003.stdout:(122/156): python3-grpcio-tools-1.46.7-10.el9.x 2.4 MB/s | 144 kB 00:00 2024-05-09T06:34:00.869 INFO:teuthology.orchestra.run.smithi190.stdout:(104/156): gperftools-libs-2.9.1-2.el9.x86_64.r 1.3 MB/s | 309 kB 00:00 2024-05-09T06:34:00.903 INFO:teuthology.orchestra.run.smithi190.stdout:(105/156): abseil-cpp-20211102.0-3.el9.x86_64.r 1.9 MB/s | 548 kB 00:00 2024-05-09T06:34:00.919 INFO:teuthology.orchestra.run.smithi190.stdout:(106/156): grpc-data-1.46.7-10.el9.noarch.rpm 392 kB/s | 19 kB 00:00 2024-05-09T06:34:00.923 INFO:teuthology.orchestra.run.smithi003.stdout:(123/156): python3-jaraco-8.2.1-3.el9.noarch.rp 184 kB/s | 11 kB 00:00 2024-05-09T06:34:00.953 INFO:teuthology.orchestra.run.smithi190.stdout:(107/156): protobuf-compiler-3.14.0-13.el9.x86_ 1.7 MB/s | 863 kB 00:00 2024-05-09T06:34:00.974 INFO:teuthology.orchestra.run.smithi003.stdout:(124/156): python3-grpcio-1.46.7-10.el9.x86_64. 10 MB/s | 2.0 MB 00:00 2024-05-09T06:34:01.007 INFO:teuthology.orchestra.run.smithi003.stdout:(125/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 1.2 MB/s | 839 kB 00:00 2024-05-09T06:34:01.032 INFO:teuthology.orchestra.run.smithi003.stdout:(126/156): python3-jaraco-classes-3.2.1-5.el9.n 162 kB/s | 18 kB 00:00 2024-05-09T06:34:01.045 INFO:teuthology.orchestra.run.smithi190.stdout:(108/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 207 kB/s | 26 kB 00:00 2024-05-09T06:34:01.058 INFO:teuthology.orchestra.run.smithi003.stdout:(127/156): python3-jaraco-collections-3.0.0-8.e 277 kB/s | 23 kB 00:00 2024-05-09T06:34:01.070 INFO:teuthology.orchestra.run.smithi190.stdout:(109/156): liboath-2.6.7-2.el9.x86_64.rpm 418 kB/s | 49 kB 00:00 2024-05-09T06:34:01.083 INFO:teuthology.orchestra.run.smithi003.stdout:(128/156): python3-jaraco-functools-3.5.0-2.el9 258 kB/s | 19 kB 00:00 2024-05-09T06:34:01.108 INFO:teuthology.orchestra.run.smithi003.stdout:(129/156): python3-jaraco-text-3.2.0-6.el9.noar 259 kB/s | 20 kB 00:00 2024-05-09T06:34:01.137 INFO:teuthology.orchestra.run.smithi190.stdout:(110/156): libarrow-9.0.0-11.el9.x86_64.rpm 19 MB/s | 4.4 MB 00:00 2024-05-09T06:34:01.142 INFO:teuthology.orchestra.run.smithi003.stdout:(130/156): python3-logutils-0.3.5-21.el9.noarch 786 kB/s | 46 kB 00:00 2024-05-09T06:34:01.163 INFO:teuthology.orchestra.run.smithi190.stdout:(111/156): libunwind-1.6.2-1.el9.x86_64.rpm 576 kB/s | 67 kB 00:00 2024-05-09T06:34:01.188 INFO:teuthology.orchestra.run.smithi190.stdout:(112/156): luarocks-3.9.2-1.el9.noarch.rpm 1.3 MB/s | 151 kB 00:00 2024-05-09T06:34:01.201 INFO:teuthology.orchestra.run.smithi003.stdout:(131/156): python3-more-itertools-8.12.0-2.el9. 857 kB/s | 79 kB 00:00 2024-05-09T06:34:01.226 INFO:teuthology.orchestra.run.smithi003.stdout:(132/156): python3-natsort-7.1.1-5.el9.noarch.r 691 kB/s | 58 kB 00:00 2024-05-09T06:34:01.238 INFO:teuthology.orchestra.run.smithi190.stdout:(113/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 8.2 MB/s | 839 kB 00:00 2024-05-09T06:34:01.260 INFO:teuthology.orchestra.run.smithi003.stdout:(133/156): python3-pecan-1.4.2-3.el9.noarch.rpm 4.6 MB/s | 272 kB 00:00 2024-05-09T06:34:01.285 INFO:teuthology.orchestra.run.smithi003.stdout:(134/156): python3-portend-3.1.0-2.el9.noarch.r 281 kB/s | 16 kB 00:00 2024-05-09T06:34:01.288 INFO:teuthology.orchestra.run.smithi190.stdout:(114/156): python3-asyncssh-2.13.2-5.el9.noarch 5.4 MB/s | 548 kB 00:00 2024-05-09T06:34:01.305 INFO:teuthology.orchestra.run.smithi190.stdout:(115/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 374 kB/s | 43 kB 00:00 2024-05-09T06:34:01.319 INFO:teuthology.orchestra.run.smithi003.stdout:(135/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.5 MB/s | 90 kB 00:00 2024-05-09T06:34:01.330 INFO:teuthology.orchestra.run.smithi190.stdout:(116/156): python3-cachetools-4.2.4-1.el9.noarc 348 kB/s | 32 kB 00:00 2024-05-09T06:34:01.344 INFO:teuthology.orchestra.run.smithi003.stdout:(136/156): python3-pyperclip-1.8.2-2.el9.noarch 483 kB/s | 28 kB 00:00 2024-05-09T06:34:01.347 INFO:teuthology.orchestra.run.smithi190.stdout:(117/156): python3-certifi-2021.10.8-2.el9.noar 255 kB/s | 15 kB 00:00 2024-05-09T06:34:01.377 INFO:teuthology.orchestra.run.smithi003.stdout:(137/156): python3-repoze-lru-0.7-16.el9.noarch 528 kB/s | 31 kB 00:00 2024-05-09T06:34:01.389 INFO:teuthology.orchestra.run.smithi190.stdout:(118/156): python3-cheroot-8.6.0-4.el9.noarch.r 2.9 MB/s | 172 kB 00:00 2024-05-09T06:34:01.403 INFO:teuthology.orchestra.run.smithi003.stdout:(138/156): python3-routes-2.5.1-5.el9.noarch.rp 3.1 MB/s | 188 kB 00:00 2024-05-09T06:34:01.415 INFO:teuthology.orchestra.run.smithi190.stdout:(119/156): python3-cherrypy-18.6.1-2.el9.noarch 4.2 MB/s | 358 kB 00:00 2024-05-09T06:34:01.436 INFO:teuthology.orchestra.run.smithi003.stdout:(139/156): python3-rsa-4.9-2.el9.noarch.rpm 1.0 MB/s | 59 kB 00:00 2024-05-09T06:34:01.440 INFO:teuthology.orchestra.run.smithi190.stdout:(120/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.2 MB/s | 303 kB 00:00 2024-05-09T06:34:01.462 INFO:teuthology.orchestra.run.smithi003.stdout:(140/156): python3-tempora-5.0.0-2.el9.noarch.r 611 kB/s | 36 kB 00:00 2024-05-09T06:34:01.465 INFO:teuthology.orchestra.run.smithi190.stdout:(121/156): python3-colorama-0.4.4-12.el9.noarch 474 kB/s | 35 kB 00:00 2024-05-09T06:34:01.495 INFO:teuthology.orchestra.run.smithi003.stdout:(141/156): python3-typing-extensions-4.4.0-2.el 873 kB/s | 51 kB 00:00 2024-05-09T06:34:01.507 INFO:teuthology.orchestra.run.smithi190.stdout:(122/156): python3-google-auth-2.29.0-1.el9.noa 3.0 MB/s | 206 kB 00:00 2024-05-09T06:34:01.521 INFO:teuthology.orchestra.run.smithi003.stdout:(142/156): python3-webob-1.8.7-6.el9.noarch.rpm 3.8 MB/s | 230 kB 00:00 2024-05-09T06:34:01.554 INFO:teuthology.orchestra.run.smithi003.stdout:(143/156): python3-websocket-client-1.2.3-2.el9 1.5 MB/s | 90 kB 00:00 2024-05-09T06:34:01.596 INFO:teuthology.orchestra.run.smithi003.stdout:(144/156): python3-kubernetes-26.1.0-2.el9.noar 1.9 MB/s | 1.0 MB 00:00 2024-05-09T06:34:01.599 INFO:teuthology.orchestra.run.smithi190.stdout:(123/156): python3-grpcio-1.46.7-10.el9.x86_64. 13 MB/s | 2.0 MB 00:00 2024-05-09T06:34:01.621 INFO:teuthology.orchestra.run.smithi003.stdout:(145/156): python3-zc-lockfile-2.0-10.el9.noarc 298 kB/s | 20 kB 00:00 2024-05-09T06:34:01.624 INFO:teuthology.orchestra.run.smithi190.stdout:(124/156): python3-grpcio-tools-1.46.7-10.el9.x 909 kB/s | 144 kB 00:00 2024-05-09T06:34:01.647 INFO:teuthology.orchestra.run.smithi003.stdout:(146/156): python3-werkzeug-2.0.3-3.el9.1.noarc 3.3 MB/s | 427 kB 00:00 2024-05-09T06:34:01.649 INFO:teuthology.orchestra.run.smithi190.stdout:(125/156): python3-jaraco-8.2.1-3.el9.noarch.rp 75 kB/s | 11 kB 00:00 2024-05-09T06:34:01.675 INFO:teuthology.orchestra.run.smithi190.stdout:(126/156): python3-jaraco-classes-3.2.1-5.el9.n 235 kB/s | 18 kB 00:00 2024-05-09T06:34:01.701 INFO:teuthology.orchestra.run.smithi190.stdout:(127/156): python3-jaraco-collections-3.0.0-8.e 461 kB/s | 23 kB 00:00 2024-05-09T06:34:01.705 INFO:teuthology.orchestra.run.smithi003.stdout:(147/156): thrift-0.15.0-2.el9.x86_64.rpm 19 MB/s | 1.6 MB 00:00 2024-05-09T06:34:01.725 INFO:teuthology.orchestra.run.smithi190.stdout:(128/156): python3-jaraco-functools-3.5.0-2.el9 258 kB/s | 19 kB 00:00 2024-05-09T06:34:01.731 INFO:teuthology.orchestra.run.smithi003.stdout:(148/156): re2-20211101-3.el9.x86_64.rpm 1.4 MB/s | 192 kB 00:00 2024-05-09T06:34:01.750 INFO:teuthology.orchestra.run.smithi190.stdout:(129/156): python3-jaraco-text-3.2.0-6.el9.noar 261 kB/s | 20 kB 00:00 2024-05-09T06:34:01.784 INFO:teuthology.orchestra.run.smithi190.stdout:(130/156): python3-kubernetes-26.1.0-2.el9.noar 12 MB/s | 1.0 MB 00:00 2024-05-09T06:34:01.834 INFO:teuthology.orchestra.run.smithi190.stdout:(131/156): python3-logutils-0.3.5-21.el9.noarch 567 kB/s | 46 kB 00:00 2024-05-09T06:34:01.860 INFO:teuthology.orchestra.run.smithi190.stdout:(132/156): python3-more-itertools-8.12.0-2.el9. 737 kB/s | 79 kB 00:00 2024-05-09T06:34:01.864 INFO:teuthology.orchestra.run.smithi003.stdout:(149/156): NetworkManager-1.47.5-1.el9.x86_64.r 17 MB/s | 2.3 MB 00:00 2024-05-09T06:34:01.885 INFO:teuthology.orchestra.run.smithi190.stdout:(133/156): python3-natsort-7.1.1-5.el9.noarch.r 574 kB/s | 58 kB 00:00 2024-05-09T06:34:01.910 INFO:teuthology.orchestra.run.smithi190.stdout:(134/156): python3-pecan-1.4.2-3.el9.noarch.rpm 3.5 MB/s | 272 kB 00:00 2024-05-09T06:34:01.935 INFO:teuthology.orchestra.run.smithi190.stdout:(135/156): python3-portend-3.1.0-2.el9.noarch.r 329 kB/s | 16 kB 00:00 2024-05-09T06:34:01.961 INFO:teuthology.orchestra.run.smithi190.stdout:(136/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.2 MB/s | 90 kB 00:00 2024-05-09T06:34:01.973 INFO:teuthology.orchestra.run.smithi003.stdout:(150/156): librados2-19.0.0-2515.g96805dcf.el9. 10 MB/s | 3.4 MB 00:00 2024-05-09T06:34:01.986 INFO:teuthology.orchestra.run.smithi190.stdout:(137/156): python3-pyperclip-1.8.2-2.el9.noarch 375 kB/s | 28 kB 00:00 2024-05-09T06:34:02.011 INFO:teuthology.orchestra.run.smithi190.stdout:(138/156): python3-repoze-lru-0.7-16.el9.noarch 409 kB/s | 31 kB 00:00 2024-05-09T06:34:02.036 INFO:teuthology.orchestra.run.smithi190.stdout:(139/156): python3-routes-2.5.1-5.el9.noarch.rp 2.5 MB/s | 188 kB 00:00 2024-05-09T06:34:02.040 INFO:teuthology.orchestra.run.smithi003.stdout:(151/156): NetworkManager-libnm-1.47.5-1.el9.x8 10 MB/s | 1.8 MB 00:00 2024-05-09T06:34:02.061 INFO:teuthology.orchestra.run.smithi190.stdout:(140/156): python3-rsa-4.9-2.el9.noarch.rpm 1.2 MB/s | 59 kB 00:00 2024-05-09T06:34:02.065 INFO:teuthology.orchestra.run.smithi003.stdout:(152/156): NetworkManager-team-1.47.5-1.el9.x86 454 kB/s | 42 kB 00:00 2024-05-09T06:34:02.087 INFO:teuthology.orchestra.run.smithi190.stdout:(141/156): python3-tempora-5.0.0-2.el9.noarch.r 478 kB/s | 36 kB 00:00 2024-05-09T06:34:02.112 INFO:teuthology.orchestra.run.smithi190.stdout:(142/156): python3-typing-extensions-4.4.0-2.el 677 kB/s | 51 kB 00:00 2024-05-09T06:34:02.132 INFO:teuthology.orchestra.run.smithi003.stdout:(153/156): NetworkManager-tui-1.47.5-1.el9.x86_ 2.6 MB/s | 248 kB 00:00 2024-05-09T06:34:02.137 INFO:teuthology.orchestra.run.smithi190.stdout:(143/156): python3-webob-1.8.7-6.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-05-09T06:34:02.162 INFO:teuthology.orchestra.run.smithi190.stdout:(144/156): python3-websocket-client-1.2.3-2.el9 1.8 MB/s | 90 kB 00:00 2024-05-09T06:34:02.191 INFO:teuthology.orchestra.run.smithi003.stdout:(154/156): librbd1-19.0.0-2515.g96805dcf.el9.x8 6.5 MB/s | 3.1 MB 00:00 2024-05-09T06:34:02.196 INFO:teuthology.orchestra.run.smithi190.stdout:(145/156): python3-werkzeug-2.0.3-3.el9.1.noarc 5.0 MB/s | 427 kB 00:00 2024-05-09T06:34:02.221 INFO:teuthology.orchestra.run.smithi190.stdout:(146/156): python3-zc-lockfile-2.0-10.el9.noarc 239 kB/s | 20 kB 00:00 2024-05-09T06:34:02.247 INFO:teuthology.orchestra.run.smithi190.stdout:(147/156): re2-20211101-3.el9.x86_64.rpm 2.3 MB/s | 192 kB 00:00 2024-05-09T06:34:02.249 INFO:teuthology.orchestra.run.smithi003.stdout:(155/156): openssl-3.2.1-1.el9.x86_64.rpm 7.3 MB/s | 1.3 MB 00:00 2024-05-09T06:34:02.314 INFO:teuthology.orchestra.run.smithi190.stdout:(148/156): thrift-0.15.0-2.el9.x86_64.rpm 17 MB/s | 1.6 MB 00:00 2024-05-09T06:34:02.349 INFO:teuthology.orchestra.run.smithi003.stdout:(156/156): openssl-libs-3.2.1-1.el9.x86_64.rpm 11 MB/s | 2.4 MB 00:00 2024-05-09T06:34:02.353 INFO:teuthology.orchestra.run.smithi003.stdout:-------------------------------------------------------------------------------- 2024-05-09T06:34:02.353 INFO:teuthology.orchestra.run.smithi003.stdout:Total 9.1 MB/s | 225 MB 00:24 2024-05-09T06:34:02.631 INFO:teuthology.orchestra.run.smithi190.stdout:(149/156): librbd1-19.0.0-2515.g96805dcf.el9.x8 8.2 MB/s | 3.1 MB 00:00 2024-05-09T06:34:02.764 INFO:teuthology.orchestra.run.smithi190.stdout:(150/156): librados2-19.0.0-2515.g96805dcf.el9. 6.2 MB/s | 3.4 MB 00:00 2024-05-09T06:34:03.381 INFO:teuthology.orchestra.run.smithi190.stdout:(151/156): NetworkManager-team-1.47.5-1.el9.x86 68 kB/s | 42 kB 00:00 2024-05-09T06:34:03.448 INFO:teuthology.orchestra.run.smithi190.stdout:(152/156): NetworkManager-1.47.5-1.el9.x86_64.r 2.0 MB/s | 2.3 MB 00:01 2024-05-09T06:34:03.604 INFO:teuthology.orchestra.run.smithi003.stdout:Running transaction check 2024-05-09T06:34:03.673 INFO:teuthology.orchestra.run.smithi190.stdout:(153/156): NetworkManager-libnm-1.47.5-1.el9.x8 1.8 MB/s | 1.8 MB 00:01 2024-05-09T06:34:03.775 INFO:teuthology.orchestra.run.smithi003.stdout:Transaction check succeeded. 2024-05-09T06:34:03.775 INFO:teuthology.orchestra.run.smithi003.stdout:Running transaction test 2024-05-09T06:34:04.207 INFO:teuthology.orchestra.run.smithi190.stdout:(154/156): NetworkManager-tui-1.47.5-1.el9.x86_ 301 kB/s | 248 kB 00:00 2024-05-09T06:34:04.465 INFO:teuthology.orchestra.run.smithi190.stdout:(155/156): openssl-3.2.1-1.el9.x86_64.rpm 1.3 MB/s | 1.3 MB 00:01 2024-05-09T06:34:04.757 INFO:teuthology.orchestra.run.smithi190.stdout:(156/156): openssl-libs-3.2.1-1.el9.x86_64.rpm 2.3 MB/s | 2.4 MB 00:01 2024-05-09T06:34:04.764 INFO:teuthology.orchestra.run.smithi190.stdout:-------------------------------------------------------------------------------- 2024-05-09T06:34:04.764 INFO:teuthology.orchestra.run.smithi190.stdout:Total 14 MB/s | 225 MB 00:15 2024-05-09T06:34:05.978 INFO:teuthology.orchestra.run.smithi190.stdout:Running transaction check 2024-05-09T06:34:06.130 INFO:teuthology.orchestra.run.smithi190.stdout:Transaction check succeeded. 2024-05-09T06:34:06.130 INFO:teuthology.orchestra.run.smithi190.stdout:Running transaction test 2024-05-09T06:34:06.152 INFO:teuthology.orchestra.run.smithi003.stdout:Transaction test succeeded. 2024-05-09T06:34:06.153 INFO:teuthology.orchestra.run.smithi003.stdout:Running transaction 2024-05-09T06:34:08.442 INFO:teuthology.orchestra.run.smithi190.stdout:Transaction test succeeded. 2024-05-09T06:34:08.443 INFO:teuthology.orchestra.run.smithi190.stdout:Running transaction 2024-05-09T06:34:12.560 INFO:teuthology.orchestra.run.smithi003.stdout: Preparing : 1/1 2024-05-09T06:34:12.638 INFO:teuthology.orchestra.run.smithi003.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/164 2024-05-09T06:34:12.676 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-05-09T06:34:12.732 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-05-09T06:34:12.772 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-05-09T06:34:12.797 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-05-09T06:34:12.848 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-05-09T06:34:13.202 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-05-09T06:34:13.306 INFO:teuthology.orchestra.run.smithi003.stdout: Upgrading : librados2-2:19.0.0-2515.g96805dcf.el9.x86_64 8/164 2024-05-09T06:34:13.431 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: librados2-2:19.0.0-2515.g96805dcf.el9.x86_64 8/164 2024-05-09T06:34:13.457 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : libcephfs2-2:19.0.0-2515.g96805dcf.el9.x86_64 9/164 2024-05-09T06:34:13.550 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: libcephfs2-2:19.0.0-2515.g96805dcf.el9.x86_64 9/164 2024-05-09T06:34:13.578 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-rados-2:19.0.0-2515.g96805dcf.el9.x86_64 10/164 2024-05-09T06:34:13.764 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/164 2024-05-09T06:34:13.818 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/164 2024-05-09T06:34:13.845 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : libcephsqlite-2:19.0.0-2515.g96805dcf.el9.x86_64 13/164 2024-05-09T06:34:13.950 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2515.g96805dcf.el9.x86_64 13/164 2024-05-09T06:34:13.979 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : libradosstriper1-2:19.0.0-2515.g96805dcf.el9.x86 14/164 2024-05-09T06:34:14.034 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2515.g96805dcf.el9.x86 14/164 2024-05-09T06:34:14.063 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-09T06:34:14.087 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-09T06:34:14.275 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-09T06:34:14.450 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-09T06:34:14.567 INFO:teuthology.orchestra.run.smithi190.stdout: Preparing : 1/1 2024-05-09T06:34:14.594 INFO:teuthology.orchestra.run.smithi003.stdout: Upgrading : NetworkManager-libnm-1:1.47.5-1.el9.x86_64 17/164 2024-05-09T06:34:14.637 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.5-1.el9.x86_64 17/164 2024-05-09T06:34:14.654 INFO:teuthology.orchestra.run.smithi190.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/164 2024-05-09T06:34:14.692 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-05-09T06:34:14.756 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-05-09T06:34:14.757 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-09T06:34:14.795 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-05-09T06:34:14.821 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-05-09T06:34:14.838 INFO:teuthology.orchestra.run.smithi003.stdout: Upgrading : NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-09T06:34:14.873 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-05-09T06:34:15.059 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-09T06:34:15.183 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : re2-1:20211101-3.el9.x86_64 19/164 2024-05-09T06:34:15.204 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-05-09T06:34:15.277 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-05-09T06:34:15.297 INFO:teuthology.orchestra.run.smithi190.stdout: Upgrading : librados2-2:19.0.0-2515.g96805dcf.el9.x86_64 8/164 2024-05-09T06:34:15.312 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-05-09T06:34:15.355 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-05-09T06:34:15.428 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: librados2-2:19.0.0-2515.g96805dcf.el9.x86_64 8/164 2024-05-09T06:34:15.430 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-05-09T06:34:15.456 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : libcephfs2-2:19.0.0-2515.g96805dcf.el9.x86_64 9/164 2024-05-09T06:34:15.471 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-05-09T06:34:15.514 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 25/164 2024-05-09T06:34:15.532 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: libcephfs2-2:19.0.0-2515.g96805dcf.el9.x86_64 9/164 2024-05-09T06:34:15.549 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 26/164 2024-05-09T06:34:15.560 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-rados-2:19.0.0-2515.g96805dcf.el9.x86_64 10/164 2024-05-09T06:34:15.591 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : lua-5.4.4-4.el9.x86_64 27/164 2024-05-09T06:34:15.663 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 28/164 2024-05-09T06:34:15.714 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 29/164 2024-05-09T06:34:15.744 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 30/164 2024-05-09T06:34:15.744 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/164 2024-05-09T06:34:15.787 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 31/164 2024-05-09T06:34:15.808 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/164 2024-05-09T06:34:15.835 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : libcephsqlite-2:19.0.0-2515.g96805dcf.el9.x86_64 13/164 2024-05-09T06:34:15.865 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 32/164 2024-05-09T06:34:15.906 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-ceph-common-2:19.0.0-2515.g96805dcf.el9. 33/164 2024-05-09T06:34:15.939 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2515.g96805dcf.el9.x86_64 13/164 2024-05-09T06:34:15.950 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-ceph-argparse-2:19.0.0-2515.g96805dcf.el 34/164 2024-05-09T06:34:15.961 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : libradosstriper1-2:19.0.0-2515.g96805dcf.el9.x86 14/164 2024-05-09T06:34:15.978 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-cephfs-2:19.0.0-2515.g96805dcf.el9.x86_6 35/164 2024-05-09T06:34:16.024 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2515.g96805dcf.el9.x86 14/164 2024-05-09T06:34:16.046 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-09T06:34:16.051 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 36/164 2024-05-09T06:34:16.067 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-09T06:34:16.093 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : luarocks-3.9.2-1.el9.noarch 37/164 2024-05-09T06:34:16.170 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 38/164 2024-05-09T06:34:16.273 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-09T06:34:16.315 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 39/164 2024-05-09T06:34:16.372 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 40/164 2024-05-09T06:34:16.434 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-09T06:34:16.585 INFO:teuthology.orchestra.run.smithi190.stdout: Upgrading : NetworkManager-libnm-1:1.47.5-1.el9.x86_64 17/164 2024-05-09T06:34:16.645 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.5-1.el9.x86_64 17/164 2024-05-09T06:34:16.770 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-09T06:34:16.838 INFO:teuthology.orchestra.run.smithi190.stdout: Upgrading : NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-09T06:34:16.925 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-rsa-4.9-2.el9.noarch 41/164 2024-05-09T06:34:17.010 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 42/164 2024-05-09T06:34:17.026 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-09T06:34:17.045 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 43/164 2024-05-09T06:34:17.088 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 44/164 2024-05-09T06:34:17.134 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : librados-devel-2:19.0.0-2515.g96805dcf.el9.x86_6 45/164 2024-05-09T06:34:17.135 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : re2-1:20211101-3.el9.x86_64 19/164 2024-05-09T06:34:17.177 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 46/164 2024-05-09T06:34:17.220 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-05-09T06:34:17.262 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-05-09T06:34:17.303 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-05-09T06:34:17.353 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 47/164 2024-05-09T06:34:17.367 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-05-09T06:34:17.402 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-05-09T06:34:17.412 INFO:teuthology.orchestra.run.smithi003.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 48/164 2024-05-09T06:34:17.445 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 25/164 2024-05-09T06:34:17.468 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 49/164 2024-05-09T06:34:17.479 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 26/164 2024-05-09T06:34:17.523 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : lua-5.4.4-4.el9.x86_64 27/164 2024-05-09T06:34:17.541 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 50/164 2024-05-09T06:34:17.579 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 51/164 2024-05-09T06:34:17.591 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 28/164 2024-05-09T06:34:17.615 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 52/164 2024-05-09T06:34:17.644 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 29/164 2024-05-09T06:34:17.663 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 53/164 2024-05-09T06:34:17.676 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 30/164 2024-05-09T06:34:17.698 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 54/164 2024-05-09T06:34:17.719 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 31/164 2024-05-09T06:34:17.753 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 55/164 2024-05-09T06:34:17.792 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 56/164 2024-05-09T06:34:17.795 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 32/164 2024-05-09T06:34:17.841 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-ceph-common-2:19.0.0-2515.g96805dcf.el9. 33/164 2024-05-09T06:34:17.892 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-ceph-argparse-2:19.0.0-2515.g96805dcf.el 34/164 2024-05-09T06:34:17.919 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-cephfs-2:19.0.0-2515.g96805dcf.el9.x86_6 35/164 2024-05-09T06:34:17.934 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 57/164 2024-05-09T06:34:17.981 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 58/164 2024-05-09T06:34:17.987 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 36/164 2024-05-09T06:34:18.011 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 59/164 2024-05-09T06:34:18.042 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : luarocks-3.9.2-1.el9.noarch 37/164 2024-05-09T06:34:18.053 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 60/164 2024-05-09T06:34:18.116 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 38/164 2024-05-09T06:34:18.150 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-05-09T06:34:18.186 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 62/164 2024-05-09T06:34:18.227 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-05-09T06:34:18.250 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 39/164 2024-05-09T06:34:18.259 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 64/164 2024-05-09T06:34:18.284 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-05-09T06:34:18.302 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 40/164 2024-05-09T06:34:18.369 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-05-09T06:34:18.499 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-05-09T06:34:18.542 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-05-09T06:34:18.570 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-05-09T06:34:18.612 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-05-09T06:34:18.651 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-05-09T06:34:18.691 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-05-09T06:34:18.721 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-05-09T06:34:18.755 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-05-09T06:34:18.789 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-05-09T06:34:18.829 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-rsa-4.9-2.el9.noarch 41/164 2024-05-09T06:34:18.831 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-05-09T06:34:18.873 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-05-09T06:34:18.914 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 42/164 2024-05-09T06:34:18.945 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 43/164 2024-05-09T06:34:18.963 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-05-09T06:34:18.987 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 44/164 2024-05-09T06:34:19.013 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-05-09T06:34:19.025 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : librados-devel-2:19.0.0-2515.g96805dcf.el9.x86_6 45/164 2024-05-09T06:34:19.043 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-05-09T06:34:19.067 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 46/164 2024-05-09T06:34:19.091 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-05-09T06:34:19.201 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-packaging-20.9-5.el9.noarch 82/164 2024-05-09T06:34:19.227 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 47/164 2024-05-09T06:34:19.262 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/164 2024-05-09T06:34:19.294 INFO:teuthology.orchestra.run.smithi190.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 48/164 2024-05-09T06:34:19.349 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 49/164 2024-05-09T06:34:19.422 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 50/164 2024-05-09T06:34:19.454 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 51/164 2024-05-09T06:34:19.490 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 52/164 2024-05-09T06:34:19.545 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 53/164 2024-05-09T06:34:19.581 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 54/164 2024-05-09T06:34:19.624 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 55/164 2024-05-09T06:34:19.666 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 56/164 2024-05-09T06:34:19.801 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 57/164 2024-05-09T06:34:19.855 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 58/164 2024-05-09T06:34:20.025 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/164 2024-05-09T06:34:20.070 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 59/164 2024-05-09T06:34:20.075 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/164 2024-05-09T06:34:20.109 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/164 2024-05-09T06:34:20.112 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 60/164 2024-05-09T06:34:20.211 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/164 2024-05-09T06:34:20.219 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-05-09T06:34:20.253 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 62/164 2024-05-09T06:34:20.267 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 88/164 2024-05-09T06:34:20.295 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-05-09T06:34:20.326 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 64/164 2024-05-09T06:34:20.351 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-05-09T06:34:20.365 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-05-09T06:34:20.402 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-05-09T06:34:20.438 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-05-09T06:34:20.570 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-05-09T06:34:20.586 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : openblas-0.3.21-2.el9.x86_64 91/164 2024-05-09T06:34:20.616 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 92/164 2024-05-09T06:34:20.617 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-05-09T06:34:20.654 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-05-09T06:34:20.687 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-05-09T06:34:20.698 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-05-09T06:34:20.726 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-05-09T06:34:20.758 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-05-09T06:34:20.788 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-05-09T06:34:20.822 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-05-09T06:34:20.856 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-05-09T06:34:20.897 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-05-09T06:34:20.940 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-05-09T06:34:21.026 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-05-09T06:34:21.071 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-05-09T06:34:21.101 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-05-09T06:34:21.148 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-05-09T06:34:21.247 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-packaging-20.9-5.el9.noarch 82/164 2024-05-09T06:34:21.294 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/164 2024-05-09T06:34:21.378 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-05-09T06:34:21.492 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-05-09T06:34:22.019 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/164 2024-05-09T06:34:22.075 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/164 2024-05-09T06:34:22.108 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/164 2024-05-09T06:34:22.205 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/164 2024-05-09T06:34:22.257 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 88/164 2024-05-09T06:34:22.354 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-05-09T06:34:22.394 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-05-09T06:34:22.570 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : openblas-0.3.21-2.el9.x86_64 91/164 2024-05-09T06:34:22.600 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 92/164 2024-05-09T06:34:22.683 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-05-09T06:34:22.809 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-05-09T06:34:22.903 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-05-09T06:34:22.931 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-05-09T06:34:22.964 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 99/164 2024-05-09T06:34:23.298 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-05-09T06:34:23.344 INFO:teuthology.orchestra.run.smithi003.stdout: Upgrading : librbd1-2:19.0.0-2515.g96805dcf.el9.x86_64 101/164 2024-05-09T06:34:23.345 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-05-09T06:34:23.435 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: librbd1-2:19.0.0-2515.g96805dcf.el9.x86_64 101/164 2024-05-09T06:34:23.456 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-05-09T06:34:23.465 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-rbd-2:19.0.0-2515.g96805dcf.el9.x86_64 102/164 2024-05-09T06:34:23.501 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-05-09T06:34:24.008 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-05-09T06:34:24.038 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : librgw2-2:19.0.0-2515.g96805dcf.el9.x86_64 105/164 2024-05-09T06:34:24.093 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: librgw2-2:19.0.0-2515.g96805dcf.el9.x86_64 105/164 2024-05-09T06:34:24.123 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-rgw-2:19.0.0-2515.g96805dcf.el9.x86_64 106/164 2024-05-09T06:34:24.712 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-05-09T06:34:24.837 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-05-09T06:34:24.865 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-05-09T06:34:24.898 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 99/164 2024-05-09T06:34:25.220 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-05-09T06:34:25.261 INFO:teuthology.orchestra.run.smithi190.stdout: Upgrading : librbd1-2:19.0.0-2515.g96805dcf.el9.x86_64 101/164 2024-05-09T06:34:25.351 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: librbd1-2:19.0.0-2515.g96805dcf.el9.x86_64 101/164 2024-05-09T06:34:25.382 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-rbd-2:19.0.0-2515.g96805dcf.el9.x86_64 102/164 2024-05-09T06:34:25.418 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-05-09T06:34:25.890 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-05-09T06:34:25.922 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : librgw2-2:19.0.0-2515.g96805dcf.el9.x86_64 105/164 2024-05-09T06:34:25.985 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: librgw2-2:19.0.0-2515.g96805dcf.el9.x86_64 105/164 2024-05-09T06:34:26.015 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-rgw-2:19.0.0-2515.g96805dcf.el9.x86_64 106/164 2024-05-09T06:34:26.506 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: ceph-common-2:19.0.0-2515.g96805dcf.el9.x86_64 107/164 2024-05-09T06:34:26.545 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-common-2:19.0.0-2515.g96805dcf.el9.x86_64 107/164 2024-05-09T06:34:27.122 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: ceph-common-2:19.0.0-2515.g96805dcf.el9.x86_64 107/164 2024-05-09T06:34:27.157 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-base-2:19.0.0-2515.g96805dcf.el9.x86_64 108/164 2024-05-09T06:34:27.251 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: ceph-base-2:19.0.0-2515.g96805dcf.el9.x86_64 108/164 2024-05-09T06:34:27.251 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-05-09T06:34:27.251 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-05-09T06:34:27.251 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-09T06:34:27.292 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-selinux-2:19.0.0-2515.g96805dcf.el9.x86_64 109/164 2024-05-09T06:34:28.321 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-common-2:19.0.0-2515.g96805dcf.el9.x86_64 107/164 2024-05-09T06:34:28.353 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-common-2:19.0.0-2515.g96805dcf.el9.x86_64 107/164 2024-05-09T06:34:28.907 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-common-2:19.0.0-2515.g96805dcf.el9.x86_64 107/164 2024-05-09T06:34:28.957 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-base-2:19.0.0-2515.g96805dcf.el9.x86_64 108/164 2024-05-09T06:34:29.050 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-base-2:19.0.0-2515.g96805dcf.el9.x86_64 108/164 2024-05-09T06:34:29.051 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-05-09T06:34:29.051 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-05-09T06:34:29.051 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-05-09T06:34:29.091 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-selinux-2:19.0.0-2515.g96805dcf.el9.x86_64 109/164 2024-05-09T06:34:39.422 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2515.g96805dcf.el9.x86_64 109/164 2024-05-09T06:34:39.422 INFO:teuthology.orchestra.run.smithi003.stdout:skipping the directory /sys 2024-05-09T06:34:39.422 INFO:teuthology.orchestra.run.smithi003.stdout:skipping the directory /proc 2024-05-09T06:34:39.422 INFO:teuthology.orchestra.run.smithi003.stdout:skipping the directory /mnt 2024-05-09T06:34:39.422 INFO:teuthology.orchestra.run.smithi003.stdout:skipping the directory /var/tmp 2024-05-09T06:34:39.422 INFO:teuthology.orchestra.run.smithi003.stdout:skipping the directory /home 2024-05-09T06:34:39.422 INFO:teuthology.orchestra.run.smithi003.stdout:skipping the directory /root 2024-05-09T06:34:39.422 INFO:teuthology.orchestra.run.smithi003.stdout:skipping the directory /tmp 2024-05-09T06:34:39.422 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-09T06:34:39.590 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-volume-2:19.0.0-2515.g96805dcf.el9.noarch 110/164 2024-05-09T06:34:39.607 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: ceph-volume-2:19.0.0-2515.g96805dcf.el9.noarch 110/164 2024-05-09T06:34:39.607 INFO:teuthology.orchestra.run.smithi003.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T06:34:39.607 INFO:teuthology.orchestra.run.smithi003.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-05-09T06:34:39.607 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-09T06:34:40.643 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2515.g96805dcf.el9.x86_64 109/164 2024-05-09T06:34:40.643 INFO:teuthology.orchestra.run.smithi190.stdout:skipping the directory /sys 2024-05-09T06:34:40.643 INFO:teuthology.orchestra.run.smithi190.stdout:skipping the directory /proc 2024-05-09T06:34:40.644 INFO:teuthology.orchestra.run.smithi190.stdout:skipping the directory /mnt 2024-05-09T06:34:40.644 INFO:teuthology.orchestra.run.smithi190.stdout:skipping the directory /var/tmp 2024-05-09T06:34:40.644 INFO:teuthology.orchestra.run.smithi190.stdout:skipping the directory /home 2024-05-09T06:34:40.644 INFO:teuthology.orchestra.run.smithi190.stdout:skipping the directory /root 2024-05-09T06:34:40.644 INFO:teuthology.orchestra.run.smithi190.stdout:skipping the directory /tmp 2024-05-09T06:34:40.644 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-05-09T06:34:40.812 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-volume-2:19.0.0-2515.g96805dcf.el9.noarch 110/164 2024-05-09T06:34:40.829 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-volume-2:19.0.0-2515.g96805dcf.el9.noarch 110/164 2024-05-09T06:34:40.829 INFO:teuthology.orchestra.run.smithi190.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T06:34:40.829 INFO:teuthology.orchestra.run.smithi190.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-05-09T06:34:40.829 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-05-09T06:34:41.135 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-osd-2:19.0.0-2515.g96805dcf.el9.x86_64 111/164 2024-05-09T06:34:41.164 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: ceph-osd-2:19.0.0-2515.g96805dcf.el9.x86_64 111/164 2024-05-09T06:34:41.164 INFO:teuthology.orchestra.run.smithi003.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T06:34:41.164 INFO:teuthology.orchestra.run.smithi003.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-05-09T06:34:41.164 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-09T06:34:41.164 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-09T06:34:41.164 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-09T06:34:41.394 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-mds-2:19.0.0-2515.g96805dcf.el9.x86_64 112/164 2024-05-09T06:34:41.419 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: ceph-mds-2:19.0.0-2515.g96805dcf.el9.x86_64 112/164 2024-05-09T06:34:41.419 INFO:teuthology.orchestra.run.smithi003.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T06:34:41.419 INFO:teuthology.orchestra.run.smithi003.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-05-09T06:34:41.419 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-09T06:34:41.419 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-09T06:34:41.419 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-09T06:34:41.894 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-mon-2:19.0.0-2515.g96805dcf.el9.x86_64 113/164 2024-05-09T06:34:41.919 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: ceph-mon-2:19.0.0-2515.g96805dcf.el9.x86_64 113/164 2024-05-09T06:34:41.919 INFO:teuthology.orchestra.run.smithi003.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T06:34:41.920 INFO:teuthology.orchestra.run.smithi003.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-05-09T06:34:41.920 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-09T06:34:41.920 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-09T06:34:41.920 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-09T06:34:42.033 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-05-09T06:34:42.091 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-05-09T06:34:42.155 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-05-09T06:34:42.301 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-osd-2:19.0.0-2515.g96805dcf.el9.x86_64 111/164 2024-05-09T06:34:42.328 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-osd-2:19.0.0-2515.g96805dcf.el9.x86_64 111/164 2024-05-09T06:34:42.328 INFO:teuthology.orchestra.run.smithi190.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T06:34:42.328 INFO:teuthology.orchestra.run.smithi190.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-05-09T06:34:42.328 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-09T06:34:42.328 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-09T06:34:42.328 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-05-09T06:34:42.351 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-05-09T06:34:42.404 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-05-09T06:34:42.483 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-05-09T06:34:42.551 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-mds-2:19.0.0-2515.g96805dcf.el9.x86_64 112/164 2024-05-09T06:34:42.573 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-05-09T06:34:42.575 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-mds-2:19.0.0-2515.g96805dcf.el9.x86_64 112/164 2024-05-09T06:34:42.575 INFO:teuthology.orchestra.run.smithi190.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T06:34:42.575 INFO:teuthology.orchestra.run.smithi190.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-05-09T06:34:42.575 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-09T06:34:42.575 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-09T06:34:42.575 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-05-09T06:34:42.715 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-05-09T06:34:42.770 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-05-09T06:34:42.809 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-05-09T06:34:42.838 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : fuse-2.9.9-15.el9.x86_64 124/164 2024-05-09T06:34:43.019 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-mon-2:19.0.0-2515.g96805dcf.el9.x86_64 113/164 2024-05-09T06:34:43.042 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-mon-2:19.0.0-2515.g96805dcf.el9.x86_64 113/164 2024-05-09T06:34:43.042 INFO:teuthology.orchestra.run.smithi190.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T06:34:43.042 INFO:teuthology.orchestra.run.smithi190.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-05-09T06:34:43.042 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-09T06:34:43.042 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-09T06:34:43.042 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-05-09T06:34:43.149 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-05-09T06:34:43.197 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: cephadm-2:19.0.0-2515.g96805dcf.el9.noarch 125/164 2024-05-09T06:34:43.206 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-05-09T06:34:43.228 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : cephadm-2:19.0.0-2515.g96805dcf.el9.noarch 125/164 2024-05-09T06:34:43.268 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2515.g96805dcf.e 126/164 2024-05-09T06:34:43.269 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-05-09T06:34:43.332 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2515.g96805dcf. 127/164 2024-05-09T06:34:43.360 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2515.g96805dcf.el9.noa 128/164 2024-05-09T06:34:43.456 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-05-09T06:34:43.503 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-05-09T06:34:43.580 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-05-09T06:34:43.677 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-05-09T06:34:43.817 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-05-09T06:34:43.861 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-05-09T06:34:43.902 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-05-09T06:34:43.928 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : fuse-2.9.9-15.el9.x86_64 124/164 2024-05-09T06:34:44.371 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: cephadm-2:19.0.0-2515.g96805dcf.el9.noarch 125/164 2024-05-09T06:34:44.394 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : cephadm-2:19.0.0-2515.g96805dcf.el9.noarch 125/164 2024-05-09T06:34:44.425 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2515.g96805dcf.e 126/164 2024-05-09T06:34:44.447 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2515.g96805dcf.el9.noa 128/164 2024-05-09T06:34:44.486 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2515.g96805dcf.el9.n 129/164 2024-05-09T06:34:44.487 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2515.g96805dcf. 127/164 2024-05-09T06:34:44.517 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2515.g96805dcf.el9.noa 128/164 2024-05-09T06:34:45.508 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2515.g96805dcf.el9.n 129/164 2024-05-09T06:34:45.537 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2515.g968 130/164 2024-05-09T06:34:45.543 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2515.g96805dcf.el9.noa 128/164 2024-05-09T06:34:45.585 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2515.g96805dcf.el9.n 129/164 2024-05-09T06:34:45.647 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2515.g968 130/164 2024-05-09T06:34:45.816 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2515.g96805dcf.el 131/164 2024-05-09T06:34:45.847 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-mgr-2:19.0.0-2515.g96805dcf.el9.x86_64 132/164 2024-05-09T06:34:45.873 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2515.g96805dcf.el9.x86_64 132/164 2024-05-09T06:34:45.873 INFO:teuthology.orchestra.run.smithi003.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T06:34:45.873 INFO:teuthology.orchestra.run.smithi003.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-05-09T06:34:45.873 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-09T06:34:45.873 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-09T06:34:45.873 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-09T06:34:45.923 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-mgr-rook-2:19.0.0-2515.g96805dcf.el9.noarch 133/164 2024-05-09T06:34:45.935 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2515.g96805dcf.el9.noarch 133/164 2024-05-09T06:34:46.036 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-2:19.0.0-2515.g96805dcf.el9.x86_64 134/164 2024-05-09T06:34:46.093 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-fuse-2:19.0.0-2515.g96805dcf.el9.x86_64 135/164 2024-05-09T06:34:46.556 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2515.g96805dcf.el9.n 129/164 2024-05-09T06:34:46.585 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2515.g968 130/164 2024-05-09T06:34:46.688 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2515.g968 130/164 2024-05-09T06:34:46.848 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2515.g96805dcf.el 131/164 2024-05-09T06:34:46.870 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-mgr-2:19.0.0-2515.g96805dcf.el9.x86_64 132/164 2024-05-09T06:34:46.895 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2515.g96805dcf.el9.x86_64 132/164 2024-05-09T06:34:46.895 INFO:teuthology.orchestra.run.smithi190.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T06:34:46.895 INFO:teuthology.orchestra.run.smithi190.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-05-09T06:34:46.895 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-09T06:34:46.895 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-09T06:34:46.895 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-05-09T06:34:46.946 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-mgr-rook-2:19.0.0-2515.g96805dcf.el9.noarch 133/164 2024-05-09T06:34:46.958 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2515.g96805dcf.el9.noarch 133/164 2024-05-09T06:34:47.037 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : libtool-2.4.6-45.el9.x86_64 136/164 2024-05-09T06:34:47.046 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-2:19.0.0-2515.g96805dcf.el9.x86_64 134/164 2024-05-09T06:34:47.067 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-radosgw-2:19.0.0-2515.g96805dcf.el9.x86_64 137/164 2024-05-09T06:34:47.092 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2515.g96805dcf.el9.x86_64 137/164 2024-05-09T06:34:47.092 INFO:teuthology.orchestra.run.smithi003.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T06:34:47.092 INFO:teuthology.orchestra.run.smithi003.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-05-09T06:34:47.093 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-09T06:34:47.093 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-09T06:34:47.093 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-09T06:34:47.104 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-fuse-2:19.0.0-2515.g96805dcf.el9.x86_64 135/164 2024-05-09T06:34:47.135 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2515.g96805 138/164 2024-05-09T06:34:47.158 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2515.g96805 138/164 2024-05-09T06:34:47.159 INFO:teuthology.orchestra.run.smithi003.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T06:34:47.159 INFO:teuthology.orchestra.run.smithi003.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-05-09T06:34:47.159 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-09T06:34:47.210 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : cephfs-mirror-2:19.0.0-2515.g96805dcf.el9.x86_64 139/164 2024-05-09T06:34:47.236 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2515.g96805dcf.el9.x86_64 139/164 2024-05-09T06:34:47.236 INFO:teuthology.orchestra.run.smithi003.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T06:34:47.236 INFO:teuthology.orchestra.run.smithi003.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-05-09T06:34:47.236 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-09T06:34:47.236 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-09T06:34:47.236 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-09T06:34:47.553 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : rbd-mirror-2:19.0.0-2515.g96805dcf.el9.x86_64 140/164 2024-05-09T06:34:47.578 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2515.g96805dcf.el9.x86_64 140/164 2024-05-09T06:34:47.578 INFO:teuthology.orchestra.run.smithi003.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T06:34:47.578 INFO:teuthology.orchestra.run.smithi003.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-05-09T06:34:47.578 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-09T06:34:47.578 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-09T06:34:47.578 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-09T06:34:48.002 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : libtool-2.4.6-45.el9.x86_64 136/164 2024-05-09T06:34:48.040 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-radosgw-2:19.0.0-2515.g96805dcf.el9.x86_64 137/164 2024-05-09T06:34:48.064 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2515.g96805dcf.el9.x86_64 137/164 2024-05-09T06:34:48.064 INFO:teuthology.orchestra.run.smithi190.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T06:34:48.064 INFO:teuthology.orchestra.run.smithi190.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-05-09T06:34:48.064 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-09T06:34:48.064 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-09T06:34:48.065 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-05-09T06:34:48.107 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2515.g96805 138/164 2024-05-09T06:34:48.130 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2515.g96805 138/164 2024-05-09T06:34:48.130 INFO:teuthology.orchestra.run.smithi190.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T06:34:48.130 INFO:teuthology.orchestra.run.smithi190.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-05-09T06:34:48.130 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-05-09T06:34:48.175 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : cephfs-mirror-2:19.0.0-2515.g96805dcf.el9.x86_64 139/164 2024-05-09T06:34:48.198 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2515.g96805dcf.el9.x86_64 139/164 2024-05-09T06:34:48.198 INFO:teuthology.orchestra.run.smithi190.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T06:34:48.199 INFO:teuthology.orchestra.run.smithi190.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-05-09T06:34:48.199 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-09T06:34:48.199 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-09T06:34:48.199 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-05-09T06:34:48.500 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : rbd-mirror-2:19.0.0-2515.g96805dcf.el9.x86_64 140/164 2024-05-09T06:34:48.524 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2515.g96805dcf.el9.x86_64 140/164 2024-05-09T06:34:48.524 INFO:teuthology.orchestra.run.smithi190.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T06:34:48.524 INFO:teuthology.orchestra.run.smithi190.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-05-09T06:34:48.524 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-09T06:34:48.524 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-09T06:34:48.525 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-05-09T06:34:52.356 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-test-2:19.0.0-2515.g96805dcf.el9.x86_64 141/164 2024-05-09T06:34:52.398 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : rbd-fuse-2:19.0.0-2515.g96805dcf.el9.x86_64 142/164 2024-05-09T06:34:52.439 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : rbd-nbd-2:19.0.0-2515.g96805dcf.el9.x86_64 143/164 2024-05-09T06:34:52.470 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 144/164 2024-05-09T06:34:52.498 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 145/164 2024-05-09T06:34:52.703 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : cephfs-shell-2:19.0.0-2515.g96805dcf.el9.x86_64 146/164 2024-05-09T06:34:52.752 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 147/164 2024-05-09T06:34:52.782 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : libcephfs-devel-2:19.0.0-2515.g96805dcf.el9.x86_ 148/164 2024-05-09T06:34:52.812 INFO:teuthology.orchestra.run.smithi003.stdout: Upgrading : NetworkManager-team-1:1.47.5-1.el9.x86_64 149/164 2024-05-09T06:34:52.858 INFO:teuthology.orchestra.run.smithi003.stdout: Upgrading : NetworkManager-tui-1:1.47.5-1.el9.x86_64 150/164 2024-05-09T06:34:53.062 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-test-2:19.0.0-2515.g96805dcf.el9.x86_64 141/164 2024-05-09T06:34:53.112 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : rbd-fuse-2:19.0.0-2515.g96805dcf.el9.x86_64 142/164 2024-05-09T06:34:53.152 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : rbd-nbd-2:19.0.0-2515.g96805dcf.el9.x86_64 143/164 2024-05-09T06:34:53.185 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 144/164 2024-05-09T06:34:53.212 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 145/164 2024-05-09T06:34:53.404 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : cephfs-shell-2:19.0.0-2515.g96805dcf.el9.x86_64 146/164 2024-05-09T06:34:53.449 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 147/164 2024-05-09T06:34:53.479 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : libcephfs-devel-2:19.0.0-2515.g96805dcf.el9.x86_ 148/164 2024-05-09T06:34:53.495 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-09T06:34:53.509 INFO:teuthology.orchestra.run.smithi190.stdout: Upgrading : NetworkManager-team-1:1.47.5-1.el9.x86_64 149/164 2024-05-09T06:34:53.531 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-09T06:34:53.547 INFO:teuthology.orchestra.run.smithi190.stdout: Upgrading : NetworkManager-tui-1:1.47.5-1.el9.x86_64 150/164 2024-05-09T06:34:53.553 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-09T06:34:54.104 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-09T06:34:54.153 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-09T06:34:54.173 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-09T06:34:54.451 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : cephfs-top-2:19.0.0-2515.g96805dcf.el9.noarch 152/164 2024-05-09T06:34:54.553 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 153/164 2024-05-09T06:34:54.603 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-05-09T06:34:54.702 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-05-09T06:34:54.739 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-05-09T06:34:54.766 INFO:teuthology.orchestra.run.smithi003.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 157/164 2024-05-09T06:34:54.766 INFO:teuthology.orchestra.run.smithi003.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-09T06:34:54.858 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-09T06:34:54.891 INFO:teuthology.orchestra.run.smithi003.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 159/164 2024-05-09T06:34:54.891 INFO:teuthology.orchestra.run.smithi003.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-09T06:34:54.975 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-09T06:34:55.001 INFO:teuthology.orchestra.run.smithi003.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 161/164 2024-05-09T06:34:55.003 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : cephfs-top-2:19.0.0-2515.g96805dcf.el9.noarch 152/164 2024-05-09T06:34:55.006 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-09T06:34:55.006 INFO:teuthology.orchestra.run.smithi003.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-09T06:34:55.082 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 153/164 2024-05-09T06:34:55.130 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-05-09T06:34:55.202 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-09T06:34:55.202 INFO:teuthology.orchestra.run.smithi003.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-09T06:34:55.213 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-05-09T06:34:55.243 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-05-09T06:34:55.269 INFO:teuthology.orchestra.run.smithi190.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 157/164 2024-05-09T06:34:55.269 INFO:teuthology.orchestra.run.smithi190.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-09T06:34:55.278 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-09T06:34:55.346 INFO:teuthology.orchestra.run.smithi003.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-09T06:34:55.362 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-09T06:34:55.386 INFO:teuthology.orchestra.run.smithi190.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 159/164 2024-05-09T06:34:55.387 INFO:teuthology.orchestra.run.smithi190.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-09T06:34:55.471 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-09T06:34:55.496 INFO:teuthology.orchestra.run.smithi190.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 161/164 2024-05-09T06:34:55.501 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-09T06:34:55.501 INFO:teuthology.orchestra.run.smithi190.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-09T06:34:55.681 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-09T06:34:55.681 INFO:teuthology.orchestra.run.smithi190.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-09T06:34:55.765 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-09T06:34:55.829 INFO:teuthology.orchestra.run.smithi190.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-09T06:35:00.725 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-09T06:35:00.726 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-2:19.0.0-2515.g96805dcf.el9.x86_64 1/164 2024-05-09T06:35:00.726 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-base-2:19.0.0-2515.g96805dcf.el9.x86_64 2/164 2024-05-09T06:35:00.726 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-common-2:19.0.0-2515.g96805dcf.el9.x86_64 3/164 2024-05-09T06:35:00.726 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-fuse-2:19.0.0-2515.g96805dcf.el9.x86_64 4/164 2024-05-09T06:35:00.726 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2515.g96805 5/164 2024-05-09T06:35:00.726 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-mds-2:19.0.0-2515.g96805dcf.el9.x86_64 6/164 2024-05-09T06:35:00.726 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-mgr-2:19.0.0-2515.g96805dcf.el9.x86_64 7/164 2024-05-09T06:35:00.726 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-mon-2:19.0.0-2515.g96805dcf.el9.x86_64 8/164 2024-05-09T06:35:00.726 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-osd-2:19.0.0-2515.g96805dcf.el9.x86_64 9/164 2024-05-09T06:35:00.726 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-radosgw-2:19.0.0-2515.g96805dcf.el9.x86_64 10/164 2024-05-09T06:35:00.726 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-selinux-2:19.0.0-2515.g96805dcf.el9.x86_64 11/164 2024-05-09T06:35:00.726 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-test-2:19.0.0-2515.g96805dcf.el9.x86_64 12/164 2024-05-09T06:35:00.726 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : cephfs-mirror-2:19.0.0-2515.g96805dcf.el9.x86_64 13/164 2024-05-09T06:35:00.727 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : cephfs-shell-2:19.0.0-2515.g96805dcf.el9.x86_64 14/164 2024-05-09T06:35:00.727 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : libcephfs-devel-2:19.0.0-2515.g96805dcf.el9.x86_ 15/164 2024-05-09T06:35:00.727 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : libcephfs2-2:19.0.0-2515.g96805dcf.el9.x86_64 16/164 2024-05-09T06:35:00.727 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : libcephsqlite-2:19.0.0-2515.g96805dcf.el9.x86_64 17/164 2024-05-09T06:35:00.727 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : librados-devel-2:19.0.0-2515.g96805dcf.el9.x86_6 18/164 2024-05-09T06:35:00.727 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : libradosstriper1-2:19.0.0-2515.g96805dcf.el9.x86 19/164 2024-05-09T06:35:00.727 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : librgw2-2:19.0.0-2515.g96805dcf.el9.x86_64 20/164 2024-05-09T06:35:00.727 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2515.g96805dcf.el 21/164 2024-05-09T06:35:00.727 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-ceph-common-2:19.0.0-2515.g96805dcf.el9. 22/164 2024-05-09T06:35:00.727 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-cephfs-2:19.0.0-2515.g96805dcf.el9.x86_6 23/164 2024-05-09T06:35:00.727 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-rados-2:19.0.0-2515.g96805dcf.el9.x86_64 24/164 2024-05-09T06:35:00.727 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-rbd-2:19.0.0-2515.g96805dcf.el9.x86_64 25/164 2024-05-09T06:35:00.727 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-rgw-2:19.0.0-2515.g96805dcf.el9.x86_64 26/164 2024-05-09T06:35:00.727 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : rbd-fuse-2:19.0.0-2515.g96805dcf.el9.x86_64 27/164 2024-05-09T06:35:00.727 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : rbd-mirror-2:19.0.0-2515.g96805dcf.el9.x86_64 28/164 2024-05-09T06:35:00.728 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : rbd-nbd-2:19.0.0-2515.g96805dcf.el9.x86_64 29/164 2024-05-09T06:35:00.728 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2515.g96805dcf. 30/164 2024-05-09T06:35:00.728 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2515.g96805dcf.el9.noa 31/164 2024-05-09T06:35:00.728 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2515.g96805dcf.el9.n 32/164 2024-05-09T06:35:00.728 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2515.g968 33/164 2024-05-09T06:35:00.728 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2515.g96805dcf.el 34/164 2024-05-09T06:35:00.728 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2515.g96805dcf.el9.noarch 35/164 2024-05-09T06:35:00.728 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2515.g96805dcf.e 36/164 2024-05-09T06:35:00.728 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-volume-2:19.0.0-2515.g96805dcf.el9.noarch 37/164 2024-05-09T06:35:00.728 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : cephadm-2:19.0.0-2515.g96805dcf.el9.noarch 38/164 2024-05-09T06:35:00.728 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : cephfs-top-2:19.0.0-2515.g96805dcf.el9.noarch 39/164 2024-05-09T06:35:00.728 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/164 2024-05-09T06:35:00.728 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/164 2024-05-09T06:35:00.729 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/164 2024-05-09T06:35:00.729 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-05-09T06:35:00.729 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-05-09T06:35:00.729 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-05-09T06:35:00.729 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-05-09T06:35:00.729 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-05-09T06:35:00.729 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/164 2024-05-09T06:35:00.729 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-05-09T06:35:00.729 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-05-09T06:35:00.729 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-05-09T06:35:00.729 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/164 2024-05-09T06:35:00.729 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-05-09T06:35:00.729 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-05-09T06:35:00.729 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-05-09T06:35:00.730 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-05-09T06:35:00.730 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-05-09T06:35:00.730 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-05-09T06:35:00.730 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-05-09T06:35:00.730 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-05-09T06:35:00.730 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/164 2024-05-09T06:35:00.730 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/164 2024-05-09T06:35:00.730 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-05-09T06:35:00.730 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-05-09T06:35:00.731 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-05-09T06:35:00.731 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/164 2024-05-09T06:35:00.731 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/164 2024-05-09T06:35:00.731 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/164 2024-05-09T06:35:00.731 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/164 2024-05-09T06:35:00.731 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/164 2024-05-09T06:35:00.731 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/164 2024-05-09T06:35:00.731 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/164 2024-05-09T06:35:00.731 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-05-09T06:35:00.731 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-05-09T06:35:00.731 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-05-09T06:35:00.731 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/164 2024-05-09T06:35:00.731 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/164 2024-05-09T06:35:00.731 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/164 2024-05-09T06:35:00.731 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/164 2024-05-09T06:35:00.732 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/164 2024-05-09T06:35:00.732 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/164 2024-05-09T06:35:00.732 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/164 2024-05-09T06:35:00.732 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/164 2024-05-09T06:35:00.732 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/164 2024-05-09T06:35:00.732 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/164 2024-05-09T06:35:00.732 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/164 2024-05-09T06:35:00.732 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/164 2024-05-09T06:35:00.732 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/164 2024-05-09T06:35:00.732 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/164 2024-05-09T06:35:00.732 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/164 2024-05-09T06:35:00.732 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/164 2024-05-09T06:35:00.732 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/164 2024-05-09T06:35:00.732 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/164 2024-05-09T06:35:00.733 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/164 2024-05-09T06:35:00.733 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/164 2024-05-09T06:35:00.733 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/164 2024-05-09T06:35:00.733 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/164 2024-05-09T06:35:00.733 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/164 2024-05-09T06:35:00.733 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/164 2024-05-09T06:35:00.733 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/164 2024-05-09T06:35:00.733 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/164 2024-05-09T06:35:00.733 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/164 2024-05-09T06:35:00.733 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/164 2024-05-09T06:35:00.733 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/164 2024-05-09T06:35:00.733 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/164 2024-05-09T06:35:00.733 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/164 2024-05-09T06:35:00.733 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/164 2024-05-09T06:35:00.734 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 108/164 2024-05-09T06:35:00.734 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 109/164 2024-05-09T06:35:00.734 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/164 2024-05-09T06:35:00.734 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/164 2024-05-09T06:35:00.734 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/164 2024-05-09T06:35:00.734 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 113/164 2024-05-09T06:35:00.734 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/164 2024-05-09T06:35:00.734 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/164 2024-05-09T06:35:00.734 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/164 2024-05-09T06:35:00.734 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/164 2024-05-09T06:35:00.734 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/164 2024-05-09T06:35:00.734 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/164 2024-05-09T06:35:00.734 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/164 2024-05-09T06:35:00.734 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/164 2024-05-09T06:35:00.735 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/164 2024-05-09T06:35:00.735 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-05-09T06:35:00.735 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-05-09T06:35:00.735 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-05-09T06:35:00.735 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-05-09T06:35:00.735 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-05-09T06:35:00.735 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-05-09T06:35:00.735 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-05-09T06:35:00.735 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-05-09T06:35:00.735 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-05-09T06:35:00.735 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-05-09T06:35:00.735 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-05-09T06:35:00.735 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-05-09T06:35:00.735 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-05-09T06:35:00.736 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-05-09T06:35:00.736 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-05-09T06:35:00.736 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-05-09T06:35:00.736 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-05-09T06:35:00.736 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-05-09T06:35:00.736 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-05-09T06:35:00.736 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/164 2024-05-09T06:35:00.736 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/164 2024-05-09T06:35:00.736 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-05-09T06:35:00.736 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-05-09T06:35:00.736 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-05-09T06:35:00.736 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/164 2024-05-09T06:35:00.736 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-05-09T06:35:00.737 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : librados2-2:19.0.0-2515.g96805dcf.el9.x86_64 149/164 2024-05-09T06:35:00.737 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-05-09T06:35:00.737 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : librbd1-2:19.0.0-2515.g96805dcf.el9.x86_64 151/164 2024-05-09T06:35:00.737 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-05-09T06:35:00.737 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : NetworkManager-1:1.47.5-1.el9.x86_64 153/164 2024-05-09T06:35:00.737 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/164 2024-05-09T06:35:00.737 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : NetworkManager-libnm-1:1.47.5-1.el9.x86_64 155/164 2024-05-09T06:35:00.737 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/164 2024-05-09T06:35:00.737 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : NetworkManager-team-1:1.47.5-1.el9.x86_64 157/164 2024-05-09T06:35:00.737 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/164 2024-05-09T06:35:00.737 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : NetworkManager-tui-1:1.47.5-1.el9.x86_64 159/164 2024-05-09T06:35:00.737 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/164 2024-05-09T06:35:00.737 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 161/164 2024-05-09T06:35:00.737 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 162/164 2024-05-09T06:35:00.737 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 163/164 2024-05-09T06:35:01.386 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-09T06:35:01.386 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-2:19.0.0-2515.g96805dcf.el9.x86_64 1/164 2024-05-09T06:35:01.386 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-base-2:19.0.0-2515.g96805dcf.el9.x86_64 2/164 2024-05-09T06:35:01.386 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-common-2:19.0.0-2515.g96805dcf.el9.x86_64 3/164 2024-05-09T06:35:01.386 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-fuse-2:19.0.0-2515.g96805dcf.el9.x86_64 4/164 2024-05-09T06:35:01.386 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2515.g96805 5/164 2024-05-09T06:35:01.386 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-mds-2:19.0.0-2515.g96805dcf.el9.x86_64 6/164 2024-05-09T06:35:01.386 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-mgr-2:19.0.0-2515.g96805dcf.el9.x86_64 7/164 2024-05-09T06:35:01.386 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-mon-2:19.0.0-2515.g96805dcf.el9.x86_64 8/164 2024-05-09T06:35:01.387 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-osd-2:19.0.0-2515.g96805dcf.el9.x86_64 9/164 2024-05-09T06:35:01.387 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-radosgw-2:19.0.0-2515.g96805dcf.el9.x86_64 10/164 2024-05-09T06:35:01.387 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-selinux-2:19.0.0-2515.g96805dcf.el9.x86_64 11/164 2024-05-09T06:35:01.387 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-test-2:19.0.0-2515.g96805dcf.el9.x86_64 12/164 2024-05-09T06:35:01.387 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : cephfs-mirror-2:19.0.0-2515.g96805dcf.el9.x86_64 13/164 2024-05-09T06:35:01.387 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : cephfs-shell-2:19.0.0-2515.g96805dcf.el9.x86_64 14/164 2024-05-09T06:35:01.387 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libcephfs-devel-2:19.0.0-2515.g96805dcf.el9.x86_ 15/164 2024-05-09T06:35:01.387 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libcephfs2-2:19.0.0-2515.g96805dcf.el9.x86_64 16/164 2024-05-09T06:35:01.387 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libcephsqlite-2:19.0.0-2515.g96805dcf.el9.x86_64 17/164 2024-05-09T06:35:01.387 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : librados-devel-2:19.0.0-2515.g96805dcf.el9.x86_6 18/164 2024-05-09T06:35:01.387 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libradosstriper1-2:19.0.0-2515.g96805dcf.el9.x86 19/164 2024-05-09T06:35:01.387 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : librgw2-2:19.0.0-2515.g96805dcf.el9.x86_64 20/164 2024-05-09T06:35:01.387 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2515.g96805dcf.el 21/164 2024-05-09T06:35:01.388 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-ceph-common-2:19.0.0-2515.g96805dcf.el9. 22/164 2024-05-09T06:35:01.388 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-cephfs-2:19.0.0-2515.g96805dcf.el9.x86_6 23/164 2024-05-09T06:35:01.388 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-rados-2:19.0.0-2515.g96805dcf.el9.x86_64 24/164 2024-05-09T06:35:01.388 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-rbd-2:19.0.0-2515.g96805dcf.el9.x86_64 25/164 2024-05-09T06:35:01.388 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-rgw-2:19.0.0-2515.g96805dcf.el9.x86_64 26/164 2024-05-09T06:35:01.388 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : rbd-fuse-2:19.0.0-2515.g96805dcf.el9.x86_64 27/164 2024-05-09T06:35:01.388 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : rbd-mirror-2:19.0.0-2515.g96805dcf.el9.x86_64 28/164 2024-05-09T06:35:01.404 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : rbd-nbd-2:19.0.0-2515.g96805dcf.el9.x86_64 29/164 2024-05-09T06:35:01.404 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2515.g96805dcf. 30/164 2024-05-09T06:35:01.404 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2515.g96805dcf.el9.noa 31/164 2024-05-09T06:35:01.404 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2515.g96805dcf.el9.n 32/164 2024-05-09T06:35:01.404 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2515.g968 33/164 2024-05-09T06:35:01.404 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2515.g96805dcf.el 34/164 2024-05-09T06:35:01.404 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2515.g96805dcf.el9.noarch 35/164 2024-05-09T06:35:01.404 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2515.g96805dcf.e 36/164 2024-05-09T06:35:01.404 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-volume-2:19.0.0-2515.g96805dcf.el9.noarch 37/164 2024-05-09T06:35:01.404 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : cephadm-2:19.0.0-2515.g96805dcf.el9.noarch 38/164 2024-05-09T06:35:01.405 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : cephfs-top-2:19.0.0-2515.g96805dcf.el9.noarch 39/164 2024-05-09T06:35:01.405 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/164 2024-05-09T06:35:01.405 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/164 2024-05-09T06:35:01.405 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/164 2024-05-09T06:35:01.405 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-05-09T06:35:01.405 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-05-09T06:35:01.405 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-05-09T06:35:01.405 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-05-09T06:35:01.405 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-05-09T06:35:01.405 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/164 2024-05-09T06:35:01.405 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-05-09T06:35:01.405 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-05-09T06:35:01.405 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-05-09T06:35:01.405 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/164 2024-05-09T06:35:01.406 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-05-09T06:35:01.406 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-05-09T06:35:01.406 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-05-09T06:35:01.406 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-05-09T06:35:01.406 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-05-09T06:35:01.406 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-05-09T06:35:01.406 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-05-09T06:35:01.406 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-05-09T06:35:01.406 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/164 2024-05-09T06:35:01.406 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/164 2024-05-09T06:35:01.406 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-05-09T06:35:01.406 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-05-09T06:35:01.406 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-05-09T06:35:01.407 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/164 2024-05-09T06:35:01.407 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/164 2024-05-09T06:35:01.407 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/164 2024-05-09T06:35:01.407 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/164 2024-05-09T06:35:01.407 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/164 2024-05-09T06:35:01.407 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/164 2024-05-09T06:35:01.407 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/164 2024-05-09T06:35:01.407 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-05-09T06:35:01.407 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-05-09T06:35:01.407 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-05-09T06:35:01.407 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/164 2024-05-09T06:35:01.407 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/164 2024-05-09T06:35:01.407 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/164 2024-05-09T06:35:01.407 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/164 2024-05-09T06:35:01.408 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/164 2024-05-09T06:35:01.408 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/164 2024-05-09T06:35:01.408 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/164 2024-05-09T06:35:01.408 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/164 2024-05-09T06:35:01.408 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/164 2024-05-09T06:35:01.408 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/164 2024-05-09T06:35:01.408 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/164 2024-05-09T06:35:01.408 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/164 2024-05-09T06:35:01.408 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/164 2024-05-09T06:35:01.408 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/164 2024-05-09T06:35:01.408 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/164 2024-05-09T06:35:01.408 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/164 2024-05-09T06:35:01.408 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/164 2024-05-09T06:35:01.408 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/164 2024-05-09T06:35:01.409 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/164 2024-05-09T06:35:01.409 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/164 2024-05-09T06:35:01.409 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/164 2024-05-09T06:35:01.409 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/164 2024-05-09T06:35:01.409 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/164 2024-05-09T06:35:01.409 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/164 2024-05-09T06:35:01.409 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/164 2024-05-09T06:35:01.409 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/164 2024-05-09T06:35:01.409 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/164 2024-05-09T06:35:01.409 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/164 2024-05-09T06:35:01.409 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/164 2024-05-09T06:35:01.410 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/164 2024-05-09T06:35:01.410 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/164 2024-05-09T06:35:01.410 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/164 2024-05-09T06:35:01.410 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 108/164 2024-05-09T06:35:01.410 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 109/164 2024-05-09T06:35:01.410 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/164 2024-05-09T06:35:01.410 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/164 2024-05-09T06:35:01.410 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/164 2024-05-09T06:35:01.410 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 113/164 2024-05-09T06:35:01.410 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/164 2024-05-09T06:35:01.410 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/164 2024-05-09T06:35:01.410 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/164 2024-05-09T06:35:01.410 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/164 2024-05-09T06:35:01.410 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/164 2024-05-09T06:35:01.411 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/164 2024-05-09T06:35:01.411 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/164 2024-05-09T06:35:01.411 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/164 2024-05-09T06:35:01.411 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/164 2024-05-09T06:35:01.411 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-05-09T06:35:01.411 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-05-09T06:35:01.411 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-05-09T06:35:01.411 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-05-09T06:35:01.411 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-05-09T06:35:01.411 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-05-09T06:35:01.411 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-05-09T06:35:01.411 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-05-09T06:35:01.411 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-05-09T06:35:01.412 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-05-09T06:35:01.412 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-05-09T06:35:01.412 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-05-09T06:35:01.412 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-05-09T06:35:01.412 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-05-09T06:35:01.412 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-05-09T06:35:01.412 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-05-09T06:35:01.412 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-05-09T06:35:01.412 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-05-09T06:35:01.412 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-05-09T06:35:01.412 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/164 2024-05-09T06:35:01.412 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/164 2024-05-09T06:35:01.412 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-05-09T06:35:01.412 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-05-09T06:35:01.412 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-05-09T06:35:01.412 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/164 2024-05-09T06:35:01.413 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-05-09T06:35:01.413 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : librados2-2:19.0.0-2515.g96805dcf.el9.x86_64 149/164 2024-05-09T06:35:01.413 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-05-09T06:35:01.413 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : librbd1-2:19.0.0-2515.g96805dcf.el9.x86_64 151/164 2024-05-09T06:35:01.413 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-05-09T06:35:01.413 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : NetworkManager-1:1.47.5-1.el9.x86_64 153/164 2024-05-09T06:35:01.413 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/164 2024-05-09T06:35:01.413 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : NetworkManager-libnm-1:1.47.5-1.el9.x86_64 155/164 2024-05-09T06:35:01.413 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/164 2024-05-09T06:35:01.413 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : NetworkManager-team-1:1.47.5-1.el9.x86_64 157/164 2024-05-09T06:35:01.413 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/164 2024-05-09T06:35:01.413 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : NetworkManager-tui-1:1.47.5-1.el9.x86_64 159/164 2024-05-09T06:35:01.413 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/164 2024-05-09T06:35:01.413 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 161/164 2024-05-09T06:35:01.414 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 162/164 2024-05-09T06:35:01.414 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 163/164 2024-05-09T06:35:03.663 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-09T06:35:03.663 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-09T06:35:03.663 INFO:teuthology.orchestra.run.smithi003.stdout:Upgraded: 2024-05-09T06:35:03.664 INFO:teuthology.orchestra.run.smithi003.stdout: NetworkManager-1:1.47.5-1.el9.x86_64 2024-05-09T06:35:03.664 INFO:teuthology.orchestra.run.smithi003.stdout: NetworkManager-libnm-1:1.47.5-1.el9.x86_64 2024-05-09T06:35:03.664 INFO:teuthology.orchestra.run.smithi003.stdout: NetworkManager-team-1:1.47.5-1.el9.x86_64 2024-05-09T06:35:03.664 INFO:teuthology.orchestra.run.smithi003.stdout: NetworkManager-tui-1:1.47.5-1.el9.x86_64 2024-05-09T06:35:03.664 INFO:teuthology.orchestra.run.smithi003.stdout: librados2-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T06:35:03.664 INFO:teuthology.orchestra.run.smithi003.stdout: librbd1-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T06:35:03.664 INFO:teuthology.orchestra.run.smithi003.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-05-09T06:35:03.664 INFO:teuthology.orchestra.run.smithi003.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-05-09T06:35:03.664 INFO:teuthology.orchestra.run.smithi003.stdout:Installed: 2024-05-09T06:35:03.664 INFO:teuthology.orchestra.run.smithi003.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-05-09T06:35:03.664 INFO:teuthology.orchestra.run.smithi003.stdout: automake-1.16.2-8.el9.noarch 2024-05-09T06:35:03.664 INFO:teuthology.orchestra.run.smithi003.stdout: bison-3.7.4-5.el9.x86_64 2024-05-09T06:35:03.664 INFO:teuthology.orchestra.run.smithi003.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-05-09T06:35:03.664 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T06:35:03.665 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-base-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T06:35:03.665 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-common-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T06:35:03.665 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-fuse-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T06:35:03.665 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-grafana-dashboards-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-09T06:35:03.665 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-immutable-object-cache-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T06:35:03.665 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mds-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T06:35:03.665 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mgr-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T06:35:03.665 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mgr-cephadm-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-09T06:35:03.665 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mgr-dashboard-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-09T06:35:03.665 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-09T06:35:03.665 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mgr-modules-core-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-09T06:35:03.665 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mgr-rook-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-09T06:35:03.665 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mon-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T06:35:03.666 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-osd-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T06:35:03.666 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-prometheus-alerts-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-09T06:35:03.666 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-radosgw-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T06:35:03.666 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-selinux-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T06:35:03.666 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-test-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T06:35:03.666 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-volume-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-09T06:35:03.666 INFO:teuthology.orchestra.run.smithi003.stdout: cephadm-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-09T06:35:03.666 INFO:teuthology.orchestra.run.smithi003.stdout: cephfs-mirror-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T06:35:03.666 INFO:teuthology.orchestra.run.smithi003.stdout: cephfs-shell-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T06:35:03.666 INFO:teuthology.orchestra.run.smithi003.stdout: cephfs-top-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-09T06:35:03.666 INFO:teuthology.orchestra.run.smithi003.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-05-09T06:35:03.666 INFO:teuthology.orchestra.run.smithi003.stdout: flex-2.6.4-9.el9.x86_64 2024-05-09T06:35:03.666 INFO:teuthology.orchestra.run.smithi003.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-05-09T06:35:03.666 INFO:teuthology.orchestra.run.smithi003.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-05-09T06:35:03.667 INFO:teuthology.orchestra.run.smithi003.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-05-09T06:35:03.667 INFO:teuthology.orchestra.run.smithi003.stdout: fuse-2.9.9-15.el9.x86_64 2024-05-09T06:35:03.667 INFO:teuthology.orchestra.run.smithi003.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-05-09T06:35:03.667 INFO:teuthology.orchestra.run.smithi003.stdout: grpc-data-1.46.7-10.el9.noarch 2024-05-09T06:35:03.667 INFO:teuthology.orchestra.run.smithi003.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-05-09T06:35:03.667 INFO:teuthology.orchestra.run.smithi003.stdout: libarrow-9.0.0-11.el9.x86_64 2024-05-09T06:35:03.667 INFO:teuthology.orchestra.run.smithi003.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-05-09T06:35:03.667 INFO:teuthology.orchestra.run.smithi003.stdout: libcephfs-devel-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T06:35:03.667 INFO:teuthology.orchestra.run.smithi003.stdout: libcephfs2-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T06:35:03.667 INFO:teuthology.orchestra.run.smithi003.stdout: libcephsqlite-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T06:35:03.667 INFO:teuthology.orchestra.run.smithi003.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-05-09T06:35:03.667 INFO:teuthology.orchestra.run.smithi003.stdout: liboath-2.6.7-2.el9.x86_64 2024-05-09T06:35:03.667 INFO:teuthology.orchestra.run.smithi003.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-05-09T06:35:03.667 INFO:teuthology.orchestra.run.smithi003.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-05-09T06:35:03.668 INFO:teuthology.orchestra.run.smithi003.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-05-09T06:35:03.668 INFO:teuthology.orchestra.run.smithi003.stdout: librados-devel-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T06:35:03.668 INFO:teuthology.orchestra.run.smithi003.stdout: libradosstriper1-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T06:35:03.668 INFO:teuthology.orchestra.run.smithi003.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-05-09T06:35:03.668 INFO:teuthology.orchestra.run.smithi003.stdout: librgw2-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T06:35:03.668 INFO:teuthology.orchestra.run.smithi003.stdout: libtool-2.4.6-45.el9.x86_64 2024-05-09T06:35:03.668 INFO:teuthology.orchestra.run.smithi003.stdout: libunwind-1.6.2-1.el9.x86_64 2024-05-09T06:35:03.668 INFO:teuthology.orchestra.run.smithi003.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-05-09T06:35:03.668 INFO:teuthology.orchestra.run.smithi003.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-05-09T06:35:03.668 INFO:teuthology.orchestra.run.smithi003.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-05-09T06:35:03.668 INFO:teuthology.orchestra.run.smithi003.stdout: lua-5.4.4-4.el9.x86_64 2024-05-09T06:35:03.668 INFO:teuthology.orchestra.run.smithi003.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-05-09T06:35:03.668 INFO:teuthology.orchestra.run.smithi003.stdout: luarocks-3.9.2-1.el9.noarch 2024-05-09T06:35:03.668 INFO:teuthology.orchestra.run.smithi003.stdout: openblas-0.3.21-2.el9.x86_64 2024-05-09T06:35:03.668 INFO:teuthology.orchestra.run.smithi003.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-05-09T06:35:03.669 INFO:teuthology.orchestra.run.smithi003.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-05-09T06:35:03.669 INFO:teuthology.orchestra.run.smithi003.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-05-09T06:35:03.669 INFO:teuthology.orchestra.run.smithi003.stdout: postgresql-13.14-1.el9.x86_64 2024-05-09T06:35:03.669 INFO:teuthology.orchestra.run.smithi003.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-05-09T06:35:03.669 INFO:teuthology.orchestra.run.smithi003.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-05-09T06:35:03.669 INFO:teuthology.orchestra.run.smithi003.stdout: postgresql-server-13.14-1.el9.x86_64 2024-05-09T06:35:03.669 INFO:teuthology.orchestra.run.smithi003.stdout: protobuf-3.14.0-13.el9.x86_64 2024-05-09T06:35:03.669 INFO:teuthology.orchestra.run.smithi003.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-05-09T06:35:03.669 INFO:teuthology.orchestra.run.smithi003.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-05-09T06:35:03.669 INFO:teuthology.orchestra.run.smithi003.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-05-09T06:35:03.669 INFO:teuthology.orchestra.run.smithi003.stdout: python3-babel-2.9.1-2.el9.noarch 2024-05-09T06:35:03.669 INFO:teuthology.orchestra.run.smithi003.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-05-09T06:35:03.669 INFO:teuthology.orchestra.run.smithi003.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-05-09T06:35:03.669 INFO:teuthology.orchestra.run.smithi003.stdout: python3-ceph-argparse-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T06:35:03.669 INFO:teuthology.orchestra.run.smithi003.stdout: python3-ceph-common-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T06:35:03.670 INFO:teuthology.orchestra.run.smithi003.stdout: python3-cephfs-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T06:35:03.670 INFO:teuthology.orchestra.run.smithi003.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-05-09T06:35:03.670 INFO:teuthology.orchestra.run.smithi003.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-05-09T06:35:03.670 INFO:teuthology.orchestra.run.smithi003.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-05-09T06:35:03.670 INFO:teuthology.orchestra.run.smithi003.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-05-09T06:35:03.670 INFO:teuthology.orchestra.run.smithi003.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-05-09T06:35:03.670 INFO:teuthology.orchestra.run.smithi003.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-05-09T06:35:03.670 INFO:teuthology.orchestra.run.smithi003.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-05-09T06:35:03.670 INFO:teuthology.orchestra.run.smithi003.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-05-09T06:35:03.670 INFO:teuthology.orchestra.run.smithi003.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-05-09T06:35:03.670 INFO:teuthology.orchestra.run.smithi003.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-05-09T06:35:03.670 INFO:teuthology.orchestra.run.smithi003.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-05-09T06:35:03.670 INFO:teuthology.orchestra.run.smithi003.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-05-09T06:35:03.671 INFO:teuthology.orchestra.run.smithi003.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-05-09T06:35:03.671 INFO:teuthology.orchestra.run.smithi003.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-05-09T06:35:03.671 INFO:teuthology.orchestra.run.smithi003.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-05-09T06:35:03.671 INFO:teuthology.orchestra.run.smithi003.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-05-09T06:35:03.671 INFO:teuthology.orchestra.run.smithi003.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-05-09T06:35:03.671 INFO:teuthology.orchestra.run.smithi003.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-05-09T06:35:03.671 INFO:teuthology.orchestra.run.smithi003.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-05-09T06:35:03.671 INFO:teuthology.orchestra.run.smithi003.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-05-09T06:35:03.671 INFO:teuthology.orchestra.run.smithi003.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-05-09T06:35:03.671 INFO:teuthology.orchestra.run.smithi003.stdout: python3-mako-1.1.4-6.el9.noarch 2024-05-09T06:35:03.671 INFO:teuthology.orchestra.run.smithi003.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-05-09T06:35:03.671 INFO:teuthology.orchestra.run.smithi003.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-05-09T06:35:03.671 INFO:teuthology.orchestra.run.smithi003.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-05-09T06:35:03.671 INFO:teuthology.orchestra.run.smithi003.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-05-09T06:35:03.671 INFO:teuthology.orchestra.run.smithi003.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-05-09T06:35:03.672 INFO:teuthology.orchestra.run.smithi003.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-05-09T06:35:03.672 INFO:teuthology.orchestra.run.smithi003.stdout: python3-packaging-20.9-5.el9.noarch 2024-05-09T06:35:03.672 INFO:teuthology.orchestra.run.smithi003.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-05-09T06:35:03.672 INFO:teuthology.orchestra.run.smithi003.stdout: python3-ply-3.11-14.el9.noarch 2024-05-09T06:35:03.672 INFO:teuthology.orchestra.run.smithi003.stdout: python3-portend-3.1.0-2.el9.noarch 2024-05-09T06:35:03.672 INFO:teuthology.orchestra.run.smithi003.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-05-09T06:35:03.672 INFO:teuthology.orchestra.run.smithi003.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-05-09T06:35:03.672 INFO:teuthology.orchestra.run.smithi003.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-05-09T06:35:03.672 INFO:teuthology.orchestra.run.smithi003.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-05-09T06:35:03.672 INFO:teuthology.orchestra.run.smithi003.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-05-09T06:35:03.672 INFO:teuthology.orchestra.run.smithi003.stdout: python3-pycparser-2.20-6.el9.noarch 2024-05-09T06:35:03.672 INFO:teuthology.orchestra.run.smithi003.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-05-09T06:35:03.672 INFO:teuthology.orchestra.run.smithi003.stdout: python3-pytz-2021.1-5.el9.noarch 2024-05-09T06:35:03.672 INFO:teuthology.orchestra.run.smithi003.stdout: python3-rados-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T06:35:03.672 INFO:teuthology.orchestra.run.smithi003.stdout: python3-rbd-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T06:35:03.672 INFO:teuthology.orchestra.run.smithi003.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-05-09T06:35:03.673 INFO:teuthology.orchestra.run.smithi003.stdout: python3-requests-2.25.1-8.el9.noarch 2024-05-09T06:35:03.673 INFO:teuthology.orchestra.run.smithi003.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-05-09T06:35:03.673 INFO:teuthology.orchestra.run.smithi003.stdout: python3-rgw-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T06:35:03.673 INFO:teuthology.orchestra.run.smithi003.stdout: python3-routes-2.5.1-5.el9.noarch 2024-05-09T06:35:03.673 INFO:teuthology.orchestra.run.smithi003.stdout: python3-rsa-4.9-2.el9.noarch 2024-05-09T06:35:03.673 INFO:teuthology.orchestra.run.smithi003.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-05-09T06:35:03.673 INFO:teuthology.orchestra.run.smithi003.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-05-09T06:35:03.673 INFO:teuthology.orchestra.run.smithi003.stdout: python3-toml-0.10.2-6.el9.noarch 2024-05-09T06:35:03.673 INFO:teuthology.orchestra.run.smithi003.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-05-09T06:35:03.673 INFO:teuthology.orchestra.run.smithi003.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-05-09T06:35:03.673 INFO:teuthology.orchestra.run.smithi003.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-05-09T06:35:03.673 INFO:teuthology.orchestra.run.smithi003.stdout: python3-webob-1.8.7-6.el9.noarch 2024-05-09T06:35:03.673 INFO:teuthology.orchestra.run.smithi003.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-05-09T06:35:03.673 INFO:teuthology.orchestra.run.smithi003.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-05-09T06:35:03.673 INFO:teuthology.orchestra.run.smithi003.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-05-09T06:35:03.674 INFO:teuthology.orchestra.run.smithi003.stdout: qatlib-24.02.0-1.el9.x86_64 2024-05-09T06:35:03.674 INFO:teuthology.orchestra.run.smithi003.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-05-09T06:35:03.674 INFO:teuthology.orchestra.run.smithi003.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-05-09T06:35:03.674 INFO:teuthology.orchestra.run.smithi003.stdout: rbd-fuse-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T06:35:03.674 INFO:teuthology.orchestra.run.smithi003.stdout: rbd-mirror-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T06:35:03.674 INFO:teuthology.orchestra.run.smithi003.stdout: rbd-nbd-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T06:35:03.674 INFO:teuthology.orchestra.run.smithi003.stdout: re2-1:20211101-3.el9.x86_64 2024-05-09T06:35:03.674 INFO:teuthology.orchestra.run.smithi003.stdout: socat-1.7.4.1-5.el9.x86_64 2024-05-09T06:35:03.674 INFO:teuthology.orchestra.run.smithi003.stdout: thrift-0.15.0-2.el9.x86_64 2024-05-09T06:35:03.674 INFO:teuthology.orchestra.run.smithi003.stdout: uuid-1.6.2-55.el9.x86_64 2024-05-09T06:35:03.674 INFO:teuthology.orchestra.run.smithi003.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-05-09T06:35:03.674 INFO:teuthology.orchestra.run.smithi003.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-05-09T06:35:03.675 INFO:teuthology.orchestra.run.smithi003.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-05-09T06:35:03.675 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-09T06:35:03.675 INFO:teuthology.orchestra.run.smithi003.stdout:Complete! 2024-05-09T06:35:03.918 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-09T06:35:03.919 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-05-09T06:35:03.919 INFO:teuthology.orchestra.run.smithi190.stdout:Upgraded: 2024-05-09T06:35:03.919 INFO:teuthology.orchestra.run.smithi190.stdout: NetworkManager-1:1.47.5-1.el9.x86_64 2024-05-09T06:35:03.919 INFO:teuthology.orchestra.run.smithi190.stdout: NetworkManager-libnm-1:1.47.5-1.el9.x86_64 2024-05-09T06:35:03.919 INFO:teuthology.orchestra.run.smithi190.stdout: NetworkManager-team-1:1.47.5-1.el9.x86_64 2024-05-09T06:35:03.919 INFO:teuthology.orchestra.run.smithi190.stdout: NetworkManager-tui-1:1.47.5-1.el9.x86_64 2024-05-09T06:35:03.919 INFO:teuthology.orchestra.run.smithi190.stdout: librados2-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T06:35:03.919 INFO:teuthology.orchestra.run.smithi190.stdout: librbd1-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T06:35:03.919 INFO:teuthology.orchestra.run.smithi190.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-05-09T06:35:03.919 INFO:teuthology.orchestra.run.smithi190.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-05-09T06:35:03.920 INFO:teuthology.orchestra.run.smithi190.stdout:Installed: 2024-05-09T06:35:03.920 INFO:teuthology.orchestra.run.smithi190.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-05-09T06:35:03.920 INFO:teuthology.orchestra.run.smithi190.stdout: automake-1.16.2-8.el9.noarch 2024-05-09T06:35:03.920 INFO:teuthology.orchestra.run.smithi190.stdout: bison-3.7.4-5.el9.x86_64 2024-05-09T06:35:03.920 INFO:teuthology.orchestra.run.smithi190.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-05-09T06:35:03.920 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T06:35:03.920 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-base-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T06:35:03.920 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-common-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T06:35:03.920 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-fuse-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T06:35:03.920 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-grafana-dashboards-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-09T06:35:03.920 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-immutable-object-cache-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T06:35:03.920 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mds-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T06:35:03.920 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mgr-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T06:35:03.920 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mgr-cephadm-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-09T06:35:03.920 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mgr-dashboard-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-09T06:35:03.921 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-09T06:35:03.921 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mgr-modules-core-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-09T06:35:03.921 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mgr-rook-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-09T06:35:03.921 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mon-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T06:35:03.921 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-osd-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T06:35:03.921 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-prometheus-alerts-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-09T06:35:03.921 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-radosgw-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T06:35:03.921 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-selinux-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T06:35:03.921 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-test-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T06:35:03.921 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-volume-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-09T06:35:03.921 INFO:teuthology.orchestra.run.smithi190.stdout: cephadm-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-09T06:35:03.921 INFO:teuthology.orchestra.run.smithi190.stdout: cephfs-mirror-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T06:35:03.921 INFO:teuthology.orchestra.run.smithi190.stdout: cephfs-shell-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T06:35:03.921 INFO:teuthology.orchestra.run.smithi190.stdout: cephfs-top-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-09T06:35:03.921 INFO:teuthology.orchestra.run.smithi190.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-05-09T06:35:03.922 INFO:teuthology.orchestra.run.smithi190.stdout: flex-2.6.4-9.el9.x86_64 2024-05-09T06:35:03.922 INFO:teuthology.orchestra.run.smithi190.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-05-09T06:35:03.922 INFO:teuthology.orchestra.run.smithi190.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-05-09T06:35:03.922 INFO:teuthology.orchestra.run.smithi190.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-05-09T06:35:03.922 INFO:teuthology.orchestra.run.smithi190.stdout: fuse-2.9.9-15.el9.x86_64 2024-05-09T06:35:03.922 INFO:teuthology.orchestra.run.smithi190.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-05-09T06:35:03.922 INFO:teuthology.orchestra.run.smithi190.stdout: grpc-data-1.46.7-10.el9.noarch 2024-05-09T06:35:03.922 INFO:teuthology.orchestra.run.smithi190.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-05-09T06:35:03.922 INFO:teuthology.orchestra.run.smithi190.stdout: libarrow-9.0.0-11.el9.x86_64 2024-05-09T06:35:03.922 INFO:teuthology.orchestra.run.smithi190.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-05-09T06:35:03.922 INFO:teuthology.orchestra.run.smithi190.stdout: libcephfs-devel-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T06:35:03.923 INFO:teuthology.orchestra.run.smithi190.stdout: libcephfs2-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T06:35:03.923 INFO:teuthology.orchestra.run.smithi190.stdout: libcephsqlite-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T06:35:03.923 INFO:teuthology.orchestra.run.smithi190.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-05-09T06:35:03.923 INFO:teuthology.orchestra.run.smithi190.stdout: liboath-2.6.7-2.el9.x86_64 2024-05-09T06:35:03.923 INFO:teuthology.orchestra.run.smithi190.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-05-09T06:35:03.923 INFO:teuthology.orchestra.run.smithi190.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-05-09T06:35:03.923 INFO:teuthology.orchestra.run.smithi190.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-05-09T06:35:03.923 INFO:teuthology.orchestra.run.smithi190.stdout: librados-devel-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T06:35:03.923 INFO:teuthology.orchestra.run.smithi190.stdout: libradosstriper1-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T06:35:03.923 INFO:teuthology.orchestra.run.smithi190.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-05-09T06:35:03.923 INFO:teuthology.orchestra.run.smithi190.stdout: librgw2-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T06:35:03.924 INFO:teuthology.orchestra.run.smithi190.stdout: libtool-2.4.6-45.el9.x86_64 2024-05-09T06:35:03.924 INFO:teuthology.orchestra.run.smithi190.stdout: libunwind-1.6.2-1.el9.x86_64 2024-05-09T06:35:03.924 INFO:teuthology.orchestra.run.smithi190.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-05-09T06:35:03.924 INFO:teuthology.orchestra.run.smithi190.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-05-09T06:35:03.924 INFO:teuthology.orchestra.run.smithi190.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-05-09T06:35:03.924 INFO:teuthology.orchestra.run.smithi190.stdout: lua-5.4.4-4.el9.x86_64 2024-05-09T06:35:03.924 INFO:teuthology.orchestra.run.smithi190.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-05-09T06:35:03.924 INFO:teuthology.orchestra.run.smithi190.stdout: luarocks-3.9.2-1.el9.noarch 2024-05-09T06:35:03.924 INFO:teuthology.orchestra.run.smithi190.stdout: openblas-0.3.21-2.el9.x86_64 2024-05-09T06:35:03.924 INFO:teuthology.orchestra.run.smithi190.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-05-09T06:35:03.924 INFO:teuthology.orchestra.run.smithi190.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-05-09T06:35:03.924 INFO:teuthology.orchestra.run.smithi190.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-05-09T06:35:03.924 INFO:teuthology.orchestra.run.smithi190.stdout: postgresql-13.14-1.el9.x86_64 2024-05-09T06:35:03.924 INFO:teuthology.orchestra.run.smithi190.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-05-09T06:35:03.925 INFO:teuthology.orchestra.run.smithi190.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-05-09T06:35:03.925 INFO:teuthology.orchestra.run.smithi190.stdout: postgresql-server-13.14-1.el9.x86_64 2024-05-09T06:35:03.925 INFO:teuthology.orchestra.run.smithi190.stdout: protobuf-3.14.0-13.el9.x86_64 2024-05-09T06:35:03.925 INFO:teuthology.orchestra.run.smithi190.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-05-09T06:35:03.925 INFO:teuthology.orchestra.run.smithi190.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-05-09T06:35:03.925 INFO:teuthology.orchestra.run.smithi190.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-05-09T06:35:03.925 INFO:teuthology.orchestra.run.smithi190.stdout: python3-babel-2.9.1-2.el9.noarch 2024-05-09T06:35:03.925 INFO:teuthology.orchestra.run.smithi190.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-05-09T06:35:03.925 INFO:teuthology.orchestra.run.smithi190.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-05-09T06:35:03.925 INFO:teuthology.orchestra.run.smithi190.stdout: python3-ceph-argparse-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T06:35:03.925 INFO:teuthology.orchestra.run.smithi190.stdout: python3-ceph-common-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T06:35:03.925 INFO:teuthology.orchestra.run.smithi190.stdout: python3-cephfs-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T06:35:03.925 INFO:teuthology.orchestra.run.smithi190.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-05-09T06:35:03.925 INFO:teuthology.orchestra.run.smithi190.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-05-09T06:35:03.925 INFO:teuthology.orchestra.run.smithi190.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-05-09T06:35:03.925 INFO:teuthology.orchestra.run.smithi190.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-05-09T06:35:03.926 INFO:teuthology.orchestra.run.smithi190.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-05-09T06:35:03.926 INFO:teuthology.orchestra.run.smithi190.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-05-09T06:35:03.926 INFO:teuthology.orchestra.run.smithi190.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-05-09T06:35:03.926 INFO:teuthology.orchestra.run.smithi190.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-05-09T06:35:03.926 INFO:teuthology.orchestra.run.smithi190.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-05-09T06:35:03.926 INFO:teuthology.orchestra.run.smithi190.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-05-09T06:35:03.926 INFO:teuthology.orchestra.run.smithi190.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-05-09T06:35:03.926 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-05-09T06:35:03.926 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-05-09T06:35:03.926 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-05-09T06:35:03.926 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-05-09T06:35:03.926 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-05-09T06:35:03.926 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-05-09T06:35:03.926 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-05-09T06:35:03.927 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-05-09T06:35:03.927 INFO:teuthology.orchestra.run.smithi190.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-05-09T06:35:03.927 INFO:teuthology.orchestra.run.smithi190.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-05-09T06:35:03.927 INFO:teuthology.orchestra.run.smithi190.stdout: python3-mako-1.1.4-6.el9.noarch 2024-05-09T06:35:03.927 INFO:teuthology.orchestra.run.smithi190.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-05-09T06:35:03.927 INFO:teuthology.orchestra.run.smithi190.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-05-09T06:35:03.927 INFO:teuthology.orchestra.run.smithi190.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-05-09T06:35:03.927 INFO:teuthology.orchestra.run.smithi190.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-05-09T06:35:03.927 INFO:teuthology.orchestra.run.smithi190.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-05-09T06:35:03.927 INFO:teuthology.orchestra.run.smithi190.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-05-09T06:35:03.927 INFO:teuthology.orchestra.run.smithi190.stdout: python3-packaging-20.9-5.el9.noarch 2024-05-09T06:35:03.927 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-05-09T06:35:03.927 INFO:teuthology.orchestra.run.smithi190.stdout: python3-ply-3.11-14.el9.noarch 2024-05-09T06:35:03.927 INFO:teuthology.orchestra.run.smithi190.stdout: python3-portend-3.1.0-2.el9.noarch 2024-05-09T06:35:03.928 INFO:teuthology.orchestra.run.smithi190.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-05-09T06:35:03.928 INFO:teuthology.orchestra.run.smithi190.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-05-09T06:35:03.928 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-05-09T06:35:03.928 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-05-09T06:35:03.928 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-05-09T06:35:03.928 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pycparser-2.20-6.el9.noarch 2024-05-09T06:35:03.928 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-05-09T06:35:03.928 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pytz-2021.1-5.el9.noarch 2024-05-09T06:35:03.928 INFO:teuthology.orchestra.run.smithi190.stdout: python3-rados-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T06:35:03.928 INFO:teuthology.orchestra.run.smithi190.stdout: python3-rbd-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T06:35:03.928 INFO:teuthology.orchestra.run.smithi190.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-05-09T06:35:03.928 INFO:teuthology.orchestra.run.smithi190.stdout: python3-requests-2.25.1-8.el9.noarch 2024-05-09T06:35:03.928 INFO:teuthology.orchestra.run.smithi190.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-05-09T06:35:03.928 INFO:teuthology.orchestra.run.smithi190.stdout: python3-rgw-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T06:35:03.928 INFO:teuthology.orchestra.run.smithi190.stdout: python3-routes-2.5.1-5.el9.noarch 2024-05-09T06:35:03.928 INFO:teuthology.orchestra.run.smithi190.stdout: python3-rsa-4.9-2.el9.noarch 2024-05-09T06:35:03.929 INFO:teuthology.orchestra.run.smithi190.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-05-09T06:35:03.929 INFO:teuthology.orchestra.run.smithi190.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-05-09T06:35:03.929 INFO:teuthology.orchestra.run.smithi190.stdout: python3-toml-0.10.2-6.el9.noarch 2024-05-09T06:35:03.929 INFO:teuthology.orchestra.run.smithi190.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-05-09T06:35:03.929 INFO:teuthology.orchestra.run.smithi190.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-05-09T06:35:03.929 INFO:teuthology.orchestra.run.smithi190.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-05-09T06:35:03.929 INFO:teuthology.orchestra.run.smithi190.stdout: python3-webob-1.8.7-6.el9.noarch 2024-05-09T06:35:03.929 INFO:teuthology.orchestra.run.smithi190.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-05-09T06:35:03.929 INFO:teuthology.orchestra.run.smithi190.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-05-09T06:35:03.929 INFO:teuthology.orchestra.run.smithi190.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-05-09T06:35:03.929 INFO:teuthology.orchestra.run.smithi190.stdout: qatlib-24.02.0-1.el9.x86_64 2024-05-09T06:35:03.929 INFO:teuthology.orchestra.run.smithi190.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-05-09T06:35:03.929 INFO:teuthology.orchestra.run.smithi190.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-05-09T06:35:03.930 INFO:teuthology.orchestra.run.smithi190.stdout: rbd-fuse-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T06:35:03.930 INFO:teuthology.orchestra.run.smithi190.stdout: rbd-mirror-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T06:35:03.930 INFO:teuthology.orchestra.run.smithi190.stdout: rbd-nbd-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T06:35:03.930 INFO:teuthology.orchestra.run.smithi190.stdout: re2-1:20211101-3.el9.x86_64 2024-05-09T06:35:03.930 INFO:teuthology.orchestra.run.smithi190.stdout: socat-1.7.4.1-5.el9.x86_64 2024-05-09T06:35:03.930 INFO:teuthology.orchestra.run.smithi190.stdout: thrift-0.15.0-2.el9.x86_64 2024-05-09T06:35:03.930 INFO:teuthology.orchestra.run.smithi190.stdout: uuid-1.6.2-55.el9.x86_64 2024-05-09T06:35:03.930 INFO:teuthology.orchestra.run.smithi190.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-05-09T06:35:03.930 INFO:teuthology.orchestra.run.smithi190.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-05-09T06:35:03.931 INFO:teuthology.orchestra.run.smithi190.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-05-09T06:35:03.931 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-05-09T06:35:03.931 INFO:teuthology.orchestra.run.smithi190.stdout:Complete! 2024-05-09T06:35:04.093 DEBUG:teuthology.parallel:result is None 2024-05-09T06:35:04.387 DEBUG:teuthology.parallel:result is None 2024-05-09T06:35:04.387 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=96805dcf909541c4ca9a1e90258de8ae5b6fa1bc 2024-05-09T06:35:04.612 DEBUG:teuthology.orchestra.run.smithi003:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-09T06:35:04.639 INFO:teuthology.orchestra.run.smithi003.stdout:19.0.0-2515.g96805dcf.el9 2024-05-09T06:35:04.640 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2515.g96805dcf.el9 2024-05-09T06:35:04.640 INFO:teuthology.task.install:The correct ceph version 19.0.0-2515.g96805dcf is installed. 2024-05-09T06:35:04.642 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=96805dcf909541c4ca9a1e90258de8ae5b6fa1bc 2024-05-09T06:35:04.861 DEBUG:teuthology.orchestra.run.smithi190:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-09T06:35:04.885 INFO:teuthology.orchestra.run.smithi190.stdout:19.0.0-2515.g96805dcf.el9 2024-05-09T06:35:04.886 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2515.g96805dcf.el9 2024-05-09T06:35:04.886 INFO:teuthology.task.install:The correct ceph version 19.0.0-2515.g96805dcf is installed. 2024-05-09T06:35:04.888 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-05-09T06:35:04.888 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-05-09T06:35:04.888 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-09T06:35:04.922 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-05-09T06:35:04.923 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-09T06:35:04.955 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-05-09T06:35:04.955 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-05-09T06:35:04.955 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd of=/usr/bin/daemon-helper 2024-05-09T06:35:04.992 DEBUG:teuthology.orchestra.run.smithi003:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-09T06:35:05.061 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-05-09T06:35:05.061 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd of=/usr/bin/daemon-helper 2024-05-09T06:35:05.088 DEBUG:teuthology.orchestra.run.smithi190:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-09T06:35:05.157 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-05-09T06:35:05.157 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-05-09T06:35:05.157 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-09T06:35:05.186 DEBUG:teuthology.orchestra.run.smithi003:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-09T06:35:05.254 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-05-09T06:35:05.254 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-09T06:35:05.279 DEBUG:teuthology.orchestra.run.smithi190:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-09T06:35:05.344 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-05-09T06:35:05.345 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-05-09T06:35:05.345 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd of=/usr/bin/stdin-killer 2024-05-09T06:35:05.373 DEBUG:teuthology.orchestra.run.smithi003:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-09T06:35:05.442 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-05-09T06:35:05.442 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd of=/usr/bin/stdin-killer 2024-05-09T06:35:05.467 DEBUG:teuthology.orchestra.run.smithi190:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-09T06:35:05.532 INFO:teuthology.run_tasks:Running task ceph... 2024-05-09T06:35:05.619 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-05-09T06:35:05.619 DEBUG:teuthology.orchestra.run.smithi003:> sudo chmod 777 /var/log/ceph 2024-05-09T06:35:05.622 DEBUG:teuthology.orchestra.run.smithi190:> sudo chmod 777 /var/log/ceph 2024-05-09T06:35:05.647 INFO:tasks.ceph:Disabling ceph logrotate... 2024-05-09T06:35:05.647 DEBUG:teuthology.orchestra.run.smithi003:> sudo rm -f -- /etc/logrotate.d/ceph 2024-05-09T06:35:05.690 DEBUG:teuthology.orchestra.run.smithi190:> sudo rm -f -- /etc/logrotate.d/ceph 2024-05-09T06:35:05.715 INFO:tasks.ceph:Creating extra log directories... 2024-05-09T06:35:05.716 DEBUG:teuthology.orchestra.run.smithi003:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-05-09T06:35:05.758 DEBUG:teuthology.orchestra.run.smithi190:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-05-09T06:35:05.789 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-05-09T06:35:05.790 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-05-09T06:35:05.790 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-05-09T06:35:05.790 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-05-09T06:35:05.790 DEBUG:teuthology.orchestra.run.smithi003:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-05-09T06:35:05.840 DEBUG:teuthology.orchestra.run.smithi003:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-05-09T06:35:05.955 DEBUG:teuthology.orchestra.run.smithi003:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-05-09T06:35:05.982 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-05-09T06:35:05.982 DEBUG:teuthology.orchestra.run.smithi190:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-05-09T06:35:05.997 DEBUG:teuthology.orchestra.run.smithi190:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-05-09T06:35:06.108 DEBUG:teuthology.orchestra.run.smithi190:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-05-09T06:35:06.153 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-05-09T06:35:06.153 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-05-09T06:35:06.153 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/teuthology-2024-05-08_21:24:04-fs-squid-distro-default-smithi/7698541', 'branch': 'squid', 'description': 'fs/volumes/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/volumes/{overrides test/finisher_per_module}}', 'email': 'ceph-qa@ceph.com', 'first_in_suite': False, 'job_id': '7698541', 'kernel': {'client': {'sha1': 'distro'}, 'kdb': 1, 'sha1': 'distro'}, 'ktype': 'distro', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'mgrmodules': {'sequential': [{'print': 'Enabling mgr modules'}]}, 'name': 'teuthology-2024-05-08_21:24:04-fs-squid-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'squid'}, 'ceph': {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'sha1': '96805dcf909541c4ca9a1e90258de8ae5b6fa1bc'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '96805dcf909541c4ca9a1e90258de8ae5b6fa1bc'}}, 'kclient': {'syntax': 'v2'}, 'selinux': {'allowlist': ['scontext=system_u:system_r:getty_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'squid', 'sha1': 'be85c202969d25af6e130678e7d73a20e9c4ca11'}}, 'owner': 'scheduled_teuthology@teuthology', 'priority': 100, 'repo': 'https://git.ceph.com/ceph.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 2334, 'sha1': '96805dcf909541c4ca9a1e90258de8ae5b6fa1bc', 'sleep_before_teardown': 0, 'subset': '359/512', 'suite': 'fs', 'suite_branch': 'squid', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph_be85c202969d25af6e130678e7d73a20e9c4ca11/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph.git', 'suite_sha1': 'be85c202969d25af6e130678e7d73a20e9c4ca11', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi003.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHDTU59s3KInmJs3dd/8Ydtk7rpoIaz7d+fqqmrBTda2032gaHXoylm0yOE65o6JjhGJ7B/rmm+wl2vgamxnxr8=', 'smithi190.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOd0Ny4wJnHAMIg19Z/g3f4ECf1K802dz6GWEEPGbzfBK6w5EeLqpVO0/i8FObMdi5eYZEtQxGdnQq5AcppZ2cs='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'client': {'sha1': 'distro'}, 'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '96805dcf909541c4ca9a1e90258de8ae5b6fa1bc'}}, {'ceph': {'create_rbd_pool': False, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'sha1': '96805dcf909541c4ca9a1e90258de8ae5b6fa1bc', 'cluster': 'ceph'}}, {'sequential': ['mgrmodules']}, {'kclient': None}, {'check-counter': {'counters': {'mgr': [{'min': 4, 'name': 'finisher-volumes.complete_latency.avgcount'}, {'expected_val': 0, 'name': 'finisher-volumes.queue_len'}]}}}, {'cephfs_test_runner': {'fail_on_skip': False, 'modules': ['tasks.cephfs.test_volumes.TestPerModuleFinsherThread']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ["local function is_kupstream() return yaml.ktype == 'upstream' end", "local function is_kdistro() return yaml.ktype == 'distro' end", 'local function is_fuse() return false end', 'local function syntax_version() return yaml.overrides.kclient.syntax end']}, 'teuthology_branch': 'main', 'teuthology_sha1': '1ae7ad82388e92a475afff437d49054826c019a1', 'timestamp': '2024-05-08_21:24:04', 'tube': 'smithi', 'user': 'teuthology', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885'} 2024-05-09T06:35:06.154 DEBUG:teuthology.orchestra.run.smithi003:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-05-09T06:35:06.156 DEBUG:teuthology.orchestra.run.smithi190:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-05-09T06:35:06.203 DEBUG:teuthology.orchestra.run.smithi003:> sudo install -d -m0777 -- /var/run/ceph 2024-05-09T06:35:06.213 DEBUG:teuthology.orchestra.run.smithi190:> sudo install -d -m0777 -- /var/run/ceph 2024-05-09T06:35:06.268 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-05-09T06:35:06.268 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/scratch_devs of=/dev/stdout 2024-05-09T06:35:06.293 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-09T06:35:06.293 DEBUG:teuthology.orchestra.run.smithi003:> stat /dev/vg_nvme/lv_1 2024-05-09T06:35:06.348 INFO:teuthology.orchestra.run.smithi003.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-09T06:35:06.349 INFO:teuthology.orchestra.run.smithi003.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-09T06:35:06.349 INFO:teuthology.orchestra.run.smithi003.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-05-09T06:35:06.349 INFO:teuthology.orchestra.run.smithi003.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-09T06:35:06.349 INFO:teuthology.orchestra.run.smithi003.stdout:Context: system_u:object_r:device_t:s0 2024-05-09T06:35:06.349 INFO:teuthology.orchestra.run.smithi003.stdout:Access: 2024-05-09 06:34:55.728547155 +0000 2024-05-09T06:35:06.349 INFO:teuthology.orchestra.run.smithi003.stdout:Modify: 2024-05-09 06:32:53.859416777 +0000 2024-05-09T06:35:06.349 INFO:teuthology.orchestra.run.smithi003.stdout:Change: 2024-05-09 06:32:53.859416777 +0000 2024-05-09T06:35:06.349 INFO:teuthology.orchestra.run.smithi003.stdout: Birth: 2024-05-09 06:32:53.859416777 +0000 2024-05-09T06:35:06.349 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-09T06:35:06.413 INFO:teuthology.orchestra.run.smithi003.stderr:1+0 records in 2024-05-09T06:35:06.413 INFO:teuthology.orchestra.run.smithi003.stderr:1+0 records out 2024-05-09T06:35:06.413 INFO:teuthology.orchestra.run.smithi003.stderr:512 bytes copied, 0.000202735 s, 2.5 MB/s 2024-05-09T06:35:06.414 DEBUG:teuthology.orchestra.run.smithi003:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-09T06:35:06.472 DEBUG:teuthology.orchestra.run.smithi003:> stat /dev/vg_nvme/lv_2 2024-05-09T06:35:06.529 INFO:teuthology.orchestra.run.smithi003.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-09T06:35:06.529 INFO:teuthology.orchestra.run.smithi003.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-09T06:35:06.529 INFO:teuthology.orchestra.run.smithi003.stdout:Device: 5h/5d Inode: 687 Links: 1 2024-05-09T06:35:06.529 INFO:teuthology.orchestra.run.smithi003.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-09T06:35:06.529 INFO:teuthology.orchestra.run.smithi003.stdout:Context: system_u:object_r:device_t:s0 2024-05-09T06:35:06.529 INFO:teuthology.orchestra.run.smithi003.stdout:Access: 2024-05-09 06:34:55.729547142 +0000 2024-05-09T06:35:06.529 INFO:teuthology.orchestra.run.smithi003.stdout:Modify: 2024-05-09 06:32:54.218412041 +0000 2024-05-09T06:35:06.529 INFO:teuthology.orchestra.run.smithi003.stdout:Change: 2024-05-09 06:32:54.218412041 +0000 2024-05-09T06:35:06.529 INFO:teuthology.orchestra.run.smithi003.stdout: Birth: 2024-05-09 06:32:54.218412041 +0000 2024-05-09T06:35:06.530 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-09T06:35:06.593 INFO:teuthology.orchestra.run.smithi003.stderr:1+0 records in 2024-05-09T06:35:06.593 INFO:teuthology.orchestra.run.smithi003.stderr:1+0 records out 2024-05-09T06:35:06.593 INFO:teuthology.orchestra.run.smithi003.stderr:512 bytes copied, 0.000192267 s, 2.7 MB/s 2024-05-09T06:35:06.595 DEBUG:teuthology.orchestra.run.smithi003:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-09T06:35:06.651 DEBUG:teuthology.orchestra.run.smithi003:> stat /dev/vg_nvme/lv_3 2024-05-09T06:35:06.708 INFO:teuthology.orchestra.run.smithi003.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-09T06:35:06.708 INFO:teuthology.orchestra.run.smithi003.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-09T06:35:06.708 INFO:teuthology.orchestra.run.smithi003.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-05-09T06:35:06.708 INFO:teuthology.orchestra.run.smithi003.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-09T06:35:06.708 INFO:teuthology.orchestra.run.smithi003.stdout:Context: system_u:object_r:device_t:s0 2024-05-09T06:35:06.708 INFO:teuthology.orchestra.run.smithi003.stdout:Access: 2024-05-09 06:34:55.729547142 +0000 2024-05-09T06:35:06.708 INFO:teuthology.orchestra.run.smithi003.stdout:Modify: 2024-05-09 06:32:54.569407411 +0000 2024-05-09T06:35:06.708 INFO:teuthology.orchestra.run.smithi003.stdout:Change: 2024-05-09 06:32:54.569407411 +0000 2024-05-09T06:35:06.708 INFO:teuthology.orchestra.run.smithi003.stdout: Birth: 2024-05-09 06:32:54.569407411 +0000 2024-05-09T06:35:06.708 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-09T06:35:06.772 INFO:teuthology.orchestra.run.smithi003.stderr:1+0 records in 2024-05-09T06:35:06.772 INFO:teuthology.orchestra.run.smithi003.stderr:1+0 records out 2024-05-09T06:35:06.772 INFO:teuthology.orchestra.run.smithi003.stderr:512 bytes copied, 0.000204225 s, 2.5 MB/s 2024-05-09T06:35:06.773 DEBUG:teuthology.orchestra.run.smithi003:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-09T06:35:06.831 DEBUG:teuthology.orchestra.run.smithi003:> stat /dev/vg_nvme/lv_4 2024-05-09T06:35:06.887 INFO:teuthology.orchestra.run.smithi003.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-09T06:35:06.887 INFO:teuthology.orchestra.run.smithi003.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-09T06:35:06.887 INFO:teuthology.orchestra.run.smithi003.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-05-09T06:35:06.887 INFO:teuthology.orchestra.run.smithi003.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-09T06:35:06.887 INFO:teuthology.orchestra.run.smithi003.stdout:Context: system_u:object_r:device_t:s0 2024-05-09T06:35:06.887 INFO:teuthology.orchestra.run.smithi003.stdout:Access: 2024-05-09 06:34:55.729547142 +0000 2024-05-09T06:35:06.887 INFO:teuthology.orchestra.run.smithi003.stdout:Modify: 2024-05-09 06:32:54.928402675 +0000 2024-05-09T06:35:06.887 INFO:teuthology.orchestra.run.smithi003.stdout:Change: 2024-05-09 06:32:54.928402675 +0000 2024-05-09T06:35:06.887 INFO:teuthology.orchestra.run.smithi003.stdout: Birth: 2024-05-09 06:32:54.928402675 +0000 2024-05-09T06:35:06.888 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-09T06:35:06.951 INFO:teuthology.orchestra.run.smithi003.stderr:1+0 records in 2024-05-09T06:35:06.951 INFO:teuthology.orchestra.run.smithi003.stderr:1+0 records out 2024-05-09T06:35:06.951 INFO:teuthology.orchestra.run.smithi003.stderr:512 bytes copied, 0.000229335 s, 2.2 MB/s 2024-05-09T06:35:06.953 DEBUG:teuthology.orchestra.run.smithi003:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-09T06:35:07.009 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-05-09T06:35:07.010 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-05-09T06:35:07.010 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/scratch_devs of=/dev/stdout 2024-05-09T06:35:07.027 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-09T06:35:07.027 DEBUG:teuthology.orchestra.run.smithi190:> stat /dev/vg_nvme/lv_1 2024-05-09T06:35:07.084 INFO:teuthology.orchestra.run.smithi190.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-09T06:35:07.084 INFO:teuthology.orchestra.run.smithi190.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-09T06:35:07.084 INFO:teuthology.orchestra.run.smithi190.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-05-09T06:35:07.084 INFO:teuthology.orchestra.run.smithi190.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-09T06:35:07.084 INFO:teuthology.orchestra.run.smithi190.stdout:Context: system_u:object_r:device_t:s0 2024-05-09T06:35:07.085 INFO:teuthology.orchestra.run.smithi190.stdout:Access: 2024-05-09 06:34:56.206939084 +0000 2024-05-09T06:35:07.085 INFO:teuthology.orchestra.run.smithi190.stdout:Modify: 2024-05-09 06:32:53.253350232 +0000 2024-05-09T06:35:07.085 INFO:teuthology.orchestra.run.smithi190.stdout:Change: 2024-05-09 06:32:53.253350232 +0000 2024-05-09T06:35:07.085 INFO:teuthology.orchestra.run.smithi190.stdout: Birth: 2024-05-09 06:32:53.253350232 +0000 2024-05-09T06:35:07.085 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-09T06:35:07.150 INFO:teuthology.orchestra.run.smithi190.stderr:1+0 records in 2024-05-09T06:35:07.150 INFO:teuthology.orchestra.run.smithi190.stderr:1+0 records out 2024-05-09T06:35:07.150 INFO:teuthology.orchestra.run.smithi190.stderr:512 bytes copied, 0.000272761 s, 1.9 MB/s 2024-05-09T06:35:07.151 DEBUG:teuthology.orchestra.run.smithi190:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-09T06:35:07.209 DEBUG:teuthology.orchestra.run.smithi190:> stat /dev/vg_nvme/lv_2 2024-05-09T06:35:07.266 INFO:teuthology.orchestra.run.smithi190.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-09T06:35:07.266 INFO:teuthology.orchestra.run.smithi190.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-09T06:35:07.266 INFO:teuthology.orchestra.run.smithi190.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-05-09T06:35:07.266 INFO:teuthology.orchestra.run.smithi190.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-09T06:35:07.266 INFO:teuthology.orchestra.run.smithi190.stdout:Context: system_u:object_r:device_t:s0 2024-05-09T06:35:07.266 INFO:teuthology.orchestra.run.smithi190.stdout:Access: 2024-05-09 06:34:56.207939055 +0000 2024-05-09T06:35:07.266 INFO:teuthology.orchestra.run.smithi190.stdout:Modify: 2024-05-09 06:32:53.637339075 +0000 2024-05-09T06:35:07.266 INFO:teuthology.orchestra.run.smithi190.stdout:Change: 2024-05-09 06:32:53.637339075 +0000 2024-05-09T06:35:07.266 INFO:teuthology.orchestra.run.smithi190.stdout: Birth: 2024-05-09 06:32:53.637339075 +0000 2024-05-09T06:35:07.267 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-09T06:35:07.330 INFO:teuthology.orchestra.run.smithi190.stderr:1+0 records in 2024-05-09T06:35:07.330 INFO:teuthology.orchestra.run.smithi190.stderr:1+0 records out 2024-05-09T06:35:07.330 INFO:teuthology.orchestra.run.smithi190.stderr:512 bytes copied, 0.00021613 s, 2.4 MB/s 2024-05-09T06:35:07.331 DEBUG:teuthology.orchestra.run.smithi190:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-09T06:35:07.388 DEBUG:teuthology.orchestra.run.smithi190:> stat /dev/vg_nvme/lv_3 2024-05-09T06:35:07.444 INFO:teuthology.orchestra.run.smithi190.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-09T06:35:07.444 INFO:teuthology.orchestra.run.smithi190.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-09T06:35:07.445 INFO:teuthology.orchestra.run.smithi190.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-05-09T06:35:07.445 INFO:teuthology.orchestra.run.smithi190.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-09T06:35:07.445 INFO:teuthology.orchestra.run.smithi190.stdout:Context: system_u:object_r:device_t:s0 2024-05-09T06:35:07.445 INFO:teuthology.orchestra.run.smithi190.stdout:Access: 2024-05-09 06:34:56.207939055 +0000 2024-05-09T06:35:07.445 INFO:teuthology.orchestra.run.smithi190.stdout:Modify: 2024-05-09 06:32:53.983329022 +0000 2024-05-09T06:35:07.445 INFO:teuthology.orchestra.run.smithi190.stdout:Change: 2024-05-09 06:32:53.983329022 +0000 2024-05-09T06:35:07.445 INFO:teuthology.orchestra.run.smithi190.stdout: Birth: 2024-05-09 06:32:53.983329022 +0000 2024-05-09T06:35:07.445 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-09T06:35:07.508 INFO:teuthology.orchestra.run.smithi190.stderr:1+0 records in 2024-05-09T06:35:07.508 INFO:teuthology.orchestra.run.smithi190.stderr:1+0 records out 2024-05-09T06:35:07.508 INFO:teuthology.orchestra.run.smithi190.stderr:512 bytes copied, 0.000196148 s, 2.6 MB/s 2024-05-09T06:35:07.509 DEBUG:teuthology.orchestra.run.smithi190:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-09T06:35:07.565 DEBUG:teuthology.orchestra.run.smithi190:> stat /dev/vg_nvme/lv_4 2024-05-09T06:35:07.621 INFO:teuthology.orchestra.run.smithi190.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-09T06:35:07.621 INFO:teuthology.orchestra.run.smithi190.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-09T06:35:07.621 INFO:teuthology.orchestra.run.smithi190.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-05-09T06:35:07.621 INFO:teuthology.orchestra.run.smithi190.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-09T06:35:07.621 INFO:teuthology.orchestra.run.smithi190.stdout:Context: system_u:object_r:device_t:s0 2024-05-09T06:35:07.621 INFO:teuthology.orchestra.run.smithi190.stdout:Access: 2024-05-09 06:34:56.207939055 +0000 2024-05-09T06:35:07.621 INFO:teuthology.orchestra.run.smithi190.stdout:Modify: 2024-05-09 06:32:54.324319115 +0000 2024-05-09T06:35:07.621 INFO:teuthology.orchestra.run.smithi190.stdout:Change: 2024-05-09 06:32:54.324319115 +0000 2024-05-09T06:35:07.621 INFO:teuthology.orchestra.run.smithi190.stdout: Birth: 2024-05-09 06:32:54.324319115 +0000 2024-05-09T06:35:07.622 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-09T06:35:07.685 INFO:teuthology.orchestra.run.smithi190.stderr:1+0 records in 2024-05-09T06:35:07.685 INFO:teuthology.orchestra.run.smithi190.stderr:1+0 records out 2024-05-09T06:35:07.685 INFO:teuthology.orchestra.run.smithi190.stderr:512 bytes copied, 0.000215215 s, 2.4 MB/s 2024-05-09T06:35:07.686 DEBUG:teuthology.orchestra.run.smithi190:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-09T06:35:07.743 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-05-09T06:35:07.743 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi003.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi190.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-05-09T06:35:07.743 INFO:tasks.ceph:Generating config... 2024-05-09T06:35:07.744 INFO:tasks.ceph:[client] client mount timeout = 600 2024-05-09T06:35:07.744 INFO:tasks.ceph:[client] debug client = 20 2024-05-09T06:35:07.744 INFO:tasks.ceph:[client] debug ms = 1 2024-05-09T06:35:07.744 INFO:tasks.ceph:[client] pid file = 2024-05-09T06:35:07.744 INFO:tasks.ceph:[client] rados mon op timeout = 900 2024-05-09T06:35:07.744 INFO:tasks.ceph:[client] rados osd op timeout = 900 2024-05-09T06:35:07.744 INFO:tasks.ceph:[global] ms die on skipped message = False 2024-05-09T06:35:07.744 INFO:tasks.ceph:[mds] debug mds = 20 2024-05-09T06:35:07.745 INFO:tasks.ceph:[mds] debug mds balancer = 20 2024-05-09T06:35:07.745 INFO:tasks.ceph:[mds] debug ms = 1 2024-05-09T06:35:07.745 INFO:tasks.ceph:[mds] mds debug frag = True 2024-05-09T06:35:07.745 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2024-05-09T06:35:07.745 INFO:tasks.ceph:[mds] mds op complaint time = 180 2024-05-09T06:35:07.745 INFO:tasks.ceph:[mds] mds verify scatter = True 2024-05-09T06:35:07.745 INFO:tasks.ceph:[mds] osd op complaint time = 180 2024-05-09T06:35:07.745 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2024-05-09T06:35:07.745 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2024-05-09T06:35:07.745 INFO:tasks.ceph:[mgr] debug client = 20 2024-05-09T06:35:07.745 INFO:tasks.ceph:[mgr] debug finisher = 20 2024-05-09T06:35:07.746 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-05-09T06:35:07.746 INFO:tasks.ceph:[mgr] debug ms = 1 2024-05-09T06:35:07.746 INFO:tasks.ceph:[mon] debug mon = 20 2024-05-09T06:35:07.746 INFO:tasks.ceph:[mon] debug ms = 1 2024-05-09T06:35:07.746 INFO:tasks.ceph:[mon] debug paxos = 20 2024-05-09T06:35:07.746 INFO:tasks.ceph:[mon] mon down mkfs grace = 300 2024-05-09T06:35:07.746 INFO:tasks.ceph:[mon] mon op complaint time = 120 2024-05-09T06:35:07.746 INFO:tasks.ceph:[osd] bdev async discard = True 2024-05-09T06:35:07.746 INFO:tasks.ceph:[osd] bdev enable discard = True 2024-05-09T06:35:07.746 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2024-05-09T06:35:07.746 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-05-09T06:35:07.746 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-05-09T06:35:07.746 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2024-05-09T06:35:07.746 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2024-05-09T06:35:07.746 INFO:tasks.ceph:[osd] debug ms = 1 2024-05-09T06:35:07.747 INFO:tasks.ceph:[osd] debug osd = 20 2024-05-09T06:35:07.747 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2024-05-09T06:35:07.747 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-05-09T06:35:07.747 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-05-09T06:35:07.747 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-05-09T06:35:07.747 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-05-09T06:35:07.747 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-05-09T06:35:07.747 INFO:tasks.ceph:[osd] osd op complaint time = 180 2024-05-09T06:35:07.747 INFO:tasks.ceph:Setting up mon.a... 2024-05-09T06:35:07.747 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-05-09T06:35:07.803 INFO:teuthology.orchestra.run.smithi003.stdout:creating /etc/ceph/ceph.keyring 2024-05-09T06:35:07.806 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-05-09T06:35:07.855 DEBUG:teuthology.orchestra.run.smithi003:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-05-09T06:35:07.881 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.3'), ('mon.b', '172.21.15.190'), ('mon.c', '[v2:172.21.15.190:3301,v1:172.21.15.190:6790]')] 2024-05-09T06:35:07.882 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.3,172.21.15.190,[v2:172.21.15.190:3301,v1:172.21.15.190:6790]', 'ms die on skipped message': False}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false', 'debug client': 20, 'debug finisher': 20}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2024-05-09T06:35:07.883 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-05-09T06:35:07.883 DEBUG:teuthology.orchestra.run.smithi003:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-05-09T06:35:07.938 DEBUG:teuthology.orchestra.run.smithi003:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.3 --add b 172.21.15.190 --addv c '[v2:172.21.15.190:3301,v1:172.21.15.190:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-05-09T06:35:08.027 INFO:teuthology.orchestra.run.smithi003.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-05-09T06:35:08.027 INFO:teuthology.orchestra.run.smithi003.stdout:monmaptool: generated fsid 16ed737d-2665-431a-8ac6-ef242be3975a 2024-05-09T06:35:08.027 INFO:teuthology.orchestra.run.smithi003.stdout:setting min_mon_release = quincy 2024-05-09T06:35:08.027 INFO:teuthology.orchestra.run.smithi003.stdout:epoch 0 2024-05-09T06:35:08.027 INFO:teuthology.orchestra.run.smithi003.stdout:fsid 16ed737d-2665-431a-8ac6-ef242be3975a 2024-05-09T06:35:08.027 INFO:teuthology.orchestra.run.smithi003.stdout:last_changed 2024-05-09T06:35:08.047677+0000 2024-05-09T06:35:08.028 INFO:teuthology.orchestra.run.smithi003.stdout:created 2024-05-09T06:35:08.047677+0000 2024-05-09T06:35:08.028 INFO:teuthology.orchestra.run.smithi003.stdout:min_mon_release 17 (quincy) 2024-05-09T06:35:08.028 INFO:teuthology.orchestra.run.smithi003.stdout:election_strategy: 1 2024-05-09T06:35:08.028 INFO:teuthology.orchestra.run.smithi003.stdout:0: [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] mon.a 2024-05-09T06:35:08.028 INFO:teuthology.orchestra.run.smithi003.stdout:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.b 2024-05-09T06:35:08.028 INFO:teuthology.orchestra.run.smithi003.stdout:2: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.c 2024-05-09T06:35:08.028 INFO:teuthology.orchestra.run.smithi003.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-05-09T06:35:08.029 DEBUG:teuthology.orchestra.run.smithi003:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-05-09T06:35:08.045 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 16ed737d-2665-431a-8ac6-ef242be3975a... 2024-05-09T06:35:08.046 DEBUG:teuthology.orchestra.run.smithi003:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-05-09T06:35:08.088 DEBUG:teuthology.orchestra.run.smithi190:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-05-09T06:35:08.130 INFO:teuthology.orchestra.run.smithi003.stdout:[global] 2024-05-09T06:35:08.131 INFO:teuthology.orchestra.run.smithi003.stdout: chdir = "" 2024-05-09T06:35:08.131 INFO:teuthology.orchestra.run.smithi003.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-05-09T06:35:08.131 INFO:teuthology.orchestra.run.smithi003.stdout: auth supported = cephx 2024-05-09T06:35:08.131 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-09T06:35:08.131 INFO:teuthology.orchestra.run.smithi003.stdout: filestore xattr use omap = true 2024-05-09T06:35:08.131 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-09T06:35:08.131 INFO:teuthology.orchestra.run.smithi003.stdout: mon clock drift allowed = 1.000 2024-05-09T06:35:08.131 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-09T06:35:08.132 INFO:teuthology.orchestra.run.smithi003.stdout: osd crush chooseleaf type = 0 2024-05-09T06:35:08.132 INFO:teuthology.orchestra.run.smithi003.stdout: auth debug = true 2024-05-09T06:35:08.132 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-09T06:35:08.132 INFO:teuthology.orchestra.run.smithi003.stdout: ms die on old message = true 2024-05-09T06:35:08.132 INFO:teuthology.orchestra.run.smithi003.stdout: ms die on bug = true 2024-05-09T06:35:08.132 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-09T06:35:08.132 INFO:teuthology.orchestra.run.smithi003.stdout: mon max pg per osd = 10000 # >= luminous 2024-05-09T06:35:08.132 INFO:teuthology.orchestra.run.smithi003.stdout: mon pg warn max object skew = 0 2024-05-09T06:35:08.132 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-09T06:35:08.132 INFO:teuthology.orchestra.run.smithi003.stdout: # disable pg_autoscaler by default for new pools 2024-05-09T06:35:08.132 INFO:teuthology.orchestra.run.smithi003.stdout: osd_pool_default_pg_autoscale_mode = off 2024-05-09T06:35:08.132 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-09T06:35:08.132 INFO:teuthology.orchestra.run.smithi003.stdout: osd pool default size = 2 2024-05-09T06:35:08.132 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-09T06:35:08.132 INFO:teuthology.orchestra.run.smithi003.stdout: mon osd allow primary affinity = true 2024-05-09T06:35:08.133 INFO:teuthology.orchestra.run.smithi003.stdout: mon osd allow pg remap = true 2024-05-09T06:35:08.133 INFO:teuthology.orchestra.run.smithi003.stdout: mon warn on legacy crush tunables = false 2024-05-09T06:35:08.133 INFO:teuthology.orchestra.run.smithi003.stdout: mon warn on crush straw calc version zero = false 2024-05-09T06:35:08.133 INFO:teuthology.orchestra.run.smithi003.stdout: mon warn on no sortbitwise = false 2024-05-09T06:35:08.133 INFO:teuthology.orchestra.run.smithi003.stdout: mon warn on osd down out interval zero = false 2024-05-09T06:35:08.133 INFO:teuthology.orchestra.run.smithi003.stdout: mon warn on too few osds = false 2024-05-09T06:35:08.133 INFO:teuthology.orchestra.run.smithi003.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-05-09T06:35:08.133 INFO:teuthology.orchestra.run.smithi003.stdout: mon_warn_on_pool_no_redundancy = false 2024-05-09T06:35:08.133 INFO:teuthology.orchestra.run.smithi003.stdout: mon_allow_pool_size_one = true 2024-05-09T06:35:08.133 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-09T06:35:08.133 INFO:teuthology.orchestra.run.smithi003.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-05-09T06:35:08.133 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-09T06:35:08.133 INFO:teuthology.orchestra.run.smithi003.stdout: osd default data pool replay window = 5 2024-05-09T06:35:08.134 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-09T06:35:08.134 INFO:teuthology.orchestra.run.smithi003.stdout: mon allow pool delete = true 2024-05-09T06:35:08.134 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-09T06:35:08.134 INFO:teuthology.orchestra.run.smithi003.stdout: mon cluster log file level = debug 2024-05-09T06:35:08.134 INFO:teuthology.orchestra.run.smithi003.stdout: debug asserts on shutdown = true 2024-05-09T06:35:08.134 INFO:teuthology.orchestra.run.smithi003.stdout: mon health detail to clog = false 2024-05-09T06:35:08.134 INFO:teuthology.orchestra.run.smithi003.stdout: mon host = "172.21.15.3,172.21.15.190,[v2:172.21.15.190:3301,v1:172.21.15.190:6790]" 2024-05-09T06:35:08.134 INFO:teuthology.orchestra.run.smithi003.stdout: ms die on skipped message = False 2024-05-09T06:35:08.134 INFO:teuthology.orchestra.run.smithi003.stdout: fsid = 16ed737d-2665-431a-8ac6-ef242be3975a 2024-05-09T06:35:08.134 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-09T06:35:08.134 INFO:teuthology.orchestra.run.smithi003.stdout:[osd] 2024-05-09T06:35:08.134 INFO:teuthology.orchestra.run.smithi003.stdout: osd journal size = 100 2024-05-09T06:35:08.134 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-09T06:35:08.134 INFO:teuthology.orchestra.run.smithi003.stdout: osd scrub load threshold = 5.0 2024-05-09T06:35:08.135 INFO:teuthology.orchestra.run.smithi003.stdout: osd scrub max interval = 600 2024-05-09T06:35:08.135 INFO:teuthology.orchestra.run.smithi003.stdout: osd mclock profile = high_recovery_ops 2024-05-09T06:35:08.135 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-09T06:35:08.135 INFO:teuthology.orchestra.run.smithi003.stdout: osd recover clone overlap = true 2024-05-09T06:35:08.135 INFO:teuthology.orchestra.run.smithi003.stdout: osd recovery max chunk = 1048576 2024-05-09T06:35:08.135 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-09T06:35:08.135 INFO:teuthology.orchestra.run.smithi003.stdout: osd debug shutdown = true 2024-05-09T06:35:08.135 INFO:teuthology.orchestra.run.smithi003.stdout: osd debug op order = true 2024-05-09T06:35:08.135 INFO:teuthology.orchestra.run.smithi003.stdout: osd debug verify stray on activate = true 2024-05-09T06:35:08.135 INFO:teuthology.orchestra.run.smithi003.stdout: osd debug trim objects = true 2024-05-09T06:35:08.135 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-09T06:35:08.135 INFO:teuthology.orchestra.run.smithi003.stdout: osd open classes on start = true 2024-05-09T06:35:08.135 INFO:teuthology.orchestra.run.smithi003.stdout: osd debug pg log writeout = true 2024-05-09T06:35:08.135 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-09T06:35:08.136 INFO:teuthology.orchestra.run.smithi003.stdout: osd deep scrub update digest min age = 30 2024-05-09T06:35:08.136 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-09T06:35:08.136 INFO:teuthology.orchestra.run.smithi003.stdout: osd map max advance = 10 2024-05-09T06:35:08.136 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-09T06:35:08.136 INFO:teuthology.orchestra.run.smithi003.stdout: journal zero on create = true 2024-05-09T06:35:08.136 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-09T06:35:08.136 INFO:teuthology.orchestra.run.smithi003.stdout: filestore ondisk finisher threads = 3 2024-05-09T06:35:08.136 INFO:teuthology.orchestra.run.smithi003.stdout: filestore apply finisher threads = 3 2024-05-09T06:35:08.136 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-09T06:35:08.136 INFO:teuthology.orchestra.run.smithi003.stdout: bdev debug aio = true 2024-05-09T06:35:08.136 INFO:teuthology.orchestra.run.smithi003.stdout: osd debug misdirected ops = true 2024-05-09T06:35:08.136 INFO:teuthology.orchestra.run.smithi003.stdout: bdev async discard = True 2024-05-09T06:35:08.136 INFO:teuthology.orchestra.run.smithi003.stdout: bdev enable discard = True 2024-05-09T06:35:08.136 INFO:teuthology.orchestra.run.smithi003.stdout: bluestore allocator = bitmap 2024-05-09T06:35:08.137 INFO:teuthology.orchestra.run.smithi003.stdout: bluestore block size = 96636764160 2024-05-09T06:35:08.137 INFO:teuthology.orchestra.run.smithi003.stdout: bluestore fsck on mount = True 2024-05-09T06:35:08.137 INFO:teuthology.orchestra.run.smithi003.stdout: debug bluefs = 1/20 2024-05-09T06:35:08.137 INFO:teuthology.orchestra.run.smithi003.stdout: debug bluestore = 1/20 2024-05-09T06:35:08.137 INFO:teuthology.orchestra.run.smithi003.stdout: debug ms = 1 2024-05-09T06:35:08.137 INFO:teuthology.orchestra.run.smithi003.stdout: debug osd = 20 2024-05-09T06:35:08.137 INFO:teuthology.orchestra.run.smithi003.stdout: debug rocksdb = 4/10 2024-05-09T06:35:08.137 INFO:teuthology.orchestra.run.smithi003.stdout: mon osd backfillfull_ratio = 0.85 2024-05-09T06:35:08.137 INFO:teuthology.orchestra.run.smithi003.stdout: mon osd full ratio = 0.9 2024-05-09T06:35:08.137 INFO:teuthology.orchestra.run.smithi003.stdout: mon osd nearfull ratio = 0.8 2024-05-09T06:35:08.137 INFO:teuthology.orchestra.run.smithi003.stdout: osd failsafe full ratio = 0.95 2024-05-09T06:35:08.137 INFO:teuthology.orchestra.run.smithi003.stdout: osd objectstore = bluestore 2024-05-09T06:35:08.137 INFO:teuthology.orchestra.run.smithi003.stdout: osd op complaint time = 180 2024-05-09T06:35:08.137 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-09T06:35:08.138 INFO:teuthology.orchestra.run.smithi003.stdout:[mgr] 2024-05-09T06:35:08.138 INFO:teuthology.orchestra.run.smithi003.stdout: debug ms = 1 2024-05-09T06:35:08.138 INFO:teuthology.orchestra.run.smithi003.stdout: debug mgr = 20 2024-05-09T06:35:08.138 INFO:teuthology.orchestra.run.smithi003.stdout: debug mon = 20 2024-05-09T06:35:08.138 INFO:teuthology.orchestra.run.smithi003.stdout: debug auth = 20 2024-05-09T06:35:08.138 INFO:teuthology.orchestra.run.smithi003.stdout: mon reweight min pgs per osd = 4 2024-05-09T06:35:08.138 INFO:teuthology.orchestra.run.smithi003.stdout: mon reweight min bytes per osd = 10 2024-05-09T06:35:08.138 INFO:teuthology.orchestra.run.smithi003.stdout: mgr/telemetry/nag = false 2024-05-09T06:35:08.138 INFO:teuthology.orchestra.run.smithi003.stdout: debug client = 20 2024-05-09T06:35:08.138 INFO:teuthology.orchestra.run.smithi003.stdout: debug finisher = 20 2024-05-09T06:35:08.138 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-09T06:35:08.138 INFO:teuthology.orchestra.run.smithi003.stdout:[mon] 2024-05-09T06:35:08.138 INFO:teuthology.orchestra.run.smithi003.stdout: debug ms = 1 2024-05-09T06:35:08.138 INFO:teuthology.orchestra.run.smithi003.stdout: debug mon = 20 2024-05-09T06:35:08.138 INFO:teuthology.orchestra.run.smithi003.stdout: debug paxos = 20 2024-05-09T06:35:08.138 INFO:teuthology.orchestra.run.smithi003.stdout: debug auth = 20 2024-05-09T06:35:08.139 INFO:teuthology.orchestra.run.smithi003.stdout: mon data avail warn = 5 2024-05-09T06:35:08.139 INFO:teuthology.orchestra.run.smithi003.stdout: mon mgr mkfs grace = 240 2024-05-09T06:35:08.139 INFO:teuthology.orchestra.run.smithi003.stdout: mon reweight min pgs per osd = 4 2024-05-09T06:35:08.139 INFO:teuthology.orchestra.run.smithi003.stdout: mon osd reporter subtree level = osd 2024-05-09T06:35:08.139 INFO:teuthology.orchestra.run.smithi003.stdout: mon osd prime pg temp = true 2024-05-09T06:35:08.139 INFO:teuthology.orchestra.run.smithi003.stdout: mon reweight min bytes per osd = 10 2024-05-09T06:35:08.139 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-09T06:35:08.139 INFO:teuthology.orchestra.run.smithi003.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-05-09T06:35:08.139 INFO:teuthology.orchestra.run.smithi003.stdout: auth mon ticket ttl = 660 # 11m 2024-05-09T06:35:08.139 INFO:teuthology.orchestra.run.smithi003.stdout: auth service ticket ttl = 240 # 4m 2024-05-09T06:35:08.139 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-09T06:35:08.139 INFO:teuthology.orchestra.run.smithi003.stdout: # don't complain about insecure global_id in the test suite 2024-05-09T06:35:08.139 INFO:teuthology.orchestra.run.smithi003.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-05-09T06:35:08.139 INFO:teuthology.orchestra.run.smithi003.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-05-09T06:35:08.139 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-09T06:35:08.140 INFO:teuthology.orchestra.run.smithi003.stdout: # 1m isn't quite enough 2024-05-09T06:35:08.140 INFO:teuthology.orchestra.run.smithi003.stdout: mon_down_mkfs_grace = 2m 2024-05-09T06:35:08.140 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-09T06:35:08.140 INFO:teuthology.orchestra.run.smithi003.stdout: mon_warn_on_filestore_osds = false 2024-05-09T06:35:08.140 INFO:teuthology.orchestra.run.smithi003.stdout: mon down mkfs grace = 300 2024-05-09T06:35:08.140 INFO:teuthology.orchestra.run.smithi003.stdout: mon op complaint time = 120 2024-05-09T06:35:08.140 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-09T06:35:08.140 INFO:teuthology.orchestra.run.smithi003.stdout:[client] 2024-05-09T06:35:08.140 INFO:teuthology.orchestra.run.smithi003.stdout: rgw cache enabled = true 2024-05-09T06:35:08.140 INFO:teuthology.orchestra.run.smithi003.stdout: rgw enable ops log = true 2024-05-09T06:35:08.140 INFO:teuthology.orchestra.run.smithi003.stdout: rgw enable usage log = true 2024-05-09T06:35:08.140 INFO:teuthology.orchestra.run.smithi003.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-05-09T06:35:08.140 INFO:teuthology.orchestra.run.smithi003.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-05-09T06:35:08.140 INFO:teuthology.orchestra.run.smithi003.stdout: client mount timeout = 600 2024-05-09T06:35:08.140 INFO:teuthology.orchestra.run.smithi003.stdout: debug client = 20 2024-05-09T06:35:08.140 INFO:teuthology.orchestra.run.smithi003.stdout: debug ms = 1 2024-05-09T06:35:08.141 INFO:teuthology.orchestra.run.smithi003.stdout: pid file = "" 2024-05-09T06:35:08.141 INFO:teuthology.orchestra.run.smithi003.stdout: rados mon op timeout = 900 2024-05-09T06:35:08.141 INFO:teuthology.orchestra.run.smithi003.stdout: rados osd op timeout = 900 2024-05-09T06:35:08.141 INFO:teuthology.orchestra.run.smithi003.stdout:[mon.a] 2024-05-09T06:35:08.141 INFO:teuthology.orchestra.run.smithi003.stdout:[mon.b] 2024-05-09T06:35:08.141 INFO:teuthology.orchestra.run.smithi003.stdout:[mon.c] 2024-05-09T06:35:08.141 INFO:teuthology.orchestra.run.smithi003.stdout:[mds.a] 2024-05-09T06:35:08.141 INFO:teuthology.orchestra.run.smithi003.stdout:[mds.c] 2024-05-09T06:35:08.141 INFO:teuthology.orchestra.run.smithi003.stdout:[mds.b] 2024-05-09T06:35:08.141 INFO:teuthology.orchestra.run.smithi003.stdout:[mds.d] 2024-05-09T06:35:08.141 INFO:teuthology.orchestra.run.smithi003.stdout:[mds] 2024-05-09T06:35:08.141 INFO:teuthology.orchestra.run.smithi003.stdout: debug mds = 20 2024-05-09T06:35:08.141 INFO:teuthology.orchestra.run.smithi003.stdout: debug mds balancer = 20 2024-05-09T06:35:08.141 INFO:teuthology.orchestra.run.smithi003.stdout: debug ms = 1 2024-05-09T06:35:08.141 INFO:teuthology.orchestra.run.smithi003.stdout: mds debug frag = True 2024-05-09T06:35:08.142 INFO:teuthology.orchestra.run.smithi003.stdout: mds debug scatterstat = True 2024-05-09T06:35:08.142 INFO:teuthology.orchestra.run.smithi003.stdout: mds op complaint time = 180 2024-05-09T06:35:08.142 INFO:teuthology.orchestra.run.smithi003.stdout: mds verify scatter = True 2024-05-09T06:35:08.142 INFO:teuthology.orchestra.run.smithi003.stdout: osd op complaint time = 180 2024-05-09T06:35:08.142 INFO:teuthology.orchestra.run.smithi003.stdout: rados mon op timeout = 900 2024-05-09T06:35:08.142 INFO:teuthology.orchestra.run.smithi003.stdout: rados osd op timeout = 900 2024-05-09T06:35:08.142 INFO:teuthology.orchestra.run.smithi190.stdout:[global] 2024-05-09T06:35:08.143 INFO:teuthology.orchestra.run.smithi190.stdout: chdir = "" 2024-05-09T06:35:08.143 INFO:teuthology.orchestra.run.smithi190.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-05-09T06:35:08.143 INFO:teuthology.orchestra.run.smithi190.stdout: auth supported = cephx 2024-05-09T06:35:08.143 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-05-09T06:35:08.143 INFO:teuthology.orchestra.run.smithi190.stdout: filestore xattr use omap = true 2024-05-09T06:35:08.143 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-05-09T06:35:08.143 INFO:teuthology.orchestra.run.smithi190.stdout: mon clock drift allowed = 1.000 2024-05-09T06:35:08.143 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-05-09T06:35:08.143 INFO:teuthology.orchestra.run.smithi190.stdout: osd crush chooseleaf type = 0 2024-05-09T06:35:08.143 INFO:teuthology.orchestra.run.smithi190.stdout: auth debug = true 2024-05-09T06:35:08.143 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-05-09T06:35:08.143 INFO:teuthology.orchestra.run.smithi190.stdout: ms die on old message = true 2024-05-09T06:35:08.143 INFO:teuthology.orchestra.run.smithi190.stdout: ms die on bug = true 2024-05-09T06:35:08.143 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-05-09T06:35:08.144 INFO:teuthology.orchestra.run.smithi190.stdout: mon max pg per osd = 10000 # >= luminous 2024-05-09T06:35:08.144 INFO:teuthology.orchestra.run.smithi190.stdout: mon pg warn max object skew = 0 2024-05-09T06:35:08.144 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-05-09T06:35:08.144 INFO:teuthology.orchestra.run.smithi190.stdout: # disable pg_autoscaler by default for new pools 2024-05-09T06:35:08.144 INFO:teuthology.orchestra.run.smithi190.stdout: osd_pool_default_pg_autoscale_mode = off 2024-05-09T06:35:08.144 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-05-09T06:35:08.144 INFO:teuthology.orchestra.run.smithi190.stdout: osd pool default size = 2 2024-05-09T06:35:08.144 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-05-09T06:35:08.144 INFO:teuthology.orchestra.run.smithi190.stdout: mon osd allow primary affinity = true 2024-05-09T06:35:08.144 INFO:teuthology.orchestra.run.smithi190.stdout: mon osd allow pg remap = true 2024-05-09T06:35:08.144 INFO:teuthology.orchestra.run.smithi190.stdout: mon warn on legacy crush tunables = false 2024-05-09T06:35:08.144 INFO:teuthology.orchestra.run.smithi190.stdout: mon warn on crush straw calc version zero = false 2024-05-09T06:35:08.144 INFO:teuthology.orchestra.run.smithi190.stdout: mon warn on no sortbitwise = false 2024-05-09T06:35:08.144 INFO:teuthology.orchestra.run.smithi190.stdout: mon warn on osd down out interval zero = false 2024-05-09T06:35:08.144 INFO:teuthology.orchestra.run.smithi190.stdout: mon warn on too few osds = false 2024-05-09T06:35:08.145 INFO:teuthology.orchestra.run.smithi190.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-05-09T06:35:08.145 INFO:teuthology.orchestra.run.smithi190.stdout: mon_warn_on_pool_no_redundancy = false 2024-05-09T06:35:08.145 INFO:teuthology.orchestra.run.smithi190.stdout: mon_allow_pool_size_one = true 2024-05-09T06:35:08.145 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-05-09T06:35:08.145 INFO:teuthology.orchestra.run.smithi190.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-05-09T06:35:08.145 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-05-09T06:35:08.145 INFO:teuthology.orchestra.run.smithi190.stdout: osd default data pool replay window = 5 2024-05-09T06:35:08.145 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-05-09T06:35:08.145 INFO:teuthology.orchestra.run.smithi190.stdout: mon allow pool delete = true 2024-05-09T06:35:08.145 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-05-09T06:35:08.145 INFO:teuthology.orchestra.run.smithi190.stdout: mon cluster log file level = debug 2024-05-09T06:35:08.145 INFO:teuthology.orchestra.run.smithi190.stdout: debug asserts on shutdown = true 2024-05-09T06:35:08.145 INFO:teuthology.orchestra.run.smithi190.stdout: mon health detail to clog = false 2024-05-09T06:35:08.145 INFO:teuthology.orchestra.run.smithi190.stdout: mon host = "172.21.15.3,172.21.15.190,[v2:172.21.15.190:3301,v1:172.21.15.190:6790]" 2024-05-09T06:35:08.145 INFO:teuthology.orchestra.run.smithi190.stdout: ms die on skipped message = False 2024-05-09T06:35:08.145 INFO:teuthology.orchestra.run.smithi190.stdout: fsid = 16ed737d-2665-431a-8ac6-ef242be3975a 2024-05-09T06:35:08.146 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-05-09T06:35:08.146 INFO:teuthology.orchestra.run.smithi190.stdout:[osd] 2024-05-09T06:35:08.146 INFO:teuthology.orchestra.run.smithi190.stdout: osd journal size = 100 2024-05-09T06:35:08.146 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-05-09T06:35:08.146 INFO:teuthology.orchestra.run.smithi190.stdout: osd scrub load threshold = 5.0 2024-05-09T06:35:08.146 INFO:teuthology.orchestra.run.smithi190.stdout: osd scrub max interval = 600 2024-05-09T06:35:08.146 INFO:teuthology.orchestra.run.smithi190.stdout: osd mclock profile = high_recovery_ops 2024-05-09T06:35:08.146 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-05-09T06:35:08.146 INFO:teuthology.orchestra.run.smithi190.stdout: osd recover clone overlap = true 2024-05-09T06:35:08.146 INFO:teuthology.orchestra.run.smithi190.stdout: osd recovery max chunk = 1048576 2024-05-09T06:35:08.146 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-05-09T06:35:08.146 INFO:teuthology.orchestra.run.smithi190.stdout: osd debug shutdown = true 2024-05-09T06:35:08.146 INFO:teuthology.orchestra.run.smithi190.stdout: osd debug op order = true 2024-05-09T06:35:08.146 INFO:teuthology.orchestra.run.smithi190.stdout: osd debug verify stray on activate = true 2024-05-09T06:35:08.147 INFO:teuthology.orchestra.run.smithi190.stdout: osd debug trim objects = true 2024-05-09T06:35:08.147 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-05-09T06:35:08.147 INFO:teuthology.orchestra.run.smithi190.stdout: osd open classes on start = true 2024-05-09T06:35:08.147 INFO:teuthology.orchestra.run.smithi190.stdout: osd debug pg log writeout = true 2024-05-09T06:35:08.147 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-05-09T06:35:08.147 INFO:teuthology.orchestra.run.smithi190.stdout: osd deep scrub update digest min age = 30 2024-05-09T06:35:08.147 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-05-09T06:35:08.147 INFO:teuthology.orchestra.run.smithi190.stdout: osd map max advance = 10 2024-05-09T06:35:08.147 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-05-09T06:35:08.147 INFO:teuthology.orchestra.run.smithi190.stdout: journal zero on create = true 2024-05-09T06:35:08.147 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-05-09T06:35:08.147 INFO:teuthology.orchestra.run.smithi190.stdout: filestore ondisk finisher threads = 3 2024-05-09T06:35:08.147 INFO:teuthology.orchestra.run.smithi190.stdout: filestore apply finisher threads = 3 2024-05-09T06:35:08.147 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-05-09T06:35:08.147 INFO:teuthology.orchestra.run.smithi190.stdout: bdev debug aio = true 2024-05-09T06:35:08.148 INFO:teuthology.orchestra.run.smithi190.stdout: osd debug misdirected ops = true 2024-05-09T06:35:08.148 INFO:teuthology.orchestra.run.smithi190.stdout: bdev async discard = True 2024-05-09T06:35:08.148 INFO:teuthology.orchestra.run.smithi190.stdout: bdev enable discard = True 2024-05-09T06:35:08.148 INFO:teuthology.orchestra.run.smithi190.stdout: bluestore allocator = bitmap 2024-05-09T06:35:08.148 INFO:teuthology.orchestra.run.smithi190.stdout: bluestore block size = 96636764160 2024-05-09T06:35:08.148 INFO:teuthology.orchestra.run.smithi190.stdout: bluestore fsck on mount = True 2024-05-09T06:35:08.148 INFO:teuthology.orchestra.run.smithi190.stdout: debug bluefs = 1/20 2024-05-09T06:35:08.148 INFO:teuthology.orchestra.run.smithi190.stdout: debug bluestore = 1/20 2024-05-09T06:35:08.148 INFO:teuthology.orchestra.run.smithi190.stdout: debug ms = 1 2024-05-09T06:35:08.148 INFO:teuthology.orchestra.run.smithi190.stdout: debug osd = 20 2024-05-09T06:35:08.148 INFO:teuthology.orchestra.run.smithi190.stdout: debug rocksdb = 4/10 2024-05-09T06:35:08.148 INFO:teuthology.orchestra.run.smithi190.stdout: mon osd backfillfull_ratio = 0.85 2024-05-09T06:35:08.148 INFO:teuthology.orchestra.run.smithi190.stdout: mon osd full ratio = 0.9 2024-05-09T06:35:08.148 INFO:teuthology.orchestra.run.smithi190.stdout: mon osd nearfull ratio = 0.8 2024-05-09T06:35:08.149 INFO:teuthology.orchestra.run.smithi190.stdout: osd failsafe full ratio = 0.95 2024-05-09T06:35:08.149 INFO:teuthology.orchestra.run.smithi190.stdout: osd objectstore = bluestore 2024-05-09T06:35:08.149 INFO:teuthology.orchestra.run.smithi190.stdout: osd op complaint time = 180 2024-05-09T06:35:08.149 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-05-09T06:35:08.149 INFO:teuthology.orchestra.run.smithi190.stdout:[mgr] 2024-05-09T06:35:08.149 INFO:teuthology.orchestra.run.smithi190.stdout: debug ms = 1 2024-05-09T06:35:08.149 INFO:teuthology.orchestra.run.smithi190.stdout: debug mgr = 20 2024-05-09T06:35:08.149 INFO:teuthology.orchestra.run.smithi190.stdout: debug mon = 20 2024-05-09T06:35:08.149 INFO:teuthology.orchestra.run.smithi190.stdout: debug auth = 20 2024-05-09T06:35:08.149 INFO:teuthology.orchestra.run.smithi190.stdout: mon reweight min pgs per osd = 4 2024-05-09T06:35:08.149 INFO:teuthology.orchestra.run.smithi190.stdout: mon reweight min bytes per osd = 10 2024-05-09T06:35:08.149 INFO:teuthology.orchestra.run.smithi190.stdout: mgr/telemetry/nag = false 2024-05-09T06:35:08.149 INFO:teuthology.orchestra.run.smithi190.stdout: debug client = 20 2024-05-09T06:35:08.149 INFO:teuthology.orchestra.run.smithi190.stdout: debug finisher = 20 2024-05-09T06:35:08.150 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-05-09T06:35:08.150 INFO:teuthology.orchestra.run.smithi190.stdout:[mon] 2024-05-09T06:35:08.150 INFO:teuthology.orchestra.run.smithi190.stdout: debug ms = 1 2024-05-09T06:35:08.150 INFO:teuthology.orchestra.run.smithi190.stdout: debug mon = 20 2024-05-09T06:35:08.150 INFO:teuthology.orchestra.run.smithi190.stdout: debug paxos = 20 2024-05-09T06:35:08.150 INFO:teuthology.orchestra.run.smithi190.stdout: debug auth = 20 2024-05-09T06:35:08.150 INFO:teuthology.orchestra.run.smithi190.stdout: mon data avail warn = 5 2024-05-09T06:35:08.150 INFO:teuthology.orchestra.run.smithi190.stdout: mon mgr mkfs grace = 240 2024-05-09T06:35:08.150 INFO:teuthology.orchestra.run.smithi190.stdout: mon reweight min pgs per osd = 4 2024-05-09T06:35:08.150 INFO:teuthology.orchestra.run.smithi190.stdout: mon osd reporter subtree level = osd 2024-05-09T06:35:08.150 INFO:teuthology.orchestra.run.smithi190.stdout: mon osd prime pg temp = true 2024-05-09T06:35:08.150 INFO:teuthology.orchestra.run.smithi190.stdout: mon reweight min bytes per osd = 10 2024-05-09T06:35:08.150 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-05-09T06:35:08.150 INFO:teuthology.orchestra.run.smithi190.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-05-09T06:35:08.151 INFO:teuthology.orchestra.run.smithi190.stdout: auth mon ticket ttl = 660 # 11m 2024-05-09T06:35:08.151 INFO:teuthology.orchestra.run.smithi190.stdout: auth service ticket ttl = 240 # 4m 2024-05-09T06:35:08.151 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-05-09T06:35:08.151 INFO:teuthology.orchestra.run.smithi190.stdout: # don't complain about insecure global_id in the test suite 2024-05-09T06:35:08.151 INFO:teuthology.orchestra.run.smithi190.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-05-09T06:35:08.151 INFO:teuthology.orchestra.run.smithi190.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-05-09T06:35:08.151 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-05-09T06:35:08.151 INFO:teuthology.orchestra.run.smithi190.stdout: # 1m isn't quite enough 2024-05-09T06:35:08.151 INFO:teuthology.orchestra.run.smithi190.stdout: mon_down_mkfs_grace = 2m 2024-05-09T06:35:08.151 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-05-09T06:35:08.151 INFO:teuthology.orchestra.run.smithi190.stdout: mon_warn_on_filestore_osds = false 2024-05-09T06:35:08.151 INFO:teuthology.orchestra.run.smithi190.stdout: mon down mkfs grace = 300 2024-05-09T06:35:08.151 INFO:teuthology.orchestra.run.smithi190.stdout: mon op complaint time = 120 2024-05-09T06:35:08.151 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-05-09T06:35:08.151 INFO:teuthology.orchestra.run.smithi190.stdout:[client] 2024-05-09T06:35:08.152 INFO:teuthology.orchestra.run.smithi190.stdout: rgw cache enabled = true 2024-05-09T06:35:08.152 INFO:teuthology.orchestra.run.smithi190.stdout: rgw enable ops log = true 2024-05-09T06:35:08.152 INFO:teuthology.orchestra.run.smithi190.stdout: rgw enable usage log = true 2024-05-09T06:35:08.152 INFO:teuthology.orchestra.run.smithi190.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-05-09T06:35:08.152 INFO:teuthology.orchestra.run.smithi190.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-05-09T06:35:08.152 INFO:teuthology.orchestra.run.smithi190.stdout: client mount timeout = 600 2024-05-09T06:35:08.152 INFO:teuthology.orchestra.run.smithi190.stdout: debug client = 20 2024-05-09T06:35:08.152 INFO:teuthology.orchestra.run.smithi190.stdout: debug ms = 1 2024-05-09T06:35:08.152 INFO:teuthology.orchestra.run.smithi190.stdout: pid file = "" 2024-05-09T06:35:08.152 INFO:teuthology.orchestra.run.smithi190.stdout: rados mon op timeout = 900 2024-05-09T06:35:08.152 INFO:teuthology.orchestra.run.smithi190.stdout: rados osd op timeout = 900 2024-05-09T06:35:08.152 INFO:teuthology.orchestra.run.smithi190.stdout:[mon.a] 2024-05-09T06:35:08.152 INFO:teuthology.orchestra.run.smithi190.stdout:[mon.b] 2024-05-09T06:35:08.152 INFO:teuthology.orchestra.run.smithi190.stdout:[mon.c] 2024-05-09T06:35:08.153 INFO:teuthology.orchestra.run.smithi190.stdout:[mds.a] 2024-05-09T06:35:08.153 INFO:teuthology.orchestra.run.smithi190.stdout:[mds.c] 2024-05-09T06:35:08.153 INFO:teuthology.orchestra.run.smithi190.stdout:[mds.b] 2024-05-09T06:35:08.153 INFO:teuthology.orchestra.run.smithi190.stdout:[mds.d] 2024-05-09T06:35:08.153 INFO:teuthology.orchestra.run.smithi190.stdout:[mds] 2024-05-09T06:35:08.153 INFO:teuthology.orchestra.run.smithi190.stdout: debug mds = 20 2024-05-09T06:35:08.153 INFO:teuthology.orchestra.run.smithi190.stdout: debug mds balancer = 20 2024-05-09T06:35:08.153 INFO:teuthology.orchestra.run.smithi190.stdout: debug ms = 1 2024-05-09T06:35:08.153 INFO:teuthology.orchestra.run.smithi190.stdout: mds debug frag = True 2024-05-09T06:35:08.153 INFO:teuthology.orchestra.run.smithi190.stdout: mds debug scatterstat = True 2024-05-09T06:35:08.153 INFO:teuthology.orchestra.run.smithi190.stdout: mds op complaint time = 180 2024-05-09T06:35:08.153 INFO:teuthology.orchestra.run.smithi190.stdout: mds verify scatter = True 2024-05-09T06:35:08.153 INFO:teuthology.orchestra.run.smithi190.stdout: osd op complaint time = 180 2024-05-09T06:35:08.153 INFO:teuthology.orchestra.run.smithi190.stdout: rados mon op timeout = 900 2024-05-09T06:35:08.153 INFO:teuthology.orchestra.run.smithi190.stdout: rados osd op timeout = 900 2024-05-09T06:35:08.155 INFO:tasks.ceph:Creating admin key on mon.a... 2024-05-09T06:35:08.155 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-05-09T06:35:08.252 INFO:tasks.ceph:Copying monmap to all nodes... 2024-05-09T06:35:08.253 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-05-09T06:35:08.253 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-05-09T06:35:08.269 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-05-09T06:35:08.269 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-05-09T06:35:08.325 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi003.front.sepia.ceph.com 2024-05-09T06:35:08.326 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-05-09T06:35:08.326 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd of=/etc/ceph/ceph.keyring 2024-05-09T06:35:08.326 DEBUG:teuthology.orchestra.run.smithi003:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-05-09T06:35:08.403 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-05-09T06:35:08.403 DEBUG:teuthology.orchestra.run.smithi003:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-05-09T06:35:08.459 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi190.front.sepia.ceph.com 2024-05-09T06:35:08.459 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-05-09T06:35:08.460 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd of=/etc/ceph/ceph.keyring 2024-05-09T06:35:08.460 DEBUG:teuthology.orchestra.run.smithi190:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-05-09T06:35:08.495 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-05-09T06:35:08.495 DEBUG:teuthology.orchestra.run.smithi190:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-05-09T06:35:08.551 INFO:tasks.ceph:Setting up mon nodes... 2024-05-09T06:35:08.552 INFO:tasks.ceph:Setting up mgr nodes... 2024-05-09T06:35:08.552 DEBUG:teuthology.orchestra.run.smithi003:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-05-09T06:35:08.605 INFO:teuthology.orchestra.run.smithi003.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-05-09T06:35:08.607 DEBUG:teuthology.orchestra.run.smithi190:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-05-09T06:35:08.680 INFO:teuthology.orchestra.run.smithi190.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-05-09T06:35:08.683 INFO:tasks.ceph:Setting up mds nodes... 2024-05-09T06:35:08.683 DEBUG:teuthology.orchestra.run.smithi003:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-05-09T06:35:08.736 INFO:teuthology.orchestra.run.smithi003.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-05-09T06:35:08.739 DEBUG:teuthology.orchestra.run.smithi003:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-05-09T06:35:08.764 DEBUG:teuthology.orchestra.run.smithi003:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2024-05-09T06:35:08.857 INFO:teuthology.orchestra.run.smithi003.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2024-05-09T06:35:08.860 DEBUG:teuthology.orchestra.run.smithi003:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2024-05-09T06:35:08.886 DEBUG:teuthology.orchestra.run.smithi190:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-05-09T06:35:08.938 INFO:teuthology.orchestra.run.smithi190.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-05-09T06:35:08.940 DEBUG:teuthology.orchestra.run.smithi190:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-05-09T06:35:08.965 DEBUG:teuthology.orchestra.run.smithi190:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2024-05-09T06:35:09.056 INFO:teuthology.orchestra.run.smithi190.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2024-05-09T06:35:09.059 DEBUG:teuthology.orchestra.run.smithi190:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2024-05-09T06:35:09.083 INFO:tasks.ceph_client:Setting up client nodes... 2024-05-09T06:35:09.084 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-05-09T06:35:09.126 INFO:teuthology.orchestra.run.smithi003.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-05-09T06:35:09.139 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2024-05-09T06:35:09.222 INFO:teuthology.orchestra.run.smithi003.stdout:creating /etc/ceph/ceph.client.2.keyring 2024-05-09T06:35:09.235 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-05-09T06:35:09.275 INFO:teuthology.orchestra.run.smithi190.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-05-09T06:35:09.288 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2024-05-09T06:35:09.370 INFO:teuthology.orchestra.run.smithi190.stdout:creating /etc/ceph/ceph.client.3.keyring 2024-05-09T06:35:09.383 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-05-09T06:35:09.383 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi003.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi190.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-05-09T06:35:09.384 DEBUG:teuthology.orchestra.run.smithi003:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-05-09T06:35:09.409 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-05-09T06:35:09.409 INFO:tasks.ceph:role: osd.0 2024-05-09T06:35:09.409 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi003.front.sepia.ceph.com 2024-05-09T06:35:09.410 DEBUG:teuthology.orchestra.run.smithi003:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-05-09T06:35:09.483 INFO:teuthology.orchestra.run.smithi003.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-05-09T06:35:09.483 INFO:teuthology.orchestra.run.smithi003.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-09T06:35:09.483 INFO:teuthology.orchestra.run.smithi003.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-09T06:35:09.483 INFO:teuthology.orchestra.run.smithi003.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-09T06:35:09.483 INFO:teuthology.orchestra.run.smithi003.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-09T06:35:09.483 INFO:teuthology.orchestra.run.smithi003.stdout: = sunit=0 swidth=0 blks 2024-05-09T06:35:09.484 INFO:teuthology.orchestra.run.smithi003.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-09T06:35:09.484 INFO:teuthology.orchestra.run.smithi003.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-09T06:35:09.484 INFO:teuthology.orchestra.run.smithi003.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-09T06:35:09.484 INFO:teuthology.orchestra.run.smithi003.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-09T06:35:09.994 INFO:teuthology.orchestra.run.smithi003.stdout:Discarding blocks...Done. 2024-05-09T06:35:09.994 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi003.front.sepia.ceph.com -o noatime 2024-05-09T06:35:09.994 DEBUG:teuthology.orchestra.run.smithi003:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-05-09T06:35:10.133 DEBUG:teuthology.orchestra.run.smithi003:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-05-09T06:35:10.166 DEBUG:teuthology.orchestra.run.smithi003:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-05-09T06:35:10.233 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-05-09T06:35:10.233 INFO:tasks.ceph:role: osd.1 2024-05-09T06:35:10.233 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi003.front.sepia.ceph.com 2024-05-09T06:35:10.234 DEBUG:teuthology.orchestra.run.smithi003:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-05-09T06:35:10.308 INFO:teuthology.orchestra.run.smithi003.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-05-09T06:35:10.308 INFO:teuthology.orchestra.run.smithi003.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-09T06:35:10.308 INFO:teuthology.orchestra.run.smithi003.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-09T06:35:10.308 INFO:teuthology.orchestra.run.smithi003.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-09T06:35:10.308 INFO:teuthology.orchestra.run.smithi003.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-09T06:35:10.308 INFO:teuthology.orchestra.run.smithi003.stdout: = sunit=0 swidth=0 blks 2024-05-09T06:35:10.308 INFO:teuthology.orchestra.run.smithi003.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-09T06:35:10.308 INFO:teuthology.orchestra.run.smithi003.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-09T06:35:10.309 INFO:teuthology.orchestra.run.smithi003.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-09T06:35:10.309 INFO:teuthology.orchestra.run.smithi003.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-09T06:35:10.755 INFO:teuthology.orchestra.run.smithi003.stdout:Discarding blocks...Done. 2024-05-09T06:35:10.757 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi003.front.sepia.ceph.com -o noatime 2024-05-09T06:35:10.757 DEBUG:teuthology.orchestra.run.smithi003:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-05-09T06:35:10.853 DEBUG:teuthology.orchestra.run.smithi003:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-05-09T06:35:10.884 DEBUG:teuthology.orchestra.run.smithi003:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-05-09T06:35:10.950 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-05-09T06:35:10.950 INFO:tasks.ceph:role: osd.2 2024-05-09T06:35:10.950 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi003.front.sepia.ceph.com 2024-05-09T06:35:10.950 DEBUG:teuthology.orchestra.run.smithi003:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-05-09T06:35:11.023 INFO:teuthology.orchestra.run.smithi003.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-05-09T06:35:11.023 INFO:teuthology.orchestra.run.smithi003.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-09T06:35:11.023 INFO:teuthology.orchestra.run.smithi003.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-09T06:35:11.023 INFO:teuthology.orchestra.run.smithi003.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-09T06:35:11.023 INFO:teuthology.orchestra.run.smithi003.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-09T06:35:11.023 INFO:teuthology.orchestra.run.smithi003.stdout: = sunit=0 swidth=0 blks 2024-05-09T06:35:11.023 INFO:teuthology.orchestra.run.smithi003.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-09T06:35:11.023 INFO:teuthology.orchestra.run.smithi003.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-09T06:35:11.024 INFO:teuthology.orchestra.run.smithi003.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-09T06:35:11.024 INFO:teuthology.orchestra.run.smithi003.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-09T06:35:11.505 INFO:teuthology.orchestra.run.smithi003.stdout:Discarding blocks...Done. 2024-05-09T06:35:11.507 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi003.front.sepia.ceph.com -o noatime 2024-05-09T06:35:11.507 DEBUG:teuthology.orchestra.run.smithi003:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-05-09T06:35:11.658 DEBUG:teuthology.orchestra.run.smithi003:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-05-09T06:35:11.689 DEBUG:teuthology.orchestra.run.smithi003:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-05-09T06:35:11.755 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-05-09T06:35:11.755 INFO:tasks.ceph:role: osd.3 2024-05-09T06:35:11.755 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi003.front.sepia.ceph.com 2024-05-09T06:35:11.755 DEBUG:teuthology.orchestra.run.smithi003:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-05-09T06:35:11.829 INFO:teuthology.orchestra.run.smithi003.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-05-09T06:35:11.829 INFO:teuthology.orchestra.run.smithi003.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-09T06:35:11.830 INFO:teuthology.orchestra.run.smithi003.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-09T06:35:11.830 INFO:teuthology.orchestra.run.smithi003.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-09T06:35:11.830 INFO:teuthology.orchestra.run.smithi003.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-09T06:35:11.830 INFO:teuthology.orchestra.run.smithi003.stdout: = sunit=0 swidth=0 blks 2024-05-09T06:35:11.830 INFO:teuthology.orchestra.run.smithi003.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-09T06:35:11.830 INFO:teuthology.orchestra.run.smithi003.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-09T06:35:11.830 INFO:teuthology.orchestra.run.smithi003.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-09T06:35:11.830 INFO:teuthology.orchestra.run.smithi003.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-09T06:35:12.344 INFO:teuthology.orchestra.run.smithi003.stdout:Discarding blocks...Done. 2024-05-09T06:35:12.346 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi003.front.sepia.ceph.com -o noatime 2024-05-09T06:35:12.347 DEBUG:teuthology.orchestra.run.smithi003:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-05-09T06:35:12.488 DEBUG:teuthology.orchestra.run.smithi003:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-05-09T06:35:12.524 DEBUG:teuthology.orchestra.run.smithi003:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-09T06:35:12.625 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:35:12.644+0000 7f27436e3640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-05-09T06:35:12.625 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:35:12.644+0000 7f27436e3640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-05-09T06:35:12.626 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:35:12.644+0000 7f27436e3640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-05-09T06:35:12.626 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:35:12.644+0000 7f27436e3640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-05-09T06:35:12.626 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:35:12.644+0000 7f27436e3640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-05-09T06:35:12.626 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:35:12.645+0000 7f27436e3640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-05-09T06:35:15.497 DEBUG:teuthology.orchestra.run.smithi003:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-05-09T06:35:15.528 DEBUG:teuthology.orchestra.run.smithi003:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-09T06:35:15.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:35:15.640+0000 7f754b63f640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-05-09T06:35:15.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:35:15.640+0000 7f754b63f640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-05-09T06:35:15.621 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:35:15.640+0000 7f754b63f640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-05-09T06:35:15.621 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:35:15.640+0000 7f754b63f640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-05-09T06:35:15.621 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:35:15.640+0000 7f754b63f640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-05-09T06:35:15.621 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:35:15.640+0000 7f754b63f640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-05-09T06:35:18.482 DEBUG:teuthology.orchestra.run.smithi003:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-05-09T06:35:18.513 DEBUG:teuthology.orchestra.run.smithi003:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-09T06:35:18.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:35:18.625+0000 7f7bddc1e640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-05-09T06:35:18.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:35:18.626+0000 7f7bddc1e640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-05-09T06:35:18.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:35:18.626+0000 7f7bddc1e640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-05-09T06:35:18.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:35:18.626+0000 7f7bddc1e640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-05-09T06:35:18.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:35:18.626+0000 7f7bddc1e640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-05-09T06:35:18.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:35:18.626+0000 7f7bddc1e640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-05-09T06:35:21.466 DEBUG:teuthology.orchestra.run.smithi003:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-05-09T06:35:21.499 DEBUG:teuthology.orchestra.run.smithi003:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-09T06:35:21.593 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:35:21.612+0000 7faf97dfc640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-05-09T06:35:21.593 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:35:21.613+0000 7faf97dfc640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-05-09T06:35:21.593 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:35:21.613+0000 7faf97dfc640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-05-09T06:35:21.593 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:35:21.613+0000 7faf97dfc640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-05-09T06:35:21.593 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:35:21.613+0000 7faf97dfc640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-05-09T06:35:21.593 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:35:21.613+0000 7faf97dfc640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-05-09T06:35:24.456 DEBUG:teuthology.orchestra.run.smithi003:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-05-09T06:35:24.487 DEBUG:teuthology.orchestra.run.smithi190:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-05-09T06:35:24.518 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-05-09T06:35:24.518 INFO:tasks.ceph:role: osd.4 2024-05-09T06:35:24.518 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi190.front.sepia.ceph.com 2024-05-09T06:35:24.518 DEBUG:teuthology.orchestra.run.smithi190:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-05-09T06:35:24.594 INFO:teuthology.orchestra.run.smithi190.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-05-09T06:35:24.594 INFO:teuthology.orchestra.run.smithi190.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-09T06:35:24.594 INFO:teuthology.orchestra.run.smithi190.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-09T06:35:24.594 INFO:teuthology.orchestra.run.smithi190.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-09T06:35:24.594 INFO:teuthology.orchestra.run.smithi190.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-09T06:35:24.594 INFO:teuthology.orchestra.run.smithi190.stdout: = sunit=0 swidth=0 blks 2024-05-09T06:35:24.594 INFO:teuthology.orchestra.run.smithi190.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-09T06:35:24.595 INFO:teuthology.orchestra.run.smithi190.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-09T06:35:24.595 INFO:teuthology.orchestra.run.smithi190.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-09T06:35:24.595 INFO:teuthology.orchestra.run.smithi190.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-09T06:35:25.115 INFO:teuthology.orchestra.run.smithi190.stdout:Discarding blocks...Done. 2024-05-09T06:35:25.117 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi190.front.sepia.ceph.com -o noatime 2024-05-09T06:35:25.117 DEBUG:teuthology.orchestra.run.smithi190:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-05-09T06:35:25.242 DEBUG:teuthology.orchestra.run.smithi190:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-05-09T06:35:25.275 DEBUG:teuthology.orchestra.run.smithi190:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-05-09T06:35:25.340 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-05-09T06:35:25.340 INFO:tasks.ceph:role: osd.5 2024-05-09T06:35:25.340 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi190.front.sepia.ceph.com 2024-05-09T06:35:25.341 DEBUG:teuthology.orchestra.run.smithi190:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-05-09T06:35:25.413 INFO:teuthology.orchestra.run.smithi190.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-05-09T06:35:25.413 INFO:teuthology.orchestra.run.smithi190.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-09T06:35:25.413 INFO:teuthology.orchestra.run.smithi190.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-09T06:35:25.413 INFO:teuthology.orchestra.run.smithi190.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-09T06:35:25.413 INFO:teuthology.orchestra.run.smithi190.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-09T06:35:25.413 INFO:teuthology.orchestra.run.smithi190.stdout: = sunit=0 swidth=0 blks 2024-05-09T06:35:25.413 INFO:teuthology.orchestra.run.smithi190.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-09T06:35:25.413 INFO:teuthology.orchestra.run.smithi190.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-09T06:35:25.413 INFO:teuthology.orchestra.run.smithi190.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-09T06:35:25.414 INFO:teuthology.orchestra.run.smithi190.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-09T06:35:25.867 INFO:teuthology.orchestra.run.smithi190.stdout:Discarding blocks...Done. 2024-05-09T06:35:25.869 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi190.front.sepia.ceph.com -o noatime 2024-05-09T06:35:25.869 DEBUG:teuthology.orchestra.run.smithi190:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-05-09T06:35:25.955 DEBUG:teuthology.orchestra.run.smithi190:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-05-09T06:35:25.983 DEBUG:teuthology.orchestra.run.smithi190:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-05-09T06:35:26.048 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-05-09T06:35:26.048 INFO:tasks.ceph:role: osd.6 2024-05-09T06:35:26.048 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi190.front.sepia.ceph.com 2024-05-09T06:35:26.049 DEBUG:teuthology.orchestra.run.smithi190:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-05-09T06:35:26.121 INFO:teuthology.orchestra.run.smithi190.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-05-09T06:35:26.121 INFO:teuthology.orchestra.run.smithi190.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-09T06:35:26.121 INFO:teuthology.orchestra.run.smithi190.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-09T06:35:26.121 INFO:teuthology.orchestra.run.smithi190.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-09T06:35:26.121 INFO:teuthology.orchestra.run.smithi190.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-09T06:35:26.121 INFO:teuthology.orchestra.run.smithi190.stdout: = sunit=0 swidth=0 blks 2024-05-09T06:35:26.121 INFO:teuthology.orchestra.run.smithi190.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-09T06:35:26.121 INFO:teuthology.orchestra.run.smithi190.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-09T06:35:26.121 INFO:teuthology.orchestra.run.smithi190.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-09T06:35:26.121 INFO:teuthology.orchestra.run.smithi190.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-09T06:35:26.621 INFO:teuthology.orchestra.run.smithi190.stdout:Discarding blocks...Done. 2024-05-09T06:35:26.623 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi190.front.sepia.ceph.com -o noatime 2024-05-09T06:35:26.623 DEBUG:teuthology.orchestra.run.smithi190:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-05-09T06:35:26.754 DEBUG:teuthology.orchestra.run.smithi190:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-05-09T06:35:26.789 DEBUG:teuthology.orchestra.run.smithi190:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-05-09T06:35:26.858 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-05-09T06:35:26.858 INFO:tasks.ceph:role: osd.7 2024-05-09T06:35:26.858 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi190.front.sepia.ceph.com 2024-05-09T06:35:26.859 DEBUG:teuthology.orchestra.run.smithi190:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-05-09T06:35:26.933 INFO:teuthology.orchestra.run.smithi190.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-05-09T06:35:26.933 INFO:teuthology.orchestra.run.smithi190.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-09T06:35:26.933 INFO:teuthology.orchestra.run.smithi190.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-09T06:35:26.933 INFO:teuthology.orchestra.run.smithi190.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-09T06:35:26.934 INFO:teuthology.orchestra.run.smithi190.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-09T06:35:26.934 INFO:teuthology.orchestra.run.smithi190.stdout: = sunit=0 swidth=0 blks 2024-05-09T06:35:26.934 INFO:teuthology.orchestra.run.smithi190.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-09T06:35:26.934 INFO:teuthology.orchestra.run.smithi190.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-09T06:35:26.934 INFO:teuthology.orchestra.run.smithi190.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-09T06:35:26.934 INFO:teuthology.orchestra.run.smithi190.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-09T06:35:27.431 INFO:teuthology.orchestra.run.smithi190.stdout:Discarding blocks...Done. 2024-05-09T06:35:27.433 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi190.front.sepia.ceph.com -o noatime 2024-05-09T06:35:27.434 DEBUG:teuthology.orchestra.run.smithi190:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-05-09T06:35:27.569 DEBUG:teuthology.orchestra.run.smithi190:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-05-09T06:35:27.604 DEBUG:teuthology.orchestra.run.smithi190:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-09T06:35:27.703 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:27.723+0000 7f22167ad640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-05-09T06:35:27.704 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:27.724+0000 7f22167ad640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-05-09T06:35:27.704 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:27.724+0000 7f22167ad640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-05-09T06:35:27.704 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:27.724+0000 7f22167ad640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-05-09T06:35:27.704 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:27.724+0000 7f22167ad640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-05-09T06:35:27.704 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:27.724+0000 7f22167ad640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-05-09T06:35:30.555 DEBUG:teuthology.orchestra.run.smithi190:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-05-09T06:35:30.585 DEBUG:teuthology.orchestra.run.smithi190:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-09T06:35:30.677 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:30.696+0000 7fb7ad20f640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-05-09T06:35:30.677 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:30.696+0000 7fb7ad20f640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-05-09T06:35:30.677 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:30.696+0000 7fb7ad20f640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-05-09T06:35:30.678 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:30.696+0000 7fb7ad20f640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-05-09T06:35:30.678 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:30.696+0000 7fb7ad20f640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-05-09T06:35:30.678 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:30.697+0000 7fb7ad20f640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-05-09T06:35:33.538 DEBUG:teuthology.orchestra.run.smithi190:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-05-09T06:35:33.567 DEBUG:teuthology.orchestra.run.smithi190:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-09T06:35:33.660 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:33.679+0000 7f319b824640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-05-09T06:35:33.660 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:33.679+0000 7f319b824640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-05-09T06:35:33.660 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:33.679+0000 7f319b824640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-05-09T06:35:33.660 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:33.679+0000 7f319b824640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-05-09T06:35:33.660 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:33.679+0000 7f319b824640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-05-09T06:35:33.661 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:33.680+0000 7f319b824640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-05-09T06:35:36.155 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-09T06:35:36.159 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-09T06:35:36.514 DEBUG:teuthology.orchestra.run.smithi190:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-05-09T06:35:36.542 DEBUG:teuthology.orchestra.run.smithi190:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-09T06:35:36.630 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:36.648+0000 7f4e7d014640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-05-09T06:35:36.630 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:36.648+0000 7f4e7d014640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-05-09T06:35:36.630 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:36.648+0000 7f4e7d014640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-05-09T06:35:36.630 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:36.648+0000 7f4e7d014640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-05-09T06:35:36.630 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:36.648+0000 7f4e7d014640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-05-09T06:35:36.630 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:36.649+0000 7f4e7d014640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-05-09T06:35:39.477 DEBUG:teuthology.orchestra.run.smithi190:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-05-09T06:35:39.504 INFO:tasks.ceph:Reading keys from all nodes... 2024-05-09T06:35:39.504 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-05-09T06:35:39.504 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-05-09T06:35:39.536 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-05-09T06:35:39.536 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-05-09T06:35:39.607 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-05-09T06:35:39.607 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2024-05-09T06:35:39.677 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-05-09T06:35:39.677 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-05-09T06:35:39.747 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-05-09T06:35:39.747 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-05-09T06:35:39.816 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-05-09T06:35:39.817 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-05-09T06:35:39.884 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-05-09T06:35:39.884 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-05-09T06:35:39.952 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-05-09T06:35:39.952 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-05-09T06:35:39.982 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-05-09T06:35:39.982 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-05-09T06:35:40.051 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-05-09T06:35:40.051 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2024-05-09T06:35:40.119 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-05-09T06:35:40.119 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-05-09T06:35:40.187 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-05-09T06:35:40.187 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-05-09T06:35:40.254 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-05-09T06:35:40.254 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-05-09T06:35:40.320 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-05-09T06:35:40.321 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-05-09T06:35:40.386 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-05-09T06:35:40.386 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-05-09T06:35:40.406 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-05-09T06:35:40.406 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2024-05-09T06:35:40.468 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-05-09T06:35:40.468 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-05-09T06:35:40.485 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-05-09T06:35:40.485 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2024-05-09T06:35:40.540 INFO:tasks.ceph:Adding keys to all mons... 2024-05-09T06:35:40.541 DEBUG:teuthology.orchestra.run.smithi003:> sudo tee -a /etc/ceph/ceph.keyring 2024-05-09T06:35:40.543 DEBUG:teuthology.orchestra.run.smithi190:> sudo tee -a /etc/ceph/ceph.keyring 2024-05-09T06:35:40.584 INFO:teuthology.orchestra.run.smithi003.stdout:[mgr.y] 2024-05-09T06:35:40.584 INFO:teuthology.orchestra.run.smithi003.stdout: key = AQCcbjxmMadJJRAALIKKxlTHKn4WuluZ6SqzjQ== 2024-05-09T06:35:40.584 INFO:teuthology.orchestra.run.smithi003.stdout:[mds.a] 2024-05-09T06:35:40.584 INFO:teuthology.orchestra.run.smithi003.stdout: key = AQCcbjxmIM8dLRAAn0rimccNJB0tY1uuOIpVWg== 2024-05-09T06:35:40.584 INFO:teuthology.orchestra.run.smithi003.stdout:[mds.c] 2024-05-09T06:35:40.584 INFO:teuthology.orchestra.run.smithi003.stdout: key = AQCcbjxmyJ1SNBAA9tBFwVRrtU2iascCqj9azg== 2024-05-09T06:35:40.584 INFO:teuthology.orchestra.run.smithi003.stdout:[osd.0] 2024-05-09T06:35:40.585 INFO:teuthology.orchestra.run.smithi003.stdout: key = AQCgbjxmpTR+JhAAdhWCazu+ZuBtc+wZa8OgIw== 2024-05-09T06:35:40.585 INFO:teuthology.orchestra.run.smithi003.stdout:[osd.1] 2024-05-09T06:35:40.585 INFO:teuthology.orchestra.run.smithi003.stdout: key = AQCjbjxmc4w0JhAA45zp+/amPniB68hpBkaysQ== 2024-05-09T06:35:40.585 INFO:teuthology.orchestra.run.smithi003.stdout:[osd.2] 2024-05-09T06:35:40.585 INFO:teuthology.orchestra.run.smithi003.stdout: key = AQCmbjxmvvBQJRAAfDrk7T9dUnlypbAmTVp5yQ== 2024-05-09T06:35:40.585 INFO:teuthology.orchestra.run.smithi003.stdout:[osd.3] 2024-05-09T06:35:40.585 INFO:teuthology.orchestra.run.smithi003.stdout: key = AQCpbjxmEo2MJBAAS+oDw1c9dBSn5dTkfia6Kg== 2024-05-09T06:35:40.585 INFO:teuthology.orchestra.run.smithi003.stdout:[mgr.x] 2024-05-09T06:35:40.585 INFO:teuthology.orchestra.run.smithi003.stdout: key = AQCcbjxmCYDJKRAAY+qhWSKa8EqdbiguI2d5KA== 2024-05-09T06:35:40.585 INFO:teuthology.orchestra.run.smithi003.stdout:[mds.b] 2024-05-09T06:35:40.585 INFO:teuthology.orchestra.run.smithi003.stdout: key = AQCcbjxm5jUhORAAuT9nVpIYSt7HW1Bdn2Yy1g== 2024-05-09T06:35:40.585 INFO:teuthology.orchestra.run.smithi003.stdout:[mds.d] 2024-05-09T06:35:40.585 INFO:teuthology.orchestra.run.smithi003.stdout: key = AQCdbjxm49uZBBAAgu9Zg8TxJP+hHWvgRCl90Q== 2024-05-09T06:35:40.585 INFO:teuthology.orchestra.run.smithi003.stdout:[osd.4] 2024-05-09T06:35:40.585 INFO:teuthology.orchestra.run.smithi003.stdout: key = AQCvbjxm7sImKxAAT6K/yI3d38j33Kzx62v6cA== 2024-05-09T06:35:40.586 INFO:teuthology.orchestra.run.smithi003.stdout:[osd.5] 2024-05-09T06:35:40.586 INFO:teuthology.orchestra.run.smithi003.stdout: key = AQCybjxmIq2UKRAAsSo6S3rfKxiFMYJUtOGFRw== 2024-05-09T06:35:40.586 INFO:teuthology.orchestra.run.smithi003.stdout:[osd.6] 2024-05-09T06:35:40.586 INFO:teuthology.orchestra.run.smithi003.stdout: key = AQC1bjxmIJWOKBAAwvyw8lN+6rck9nRcFVV2ZA== 2024-05-09T06:35:40.586 INFO:teuthology.orchestra.run.smithi003.stdout:[osd.7] 2024-05-09T06:35:40.586 INFO:teuthology.orchestra.run.smithi003.stdout: key = AQC4bjxmc367JhAADo/hC2C8Mr3HxcxbKga10Q== 2024-05-09T06:35:40.586 INFO:teuthology.orchestra.run.smithi003.stdout:[client.0] 2024-05-09T06:35:40.586 INFO:teuthology.orchestra.run.smithi003.stdout: key = AQCdbjxmhD/JCBAAcZ8c4sZXw8rR2/s1Lr++wQ== 2024-05-09T06:35:40.586 INFO:teuthology.orchestra.run.smithi003.stdout:[client.2] 2024-05-09T06:35:40.586 INFO:teuthology.orchestra.run.smithi003.stdout: key = AQCdbjxmSqR/DhAA3AJMR+k+5sFEe+XJ8zJrSg== 2024-05-09T06:35:40.586 INFO:teuthology.orchestra.run.smithi003.stdout:[client.1] 2024-05-09T06:35:40.586 INFO:teuthology.orchestra.run.smithi003.stdout: key = AQCdbjxmTm+qERAACTtDZrYaqM/GfaXdtuzksw== 2024-05-09T06:35:40.586 INFO:teuthology.orchestra.run.smithi003.stdout:[client.3] 2024-05-09T06:35:40.586 INFO:teuthology.orchestra.run.smithi003.stdout: key = AQCdbjxm9ktOFxAA5520o6cXlTpMtyhm05SLfw== 2024-05-09T06:35:40.604 INFO:teuthology.orchestra.run.smithi190.stdout:[mgr.y] 2024-05-09T06:35:40.604 INFO:teuthology.orchestra.run.smithi190.stdout: key = AQCcbjxmMadJJRAALIKKxlTHKn4WuluZ6SqzjQ== 2024-05-09T06:35:40.605 INFO:teuthology.orchestra.run.smithi190.stdout:[mds.a] 2024-05-09T06:35:40.605 INFO:teuthology.orchestra.run.smithi190.stdout: key = AQCcbjxmIM8dLRAAn0rimccNJB0tY1uuOIpVWg== 2024-05-09T06:35:40.605 INFO:teuthology.orchestra.run.smithi190.stdout:[mds.c] 2024-05-09T06:35:40.605 INFO:teuthology.orchestra.run.smithi190.stdout: key = AQCcbjxmyJ1SNBAA9tBFwVRrtU2iascCqj9azg== 2024-05-09T06:35:40.605 INFO:teuthology.orchestra.run.smithi190.stdout:[osd.0] 2024-05-09T06:35:40.605 INFO:teuthology.orchestra.run.smithi190.stdout: key = AQCgbjxmpTR+JhAAdhWCazu+ZuBtc+wZa8OgIw== 2024-05-09T06:35:40.605 INFO:teuthology.orchestra.run.smithi190.stdout:[osd.1] 2024-05-09T06:35:40.605 INFO:teuthology.orchestra.run.smithi190.stdout: key = AQCjbjxmc4w0JhAA45zp+/amPniB68hpBkaysQ== 2024-05-09T06:35:40.605 INFO:teuthology.orchestra.run.smithi190.stdout:[osd.2] 2024-05-09T06:35:40.605 INFO:teuthology.orchestra.run.smithi190.stdout: key = AQCmbjxmvvBQJRAAfDrk7T9dUnlypbAmTVp5yQ== 2024-05-09T06:35:40.605 INFO:teuthology.orchestra.run.smithi190.stdout:[osd.3] 2024-05-09T06:35:40.605 INFO:teuthology.orchestra.run.smithi190.stdout: key = AQCpbjxmEo2MJBAAS+oDw1c9dBSn5dTkfia6Kg== 2024-05-09T06:35:40.605 INFO:teuthology.orchestra.run.smithi190.stdout:[mgr.x] 2024-05-09T06:35:40.605 INFO:teuthology.orchestra.run.smithi190.stdout: key = AQCcbjxmCYDJKRAAY+qhWSKa8EqdbiguI2d5KA== 2024-05-09T06:35:40.606 INFO:teuthology.orchestra.run.smithi190.stdout:[mds.b] 2024-05-09T06:35:40.606 INFO:teuthology.orchestra.run.smithi190.stdout: key = AQCcbjxm5jUhORAAuT9nVpIYSt7HW1Bdn2Yy1g== 2024-05-09T06:35:40.606 INFO:teuthology.orchestra.run.smithi190.stdout:[mds.d] 2024-05-09T06:35:40.606 INFO:teuthology.orchestra.run.smithi190.stdout: key = AQCdbjxm49uZBBAAgu9Zg8TxJP+hHWvgRCl90Q== 2024-05-09T06:35:40.606 INFO:teuthology.orchestra.run.smithi190.stdout:[osd.4] 2024-05-09T06:35:40.606 INFO:teuthology.orchestra.run.smithi190.stdout: key = AQCvbjxm7sImKxAAT6K/yI3d38j33Kzx62v6cA== 2024-05-09T06:35:40.606 INFO:teuthology.orchestra.run.smithi190.stdout:[osd.5] 2024-05-09T06:35:40.606 INFO:teuthology.orchestra.run.smithi190.stdout: key = AQCybjxmIq2UKRAAsSo6S3rfKxiFMYJUtOGFRw== 2024-05-09T06:35:40.606 INFO:teuthology.orchestra.run.smithi190.stdout:[osd.6] 2024-05-09T06:35:40.606 INFO:teuthology.orchestra.run.smithi190.stdout: key = AQC1bjxmIJWOKBAAwvyw8lN+6rck9nRcFVV2ZA== 2024-05-09T06:35:40.606 INFO:teuthology.orchestra.run.smithi190.stdout:[osd.7] 2024-05-09T06:35:40.606 INFO:teuthology.orchestra.run.smithi190.stdout: key = AQC4bjxmc367JhAADo/hC2C8Mr3HxcxbKga10Q== 2024-05-09T06:35:40.606 INFO:teuthology.orchestra.run.smithi190.stdout:[client.0] 2024-05-09T06:35:40.606 INFO:teuthology.orchestra.run.smithi190.stdout: key = AQCdbjxmhD/JCBAAcZ8c4sZXw8rR2/s1Lr++wQ== 2024-05-09T06:35:40.606 INFO:teuthology.orchestra.run.smithi190.stdout:[client.2] 2024-05-09T06:35:40.607 INFO:teuthology.orchestra.run.smithi190.stdout: key = AQCdbjxmSqR/DhAA3AJMR+k+5sFEe+XJ8zJrSg== 2024-05-09T06:35:40.607 INFO:teuthology.orchestra.run.smithi190.stdout:[client.1] 2024-05-09T06:35:40.607 INFO:teuthology.orchestra.run.smithi190.stdout: key = AQCdbjxmTm+qERAACTtDZrYaqM/GfaXdtuzksw== 2024-05-09T06:35:40.607 INFO:teuthology.orchestra.run.smithi190.stdout:[client.3] 2024-05-09T06:35:40.607 INFO:teuthology.orchestra.run.smithi190.stdout: key = AQCdbjxm9ktOFxAA5520o6cXlTpMtyhm05SLfw== 2024-05-09T06:35:40.608 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-05-09T06:35:40.633 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-05-09T06:35:40.700 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-09T06:35:40.703 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-09T06:35:40.759 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-09T06:35:40.762 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-09T06:35:40.816 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-09T06:35:40.819 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-09T06:35:40.870 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-09T06:35:40.872 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-09T06:35:40.923 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-09T06:35:40.925 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-09T06:35:40.976 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-09T06:35:40.979 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-09T06:35:41.029 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-05-09T06:35:41.033 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-05-09T06:35:41.084 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-09T06:35:41.086 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-09T06:35:41.139 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-09T06:35:41.142 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-09T06:35:41.195 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-09T06:35:41.198 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-09T06:35:41.249 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-09T06:35:41.251 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-09T06:35:41.303 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-09T06:35:41.305 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-09T06:35:41.357 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-09T06:35:41.359 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-09T06:35:41.410 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-09T06:35:41.416 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-09T06:35:41.471 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-09T06:35:41.473 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-09T06:35:41.528 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-09T06:35:41.530 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-09T06:35:41.584 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-09T06:35:41.586 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-09T06:35:41.640 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-05-09T06:35:41.640 DEBUG:teuthology.orchestra.run.smithi003:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-05-09T06:35:41.665 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-05-09T06:35:41.763 DEBUG:teuthology.orchestra.run.smithi003:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-05-09T06:35:41.792 DEBUG:teuthology.orchestra.run.smithi190:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-05-09T06:35:41.823 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-05-09T06:35:41.947 DEBUG:teuthology.orchestra.run.smithi190:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-05-09T06:35:41.973 DEBUG:teuthology.orchestra.run.smithi190:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-05-09T06:35:42.039 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-05-09T06:35:42.135 DEBUG:teuthology.orchestra.run.smithi190:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-05-09T06:35:42.160 DEBUG:teuthology.orchestra.run.smithi003:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-05-09T06:35:42.162 DEBUG:teuthology.orchestra.run.smithi190:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-05-09T06:35:42.216 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-05-09T06:35:42.216 INFO:tasks.ceph.mon.a:Restarting daemon 2024-05-09T06:35:42.217 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-05-09T06:35:42.219 INFO:tasks.ceph.mon.a:Started 2024-05-09T06:35:42.219 INFO:tasks.ceph.mon.b:Restarting daemon 2024-05-09T06:35:42.219 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-05-09T06:35:42.259 INFO:tasks.ceph.mon.b:Started 2024-05-09T06:35:42.259 INFO:tasks.ceph.mon.c:Restarting daemon 2024-05-09T06:35:42.259 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-05-09T06:35:42.261 INFO:tasks.ceph.mon.c:Started 2024-05-09T06:35:42.261 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-05-09T06:35:42.261 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-05-09T06:35:42.262 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-05-09T06:35:42.264 INFO:tasks.ceph.mgr.y:Started 2024-05-09T06:35:42.264 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-05-09T06:35:42.264 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-05-09T06:35:42.266 INFO:tasks.ceph.mgr.x:Started 2024-05-09T06:35:42.266 INFO:tasks.ceph:Setting crush tunables to default 2024-05-09T06:35:42.266 DEBUG:teuthology.orchestra.run.smithi003:> sudo ceph --cluster ceph osd crush tunables default 2024-05-09T06:35:42.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:35:42.390+0000 7f8bcf148640 1 Processor -- start 2024-05-09T06:35:42.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:35:42.390+0000 7f8bcf148640 1 -- start start 2024-05-09T06:35:42.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:35:42.391+0000 7f8bcf148640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f8bc80ce760 0x7f8bc80cebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:42.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:35:42.391+0000 7f8bcf148640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f8bc80b3e50 0x7f8bc80b42e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:42.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:35:42.391+0000 7f8bcf148640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8bc80b49c0 con 0x7f8bc8072020 2024-05-09T06:35:42.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:35:42.391+0000 7f8bcf148640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f8bc80b4ca0 con 0x7f8bc80ce760 2024-05-09T06:35:42.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:35:42.391+0000 7f8bcf148640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f8bc806e3f0 con 0x7f8bc80b3e50 2024-05-09T06:35:42.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:35:42.391+0000 7f8bc7fff640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f8bc80ce760 0x7f8bc80cebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:42.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:35:42.391+0000 7f8bc7fff640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f8bc80ce760 0x7f8bc80cebc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:42082/0 (socket says 172.21.15.3:42082) 2024-05-09T06:35:42.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:35:42.391+0000 7f8bc7fff640 1 -- 172.21.15.3:0/1908285668 learned_addr learned my addr 172.21.15.3:0/1908285668 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:35:42.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:35:42.391+0000 7f8bc7fff640 1 -- 172.21.15.3:0/1908285668 >> v2:172.21.15.3:3300/0 conn(0x7f8bc80ce760 msgr2=0x7f8bc80cebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-05-09T06:35:42.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:35:42.391+0000 7f8bc7fff640 1 -- 172.21.15.3:0/1908285668 >> v2:172.21.15.3:3300/0 conn(0x7f8bc80ce760 msgr2=0x7f8bc80cebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-09T06:35:42.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:35:42.391+0000 7f8bc7fff640 1 --2- 172.21.15.3:0/1908285668 >> v2:172.21.15.3:3300/0 conn(0x7f8bc80ce760 0x7f8bc80cebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-09T06:35:42.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:35:42.391+0000 7f8bc7fff640 1 --2- 172.21.15.3:0/1908285668 >> v2:172.21.15.3:3300/0 conn(0x7f8bc80ce760 0x7f8bc80cebc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-09T06:35:42.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:35:42.391+0000 7f8bcd6e5640 1 --2- 172.21.15.3:0/1908285668 >> v2:172.21.15.190:3300/0 conn(0x7f8bc80b3e50 0x7f8bc80b42e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:42.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:35:42.391+0000 7f8bcd6e5640 1 -- 172.21.15.3:0/1908285668 >> v2:172.21.15.190:3300/0 conn(0x7f8bc80b3e50 msgr2=0x7f8bc80b42e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-05-09T06:35:42.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:35:42.391+0000 7f8bcd6e5640 1 -- 172.21.15.3:0/1908285668 >> v2:172.21.15.190:3300/0 conn(0x7f8bc80b3e50 msgr2=0x7f8bc80b42e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-09T06:35:42.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:35:42.391+0000 7f8bcd6e5640 1 --2- 172.21.15.3:0/1908285668 >> v2:172.21.15.190:3300/0 conn(0x7f8bc80b3e50 0x7f8bc80b42e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-09T06:35:42.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:35:42.391+0000 7f8bcd6e5640 1 --2- 172.21.15.3:0/1908285668 >> v2:172.21.15.190:3300/0 conn(0x7f8bc80b3e50 0x7f8bc80b42e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-09T06:35:42.541 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:35:42.558+0000 7f8bc77fe640 1 -- 172.21.15.3:0/1908285668 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2443003873 0 0) 0x7f8bc80b49c0 con 0x7f8bc8072020 2024-05-09T06:35:42.541 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:35:42.558+0000 7f8bc77fe640 1 -- 172.21.15.3:0/1908285668 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8bb8003520 con 0x7f8bc8072020 2024-05-09T06:35:42.541 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:35:42.558+0000 7f8bc77fe640 1 -- 172.21.15.3:0/1908285668 <== mon.1 v1:172.21.15.190:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 85836888 0 0) 0x7f8bb8003520 con 0x7f8bc8072020 2024-05-09T06:35:42.541 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:35:42.558+0000 7f8bc77fe640 1 -- 172.21.15.3:0/1908285668 >> v2:172.21.15.190:3300/0 conn(0x7f8bc80b3e50 msgr2=0x7f8bc80b42e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T06:35:42.541 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:35:42.558+0000 7f8bc77fe640 1 --2- 172.21.15.3:0/1908285668 >> v2:172.21.15.190:3300/0 conn(0x7f8bc80b3e50 0x7f8bc80b42e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:42.541 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:35:42.558+0000 7f8bc77fe640 1 -- 172.21.15.3:0/1908285668 >> v2:172.21.15.3:3300/0 conn(0x7f8bc80ce760 msgr2=0x7f8bc80cebc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T06:35:42.541 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:35:42.558+0000 7f8bc77fe640 1 --2- 172.21.15.3:0/1908285668 >> v2:172.21.15.3:3300/0 conn(0x7f8bc80ce760 0x7f8bc80cebc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:42.541 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:35:42.558+0000 7f8bc77fe640 1 -- 172.21.15.3:0/1908285668 --> v1:172.21.15.190:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bc806e6d0 con 0x7f8bc8072020 2024-05-09T06:35:42.542 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:35:42.558+0000 7f8bc77fe640 1 -- 172.21.15.3:0/1908285668 <== mon.1 v1:172.21.15.190:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 563312031 0 0) 0x7f8bbc001670 con 0x7f8bc8072020 2024-05-09T06:35:42.542 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:35:42.558+0000 7f8bc77fe640 1 -- 172.21.15.3:0/1908285668 >> v1:172.21.15.190:6789/0 conn(0x7f8bc8072020 legacy=0x7f8bc8072400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:42.542 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:35:42.558+0000 7f8bc77fe640 1 --2- 172.21.15.3:0/1908285668 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8bb8003a00 0x7f8bb8003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:42.542 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:35:42.558+0000 7f8bc77fe640 1 --2- 172.21.15.3:0/1908285668 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8bb8004800 0x7f8bb8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:42.542 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:35:42.558+0000 7f8bc77fe640 1 --2- 172.21.15.3:0/1908285668 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8bb80051c0 0x7f8bb800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:42.542 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:35:42.558+0000 7f8bc77fe640 1 -- 172.21.15.3:0/1908285668 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f8bc80b4ca0 con 0x7f8bb80051c0 2024-05-09T06:35:42.542 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:35:42.558+0000 7f8bc77fe640 1 -- 172.21.15.3:0/1908285668 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f8bc806e3f0 con 0x7f8bb8003a00 2024-05-09T06:35:42.542 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:35:42.558+0000 7f8bc77fe640 1 -- 172.21.15.3:0/1908285668 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f8bb800c110 con 0x7f8bb8004800 2024-05-09T06:35:42.542 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:35:42.559+0000 7f8bcd6e5640 1 --2- 172.21.15.3:0/1908285668 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8bb80051c0 0x7f8bb800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:42.542 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:35:42.559+0000 7f8bccee4640 1 --2- 172.21.15.3:0/1908285668 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8bb8003a00 0x7f8bb8003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:42.542 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:35:42.559+0000 7f8bc7fff640 1 --2- 172.21.15.3:0/1908285668 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8bb8004800 0x7f8bb8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:42.542 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:35:42.559+0000 7f8bcd6e5640 1 -- 172.21.15.3:0/1908285668 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8bb8004800 msgr2=0x7f8bb8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:35:42.542 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:35:42.559+0000 7f8bcd6e5640 1 --2- 172.21.15.3:0/1908285668 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8bb8004800 0x7f8bb8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:42.542 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:35:42.559+0000 7f8bcd6e5640 1 -- 172.21.15.3:0/1908285668 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8bb8003a00 msgr2=0x7f8bb8003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:35:42.543 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:35:42.559+0000 7f8bcd6e5640 1 --2- 172.21.15.3:0/1908285668 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8bb8003a00 0x7f8bb8003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:42.543 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:35:42.559+0000 7f8bcd6e5640 1 -- 172.21.15.3:0/1908285668 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bb8003520 con 0x7f8bb80051c0 2024-05-09T06:35:42.543 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:35:42.560+0000 7f8bcd6e5640 1 -- 172.21.15.3:0/1908285668 reap_dead start 2024-05-09T06:35:42.543 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:35:42.560+0000 7f8bcd6e5640 1 --2- 172.21.15.3:0/1908285668 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8bb80051c0 0x7f8bb800b9c0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f8bc0009fd0 tx=0x7f8bb8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f55e6b783e64194 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:35:42.543 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:35:42.560+0000 7f8bc77fe640 1 -- 172.21.15.3:0/1908285668 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bc003a070 con 0x7f8bb80051c0 2024-05-09T06:35:42.543 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:35:42.560+0000 7f8bc77fe640 1 -- 172.21.15.3:0/1908285668 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8bc002d790 con 0x7f8bb80051c0 2024-05-09T06:35:42.543 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:35:42.560+0000 7f8bc77fe640 1 -- 172.21.15.3:0/1908285668 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bc0009bd0 con 0x7f8bb80051c0 2024-05-09T06:35:42.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:35:42.561+0000 7f8bcf148640 1 -- 172.21.15.3:0/1908285668 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8bb80051c0 msgr2=0x7f8bb800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:42.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:35:42.561+0000 7f8bcf148640 1 --2- 172.21.15.3:0/1908285668 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8bb80051c0 0x7f8bb800b9c0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f8bc0009fd0 tx=0x7f8bb8005830 comp rx=0 tx=0).stop 2024-05-09T06:35:42.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:35:42.561+0000 7f8bcf148640 1 -- 172.21.15.3:0/1908285668 shutdown_connections 2024-05-09T06:35:42.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:35:42.561+0000 7f8bcf148640 1 --2- 172.21.15.3:0/1908285668 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8bb80051c0 0x7f8bb800b9c0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:42.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:35:42.561+0000 7f8bcf148640 1 -- 172.21.15.3:0/1908285668 >> 172.21.15.3:0/1908285668 conn(0x7f8bc80b2b50 msgr2=0x7f8bc80b2f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:35:42.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:35:42.561+0000 7f8bcf148640 1 -- 172.21.15.3:0/1908285668 shutdown_connections 2024-05-09T06:35:42.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:35:42.561+0000 7f8bcf148640 1 -- 172.21.15.3:0/1908285668 wait complete. 2024-05-09T06:35:42.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:35:42.562+0000 7f8bcf148640 1 Processor -- start 2024-05-09T06:35:42.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:35:42.562+0000 7f8bcf148640 1 -- start start 2024-05-09T06:35:42.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:35:42.563+0000 7f8bcf148640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8bb80051c0 0x7f8bc815eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:42.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:35:42.563+0000 7f8bcf148640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8bc8159160 0x7f8bc813c700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:42.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:35:42.563+0000 7f8bcf148640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8bc81359e0 0x7f8bc8135e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:42.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:35:42.563+0000 7f8bcf148640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f8bc8144e50 con 0x7f8bc81359e0 2024-05-09T06:35:42.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:35:42.563+0000 7f8bcf148640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f8bc8144cd0 con 0x7f8bb80051c0 2024-05-09T06:35:42.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:35:42.563+0000 7f8bcf148640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f8bc80cebc0 con 0x7f8bc8159160 2024-05-09T06:35:42.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:35:42.564+0000 7f8bcd6e5640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8bc81359e0 0x7f8bc8135e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:42.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:35:42.564+0000 7f8bc7fff640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8bc8159160 0x7f8bc813c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:42.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:35:42.564+0000 7f8bcd6e5640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8bc81359e0 0x7f8bc8135e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:42108/0 (socket says 172.21.15.3:42108) 2024-05-09T06:35:42.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:35:42.564+0000 7f8bcd6e5640 1 -- 172.21.15.3:0/3217131433 learned_addr learned my addr 172.21.15.3:0/3217131433 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:35:42.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:35:42.564+0000 7f8bccee4640 1 --2- 172.21.15.3:0/3217131433 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8bb80051c0 0x7f8bc815eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:42.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:35:42.564+0000 7f8bcd6e5640 1 -- 172.21.15.3:0/3217131433 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8bc8159160 msgr2=0x7f8bc813c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:42.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:35:42.564+0000 7f8bcd6e5640 1 --2- 172.21.15.3:0/3217131433 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8bc8159160 0x7f8bc813c700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:42.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:35:42.564+0000 7f8bcd6e5640 1 -- 172.21.15.3:0/3217131433 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8bb80051c0 msgr2=0x7f8bc815eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:42.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:35:42.564+0000 7f8bcd6e5640 1 --2- 172.21.15.3:0/3217131433 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8bb80051c0 0x7f8bc815eac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:42.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:35:42.564+0000 7f8bcd6e5640 1 -- 172.21.15.3:0/3217131433 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bc81595e0 con 0x7f8bc81359e0 2024-05-09T06:35:42.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:35:42.564+0000 7f8bcd6e5640 1 --2- 172.21.15.3:0/3217131433 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8bc81359e0 0x7f8bc8135e60 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f8bc0007420 tx=0x7f8bc0004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:35:42.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:35:42.564+0000 7f8bc57fa640 1 -- 172.21.15.3:0/3217131433 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bc0046070 con 0x7f8bc81359e0 2024-05-09T06:35:42.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:35:42.564+0000 7f8bc57fa640 1 -- 172.21.15.3:0/3217131433 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8bc0035660 con 0x7f8bc81359e0 2024-05-09T06:35:42.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:35:42.564+0000 7f8bcf148640 1 -- 172.21.15.3:0/3217131433 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8bc813cc40 con 0x7f8bc81359e0 2024-05-09T06:35:42.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:35:42.564+0000 7f8bc57fa640 1 -- 172.21.15.3:0/3217131433 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bc003a040 con 0x7f8bc81359e0 2024-05-09T06:35:42.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:35:42.564+0000 7f8bcf148640 1 -- 172.21.15.3:0/3217131433 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8bc813d1c0 con 0x7f8bc81359e0 2024-05-09T06:35:42.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:35:42.565+0000 7f8bc57fa640 1 -- 172.21.15.3:0/3217131433 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f8bc0049070 con 0x7f8bc81359e0 2024-05-09T06:35:42.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:35:42.565+0000 7f8bc57fa640 1 -- 172.21.15.3:0/3217131433 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f8bc004bb80 con 0x7f8bc81359e0 2024-05-09T06:35:42.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:35:42.566+0000 7f8bcf148640 1 -- 172.21.15.3:0/3217131433 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8bc80682e0 con 0x7f8bc81359e0 2024-05-09T06:35:42.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:35:42.568+0000 7f8bc57fa640 1 -- 172.21.15.3:0/3217131433 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f8bc0050080 con 0x7f8bc81359e0 2024-05-09T06:35:42.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:35:42.635+0000 7f8bcf148640 1 -- 172.21.15.3:0/3217131433 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f8bc8072d70 con 0x7f8bc81359e0 2024-05-09T06:35:42.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:35:42.637+0000 7f8bc57fa640 1 -- 172.21.15.3:0/3217131433 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) ==== 124+0+0 (secure 0 0 0) 0x7f8bc0009bd0 con 0x7f8bc81359e0 2024-05-09T06:35:42.618 INFO:teuthology.orchestra.run.smithi003.stderr:adjusted tunables profile to default 2024-05-09T06:35:42.619 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:35:42.638+0000 7f8bcf148640 1 -- 172.21.15.3:0/3217131433 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8bc81359e0 msgr2=0x7f8bc8135e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:42.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:35:42.638+0000 7f8bcf148640 1 --2- 172.21.15.3:0/3217131433 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8bc81359e0 0x7f8bc8135e60 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f8bc0007420 tx=0x7f8bc0004ab0 comp rx=0 tx=0).stop 2024-05-09T06:35:42.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:35:42.638+0000 7f8bcf148640 1 -- 172.21.15.3:0/3217131433 shutdown_connections 2024-05-09T06:35:42.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:35:42.638+0000 7f8bcf148640 1 --2- 172.21.15.3:0/3217131433 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8bc81359e0 0x7f8bc8135e60 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:42.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:35:42.638+0000 7f8bcf148640 1 --2- 172.21.15.3:0/3217131433 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8bc8159160 0x7f8bc813c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:42.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:35:42.638+0000 7f8bcf148640 1 --2- 172.21.15.3:0/3217131433 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8bb80051c0 0x7f8bc815eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:42.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:35:42.638+0000 7f8bcf148640 1 -- 172.21.15.3:0/3217131433 >> 172.21.15.3:0/3217131433 conn(0x7f8bc80b2b50 msgr2=0x7f8bc80d0620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:35:42.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:35:42.638+0000 7f8bcf148640 1 -- 172.21.15.3:0/3217131433 shutdown_connections 2024-05-09T06:35:42.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:35:42.638+0000 7f8bcf148640 1 -- 172.21.15.3:0/3217131433 wait complete. 2024-05-09T06:35:42.630 INFO:tasks.ceph:check_enable_crimson: False 2024-05-09T06:35:42.630 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-05-09T06:35:42.630 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-05-09T06:35:42.630 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-05-09T06:35:42.649 INFO:tasks.ceph.mgr.x.smithi190.stderr:2024-05-09T06:35:42.668+0000 7fab08831040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-09T06:35:42.655 INFO:tasks.ceph.mgr.y.smithi003.stderr:2024-05-09T06:35:42.674+0000 7f733de11040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-09T06:35:42.656 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-05-09T06:35:42.656 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-05-09T06:35:42.723 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-05-09T06:35:42.723 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-05-09T06:35:42.789 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-05-09T06:35:42.790 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-05-09T06:35:42.809 INFO:tasks.ceph.mgr.x.smithi190.stderr:2024-05-09T06:35:42.829+0000 7fab08831040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-09T06:35:42.819 INFO:tasks.ceph.mgr.y.smithi003.stderr:2024-05-09T06:35:42.838+0000 7f733de11040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-09T06:35:42.844 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-05-09T06:35:42.844 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-05-09T06:35:42.863 INFO:tasks.ceph.mgr.x.smithi190.stderr:2024-05-09T06:35:42.882+0000 7fab08831040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-09T06:35:42.870 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-05-09T06:35:42.870 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-05-09T06:35:42.876 INFO:tasks.ceph.mgr.y.smithi003.stderr:2024-05-09T06:35:42.895+0000 7f733de11040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-09T06:35:42.934 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-05-09T06:35:42.935 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-05-09T06:35:42.999 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-05-09T06:35:42.999 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-05-09T06:35:43.064 DEBUG:teuthology.orchestra.run.smithi190:> sudo ceph --cluster ceph osd new fae3a8db-4826-4f3c-b33a-12c09bb5df6a 0 2024-05-09T06:35:43.118 INFO:tasks.ceph.mgr.x.smithi190.stderr:2024-05-09T06:35:43.137+0000 7fab08831040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-09T06:35:43.142 INFO:tasks.ceph.mgr.y.smithi003.stderr:2024-05-09T06:35:43.161+0000 7f733de11040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-09T06:35:43.166 INFO:tasks.ceph.mgr.x.smithi190.stderr:2024-05-09T06:35:43.186+0000 7fab08831040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-09T06:35:43.189 INFO:tasks.ceph.mgr.y.smithi003.stderr:2024-05-09T06:35:43.209+0000 7f733de11040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-09T06:35:43.204 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.223+0000 7f97bfde7640 1 Processor -- start 2024-05-09T06:35:43.204 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.223+0000 7f97bfde7640 1 -- start start 2024-05-09T06:35:43.206 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.225+0000 7f97bfde7640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f97b813f0a0 0x7f97b8141f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:43.207 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.225+0000 7f97bfde7640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f97b8141860 0x7f97b806b250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:43.207 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.225+0000 7f97bfde7640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f97b806b790 0x7f97b8072090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:43.207 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.225+0000 7f97bfde7640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f97b815fc50 con 0x7f97b813f0a0 2024-05-09T06:35:43.207 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.225+0000 7f97bfde7640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f97b815ff30 con 0x7f97b8141860 2024-05-09T06:35:43.207 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.225+0000 7f97bfde7640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f97b8160210 con 0x7f97b806b790 2024-05-09T06:35:43.207 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.225+0000 7f97bd382640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f97b8141860 0x7f97b806b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:43.207 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.225+0000 7f97be384640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f97b806b790 0x7f97b8072090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:43.207 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.225+0000 7f97bd382640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f97b8141860 0x7f97b806b250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.190:36272/0 (socket says 172.21.15.190:36272) 2024-05-09T06:35:43.207 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.225+0000 7f97bd382640 1 -- 172.21.15.190:0/2525382596 learned_addr learned my addr 172.21.15.190:0/2525382596 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-05-09T06:35:43.207 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.225+0000 7f97bdb83640 1 --2- 172.21.15.190:0/2525382596 >> v2:172.21.15.3:3300/0 conn(0x7f97b813f0a0 0x7f97b8141f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:43.207 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.225+0000 7f97bd382640 1 -- 172.21.15.190:0/2525382596 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f97b806b790 msgr2=0x7f97b8072090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:35:43.207 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.225+0000 7f97bd382640 1 --2- 172.21.15.190:0/2525382596 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f97b806b790 0x7f97b8072090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:43.207 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.225+0000 7f97bd382640 1 -- 172.21.15.190:0/2525382596 >> v2:172.21.15.3:3300/0 conn(0x7f97b813f0a0 msgr2=0x7f97b8141f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:35:43.208 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.225+0000 7f97bd382640 1 --2- 172.21.15.190:0/2525382596 >> v2:172.21.15.3:3300/0 conn(0x7f97b813f0a0 0x7f97b8141f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:43.208 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.225+0000 7f97bd382640 1 -- 172.21.15.190:0/2525382596 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97b81604f0 con 0x7f97b8141860 2024-05-09T06:35:43.208 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.225+0000 7f97be384640 1 --2- 172.21.15.190:0/2525382596 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f97b806b790 0x7f97b8072090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T06:35:43.208 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.226+0000 7f97bd382640 1 --2- 172.21.15.190:0/2525382596 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f97b8141860 0x7f97b806b250 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f97b80b64e0 tx=0x7f97b402a1e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b0303dd99d9aef49 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:35:43.208 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.226+0000 7f97bcb81640 1 -- 172.21.15.190:0/2525382596 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97b4037070 con 0x7f97b8141860 2024-05-09T06:35:43.208 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.226+0000 7f97bcb81640 1 -- 172.21.15.190:0/2525382596 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f97b4032440 con 0x7f97b8141860 2024-05-09T06:35:43.208 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.226+0000 7f97bcb81640 1 -- 172.21.15.190:0/2525382596 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97b4032810 con 0x7f97b8141860 2024-05-09T06:35:43.209 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.227+0000 7f97bfde7640 1 -- 172.21.15.190:0/2525382596 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f97b8141860 msgr2=0x7f97b806b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:43.209 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.227+0000 7f97bfde7640 1 --2- 172.21.15.190:0/2525382596 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f97b8141860 0x7f97b806b250 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f97b80b64e0 tx=0x7f97b402a1e0 comp rx=0 tx=0).stop 2024-05-09T06:35:43.209 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.227+0000 7f97bfde7640 1 -- 172.21.15.190:0/2525382596 shutdown_connections 2024-05-09T06:35:43.209 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.227+0000 7f97bfde7640 1 --2- 172.21.15.190:0/2525382596 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f97b806b790 0x7f97b8072090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:43.209 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.227+0000 7f97bfde7640 1 --2- 172.21.15.190:0/2525382596 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f97b8141860 0x7f97b806b250 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:43.209 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.227+0000 7f97bfde7640 1 --2- 172.21.15.190:0/2525382596 >> v2:172.21.15.3:3300/0 conn(0x7f97b813f0a0 0x7f97b8141f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:43.209 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.227+0000 7f97bfde7640 1 -- 172.21.15.190:0/2525382596 >> 172.21.15.190:0/2525382596 conn(0x7f97b80b2c30 msgr2=0x7f97b80b3010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:35:43.210 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.227+0000 7f97bfde7640 1 -- 172.21.15.190:0/2525382596 shutdown_connections 2024-05-09T06:35:43.210 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.227+0000 7f97bfde7640 1 -- 172.21.15.190:0/2525382596 wait complete. 2024-05-09T06:35:43.210 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.228+0000 7f97bfde7640 1 Processor -- start 2024-05-09T06:35:43.210 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.228+0000 7f97bfde7640 1 -- start start 2024-05-09T06:35:43.212 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.229+0000 7f97bfde7640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f97b806b790 0x7f97b815e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:43.212 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.229+0000 7f97bfde7640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f97b813f0a0 0x7f97b80afda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:43.212 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.229+0000 7f97bfde7640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f97b80ae0b0 0x7f97b80b02e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:43.212 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.229+0000 7f97bfde7640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f97b81425c0 con 0x7f97b806b790 2024-05-09T06:35:43.212 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.229+0000 7f97bfde7640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f97b8142440 con 0x7f97b80ae0b0 2024-05-09T06:35:43.213 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.229+0000 7f97bfde7640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f97b80725d0 con 0x7f97b813f0a0 2024-05-09T06:35:43.213 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.230+0000 7f97bd382640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f97b813f0a0 0x7f97b80afda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:43.213 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.230+0000 7f97be384640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f97b80ae0b0 0x7f97b80b02e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:43.213 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.230+0000 7f97bd382640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f97b813f0a0 0x7f97b80afda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.190:44508/0 (socket says 172.21.15.190:44508) 2024-05-09T06:35:43.213 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.230+0000 7f97bd382640 1 -- 172.21.15.190:0/1824273330 learned_addr learned my addr 172.21.15.190:0/1824273330 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-05-09T06:35:43.213 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.230+0000 7f97bdb83640 1 --2- 172.21.15.190:0/1824273330 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f97b806b790 0x7f97b815e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:43.213 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.230+0000 7f97be384640 1 -- 172.21.15.190:0/1824273330 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f97b813f0a0 msgr2=0x7f97b80afda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:43.213 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.230+0000 7f97be384640 1 --2- 172.21.15.190:0/1824273330 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f97b813f0a0 0x7f97b80afda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:43.213 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.230+0000 7f97be384640 1 -- 172.21.15.190:0/1824273330 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f97b806b790 msgr2=0x7f97b815e3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:43.213 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.230+0000 7f97be384640 1 --2- 172.21.15.190:0/1824273330 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f97b806b790 0x7f97b815e3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:43.213 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.230+0000 7f97be384640 1 -- 172.21.15.190:0/1824273330 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97b80ae670 con 0x7f97b80ae0b0 2024-05-09T06:35:43.213 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.230+0000 7f97bd382640 1 --2- 172.21.15.190:0/1824273330 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f97b813f0a0 0x7f97b80afda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T06:35:43.213 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.230+0000 7f97be384640 1 --2- 172.21.15.190:0/1824273330 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f97b80ae0b0 0x7f97b80b02e0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f97b000efc0 tx=0x7f97b000c490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:35:43.213 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.230+0000 7f97ae7fc640 1 -- 172.21.15.190:0/1824273330 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97b0018030 con 0x7f97b80ae0b0 2024-05-09T06:35:43.213 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.230+0000 7f97ae7fc640 1 -- 172.21.15.190:0/1824273330 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f97b000cd30 con 0x7f97b80ae0b0 2024-05-09T06:35:43.214 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.230+0000 7f97bfde7640 1 -- 172.21.15.190:0/1824273330 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97b80b5360 con 0x7f97b80ae0b0 2024-05-09T06:35:43.214 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.230+0000 7f97ae7fc640 1 -- 172.21.15.190:0/1824273330 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97b0003bb0 con 0x7f97b80ae0b0 2024-05-09T06:35:43.214 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.230+0000 7f97bfde7640 1 -- 172.21.15.190:0/1824273330 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97b80b5920 con 0x7f97b80ae0b0 2024-05-09T06:35:43.214 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.230+0000 7f97ae7fc640 1 -- 172.21.15.190:0/1824273330 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f97b0005d10 con 0x7f97b80ae0b0 2024-05-09T06:35:43.214 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.230+0000 7f97ae7fc640 1 -- 172.21.15.190:0/1824273330 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 725+0+0 (secure 0 0 0) 0x7f97b001d880 con 0x7f97b80ae0b0 2024-05-09T06:35:43.214 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.231+0000 7f97bfde7640 1 -- 172.21.15.190:0/1824273330 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97b813e900 con 0x7f97b80ae0b0 2024-05-09T06:35:43.214 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.233+0000 7f97ae7fc640 1 -- 172.21.15.190:0/1824273330 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f97b813e900 con 0x7f97b80ae0b0 2024-05-09T06:35:43.285 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.304+0000 7f97bfde7640 1 -- 172.21.15.190:0/1824273330 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "osd new", "uuid": "fae3a8db-4826-4f3c-b33a-12c09bb5df6a", "id": 0} v 0) -- 0x7f97b80682e0 con 0x7f97b80ae0b0 2024-05-09T06:35:43.291 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.310+0000 7f97ae7fc640 1 -- 172.21.15.190:0/1824273330 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "fae3a8db-4826-4f3c-b33a-12c09bb5df6a", "id": 0}]=0 v3) ==== 112+0+2 (secure 0 0 0) 0x7f97b0022070 con 0x7f97b80ae0b0 2024-05-09T06:35:43.291 INFO:teuthology.orchestra.run.smithi190.stdout:0 2024-05-09T06:35:43.292 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.311+0000 7f97bfde7640 1 -- 172.21.15.190:0/1824273330 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f97b80ae0b0 msgr2=0x7f97b80b02e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:43.292 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.311+0000 7f97bfde7640 1 --2- 172.21.15.190:0/1824273330 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f97b80ae0b0 0x7f97b80b02e0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f97b000efc0 tx=0x7f97b000c490 comp rx=0 tx=0).stop 2024-05-09T06:35:43.293 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.311+0000 7f97bfde7640 1 -- 172.21.15.190:0/1824273330 shutdown_connections 2024-05-09T06:35:43.293 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.311+0000 7f97bfde7640 1 --2- 172.21.15.190:0/1824273330 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f97b80ae0b0 0x7f97b80b02e0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:43.293 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.311+0000 7f97bfde7640 1 --2- 172.21.15.190:0/1824273330 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f97b813f0a0 0x7f97b80afda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:43.293 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.311+0000 7f97bfde7640 1 --2- 172.21.15.190:0/1824273330 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f97b806b790 0x7f97b815e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:43.293 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.311+0000 7f97bfde7640 1 -- 172.21.15.190:0/1824273330 >> 172.21.15.190:0/1824273330 conn(0x7f97b80b2c30 msgr2=0x7f97b815ea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:35:43.293 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.311+0000 7f97bfde7640 1 -- 172.21.15.190:0/1824273330 shutdown_connections 2024-05-09T06:35:43.293 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.312+0000 7f97bfde7640 1 -- 172.21.15.190:0/1824273330 wait complete. 2024-05-09T06:35:43.303 DEBUG:teuthology.orchestra.run.smithi190:> sudo ceph --cluster ceph osd new 646e882d-4ce4-4720-8c84-f71df563a35e 1 2024-05-09T06:35:43.444 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.463+0000 7f20d89b7640 1 Processor -- start 2024-05-09T06:35:43.444 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.463+0000 7f20d89b7640 1 -- start start 2024-05-09T06:35:43.445 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.464+0000 7f20d89b7640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f20d4147680 0x7f20d406e600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:43.446 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.464+0000 7f20d89b7640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f20d406eb40 0x7f20d40b5110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:43.446 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.464+0000 7f20d89b7640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f20d40b5650 0x7f20d40b2ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:43.446 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.464+0000 7f20d89b7640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f20d415f500 con 0x7f20d4147680 2024-05-09T06:35:43.446 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.464+0000 7f20d89b7640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f20d415ebd0 con 0x7f20d40b5650 2024-05-09T06:35:43.446 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.464+0000 7f20d89b7640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f20d415eeb0 con 0x7f20d406eb40 2024-05-09T06:35:43.446 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.464+0000 7f20d2d9d640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f20d40b5650 0x7f20d40b2ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:43.446 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.464+0000 7f20d1d9b640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f20d406eb40 0x7f20d40b5110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:43.446 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.464+0000 7f20d2d9d640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f20d40b5650 0x7f20d40b2ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.190:36282/0 (socket says 172.21.15.190:36282) 2024-05-09T06:35:43.446 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.464+0000 7f20d1d9b640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f20d406eb40 0x7f20d40b5110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.190:44520/0 (socket says 172.21.15.190:44520) 2024-05-09T06:35:43.446 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.464+0000 7f20d2d9d640 1 -- 172.21.15.190:0/3565019911 learned_addr learned my addr 172.21.15.190:0/3565019911 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-05-09T06:35:43.446 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.464+0000 7f20d259c640 1 --2- 172.21.15.190:0/3565019911 >> v2:172.21.15.3:3300/0 conn(0x7f20d4147680 0x7f20d406e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:43.446 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.465+0000 7f20d1d9b640 1 -- 172.21.15.190:0/3565019911 >> v2:172.21.15.190:3300/0 conn(0x7f20d40b5650 msgr2=0x7f20d40b2ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:35:43.446 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.465+0000 7f20d1d9b640 1 --2- 172.21.15.190:0/3565019911 >> v2:172.21.15.190:3300/0 conn(0x7f20d40b5650 0x7f20d40b2ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:43.447 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.465+0000 7f20d1d9b640 1 -- 172.21.15.190:0/3565019911 >> v2:172.21.15.3:3300/0 conn(0x7f20d4147680 msgr2=0x7f20d406e600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:35:43.447 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.465+0000 7f20d1d9b640 1 --2- 172.21.15.190:0/3565019911 >> v2:172.21.15.3:3300/0 conn(0x7f20d4147680 0x7f20d406e600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:43.447 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.465+0000 7f20d1d9b640 1 -- 172.21.15.190:0/3565019911 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20d415f190 con 0x7f20d406eb40 2024-05-09T06:35:43.447 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.465+0000 7f20d2d9d640 1 --2- 172.21.15.190:0/3565019911 >> v2:172.21.15.190:3300/0 conn(0x7f20d40b5650 0x7f20d40b2ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T06:35:43.448 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.465+0000 7f20d1d9b640 1 --2- 172.21.15.190:0/3565019911 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f20d406eb40 0x7f20d40b5110 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f20d406f3b0 tx=0x7f20c802a080 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5c5b784180fbacc4 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:35:43.448 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.465+0000 7f20d159a640 1 -- 172.21.15.190:0/3565019911 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20c8037070 con 0x7f20d406eb40 2024-05-09T06:35:43.448 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.466+0000 7f20d159a640 1 -- 172.21.15.190:0/3565019911 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f20c802add0 con 0x7f20d406eb40 2024-05-09T06:35:43.448 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.466+0000 7f20d159a640 1 -- 172.21.15.190:0/3565019911 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20c8032660 con 0x7f20d406eb40 2024-05-09T06:35:43.448 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.466+0000 7f20d89b7640 1 -- 172.21.15.190:0/3565019911 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f20d406eb40 msgr2=0x7f20d40b5110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:43.448 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.466+0000 7f20d89b7640 1 --2- 172.21.15.190:0/3565019911 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f20d406eb40 0x7f20d40b5110 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f20d406f3b0 tx=0x7f20c802a080 comp rx=0 tx=0).stop 2024-05-09T06:35:43.448 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.466+0000 7f20d89b7640 1 -- 172.21.15.190:0/3565019911 shutdown_connections 2024-05-09T06:35:43.449 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.466+0000 7f20d89b7640 1 --2- 172.21.15.190:0/3565019911 >> v2:172.21.15.190:3300/0 conn(0x7f20d40b5650 0x7f20d40b2ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:43.449 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.466+0000 7f20d89b7640 1 --2- 172.21.15.190:0/3565019911 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f20d406eb40 0x7f20d40b5110 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:43.449 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.466+0000 7f20d89b7640 1 --2- 172.21.15.190:0/3565019911 >> v2:172.21.15.3:3300/0 conn(0x7f20d4147680 0x7f20d406e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:43.449 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.466+0000 7f20d89b7640 1 -- 172.21.15.190:0/3565019911 >> 172.21.15.190:0/3565019911 conn(0x7f20d40b6530 msgr2=0x7f20d40b6910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:35:43.449 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.466+0000 7f20d89b7640 1 -- 172.21.15.190:0/3565019911 shutdown_connections 2024-05-09T06:35:43.449 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.466+0000 7f20d89b7640 1 -- 172.21.15.190:0/3565019911 wait complete. 2024-05-09T06:35:43.449 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.468+0000 7f20d89b7640 1 Processor -- start 2024-05-09T06:35:43.449 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.469+0000 7f20d89b7640 1 -- start start 2024-05-09T06:35:43.452 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.470+0000 7f20d89b7640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f20d406eb40 0x7f20d415d910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:43.452 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.470+0000 7f20d89b7640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f20d40b5650 0x7f20d413f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:43.452 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.470+0000 7f20d89b7640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f20d4147680 0x7f20d413f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:43.452 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.470+0000 7f20d89b7640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f20d40b5c60 con 0x7f20d4147680 2024-05-09T06:35:43.452 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.470+0000 7f20d89b7640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f20d40b5ae0 con 0x7f20d406eb40 2024-05-09T06:35:43.452 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.470+0000 7f20d89b7640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f20d40b31e0 con 0x7f20d40b5650 2024-05-09T06:35:43.452 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.470+0000 7f20d259c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f20d406eb40 0x7f20d415d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:43.452 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.470+0000 7f20d1d9b640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f20d40b5650 0x7f20d413f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:43.452 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.470+0000 7f20d1d9b640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f20d40b5650 0x7f20d413f1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.190:44522/0 (socket says 172.21.15.190:44522) 2024-05-09T06:35:43.453 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.470+0000 7f20d259c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f20d406eb40 0x7f20d415d910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.190:36284/0 (socket says 172.21.15.190:36284) 2024-05-09T06:35:43.453 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.470+0000 7f20d1d9b640 1 -- 172.21.15.190:0/415809586 learned_addr learned my addr 172.21.15.190:0/415809586 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-05-09T06:35:43.453 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.470+0000 7f20d2d9d640 1 --2- 172.21.15.190:0/415809586 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f20d4147680 0x7f20d413f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:43.453 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.470+0000 7f20d259c640 1 -- 172.21.15.190:0/415809586 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f20d40b5650 msgr2=0x7f20d413f1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:43.453 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.470+0000 7f20d259c640 1 --2- 172.21.15.190:0/415809586 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f20d40b5650 0x7f20d413f1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:43.453 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.470+0000 7f20d259c640 1 -- 172.21.15.190:0/415809586 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f20d4147680 msgr2=0x7f20d413f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:43.453 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.470+0000 7f20d259c640 1 --2- 172.21.15.190:0/415809586 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f20d4147680 0x7f20d413f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:43.453 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.470+0000 7f20d259c640 1 -- 172.21.15.190:0/415809586 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20d40af1a0 con 0x7f20d406eb40 2024-05-09T06:35:43.453 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.470+0000 7f20d259c640 1 --2- 172.21.15.190:0/415809586 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f20d406eb40 0x7f20d415d910 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f20c400e870 tx=0x7f20c400ed30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:35:43.453 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.470+0000 7f20c2ffd640 1 -- 172.21.15.190:0/415809586 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20c4017070 con 0x7f20d406eb40 2024-05-09T06:35:43.453 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.470+0000 7f20c2ffd640 1 -- 172.21.15.190:0/415809586 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f20c400ee60 con 0x7f20d406eb40 2024-05-09T06:35:43.453 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.470+0000 7f20d89b7640 1 -- 172.21.15.190:0/415809586 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20d40af480 con 0x7f20d406eb40 2024-05-09T06:35:43.453 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.470+0000 7f20c2ffd640 1 -- 172.21.15.190:0/415809586 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20c400cb60 con 0x7f20d406eb40 2024-05-09T06:35:43.453 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.471+0000 7f20d89b7640 1 -- 172.21.15.190:0/415809586 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20d40afa40 con 0x7f20d406eb40 2024-05-09T06:35:43.454 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.471+0000 7f20c2ffd640 1 -- 172.21.15.190:0/415809586 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f20c401a070 con 0x7f20d406eb40 2024-05-09T06:35:43.454 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.471+0000 7f20c2ffd640 1 -- 172.21.15.190:0/415809586 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 840+0+0 (secure 0 0 0) 0x7f20c401c800 con 0x7f20d406eb40 2024-05-09T06:35:43.454 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.471+0000 7f20d89b7640 1 -- 172.21.15.190:0/415809586 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20d40682e0 con 0x7f20d406eb40 2024-05-09T06:35:43.454 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.473+0000 7f20c2ffd640 1 -- 172.21.15.190:0/415809586 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f20c400cd00 con 0x7f20d406eb40 2024-05-09T06:35:43.527 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.546+0000 7f20d89b7640 1 -- 172.21.15.190:0/415809586 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "osd new", "uuid": "646e882d-4ce4-4720-8c84-f71df563a35e", "id": 1} v 0) -- 0x7f20d406e770 con 0x7f20d406eb40 2024-05-09T06:35:43.535 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.554+0000 7f20c2ffd640 1 -- 172.21.15.190:0/415809586 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "646e882d-4ce4-4720-8c84-f71df563a35e", "id": 1}]=0 v4) ==== 112+0+2 (secure 0 0 0) 0x7f20c401ca50 con 0x7f20d406eb40 2024-05-09T06:35:43.535 INFO:teuthology.orchestra.run.smithi190.stdout:1 2024-05-09T06:35:43.536 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.555+0000 7f20d89b7640 1 -- 172.21.15.190:0/415809586 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f20d406eb40 msgr2=0x7f20d415d910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:43.536 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.555+0000 7f20d89b7640 1 --2- 172.21.15.190:0/415809586 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f20d406eb40 0x7f20d415d910 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f20c400e870 tx=0x7f20c400ed30 comp rx=0 tx=0).stop 2024-05-09T06:35:43.536 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.555+0000 7f20d89b7640 1 -- 172.21.15.190:0/415809586 shutdown_connections 2024-05-09T06:35:43.536 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.555+0000 7f20d89b7640 1 --2- 172.21.15.190:0/415809586 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f20d4147680 0x7f20d413f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:43.536 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.555+0000 7f20d89b7640 1 --2- 172.21.15.190:0/415809586 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f20d40b5650 0x7f20d413f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:43.536 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.555+0000 7f20d89b7640 1 --2- 172.21.15.190:0/415809586 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f20d406eb40 0x7f20d415d910 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:43.537 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.555+0000 7f20d89b7640 1 -- 172.21.15.190:0/415809586 >> 172.21.15.190:0/415809586 conn(0x7f20d40b6530 msgr2=0x7f20d4072570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:35:43.537 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.555+0000 7f20d89b7640 1 -- 172.21.15.190:0/415809586 shutdown_connections 2024-05-09T06:35:43.537 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.555+0000 7f20d89b7640 1 -- 172.21.15.190:0/415809586 wait complete. 2024-05-09T06:35:43.547 DEBUG:teuthology.orchestra.run.smithi190:> sudo ceph --cluster ceph osd new 1abbadb5-a570-4359-9643-acff3ede35a1 2 2024-05-09T06:35:43.646 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.665+0000 7f1e6bea5640 1 Processor -- start 2024-05-09T06:35:43.646 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.665+0000 7f1e6bea5640 1 -- start start 2024-05-09T06:35:43.647 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.666+0000 7f1e6bea5640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f1e640b1460 0x7f1e640b1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:43.648 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.666+0000 7f1e6bea5640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1e640d0970 0x7f1e640b2210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:43.648 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.666+0000 7f1e6bea5640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1e640cd690 con 0x7f1e6415c350 2024-05-09T06:35:43.648 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.666+0000 7f1e6bea5640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f1e640cdf30 con 0x7f1e640b1460 2024-05-09T06:35:43.648 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.666+0000 7f1e6bea5640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f1e640d0010 con 0x7f1e640d0970 2024-05-09T06:35:43.648 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.666+0000 7f1e6a442640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1e640d0970 0x7f1e640b2210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:43.648 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.666+0000 7f1e6a442640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1e640d0970 0x7f1e640b2210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.190:44524/0 (socket says 172.21.15.190:44524) 2024-05-09T06:35:43.648 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.666+0000 7f1e69c41640 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7f1e6415c350 0x7f1e640d0260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.190:51348/0 (socket says 172.21.15.190:51348) 2024-05-09T06:35:43.648 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.666+0000 7f1e6a442640 1 -- 172.21.15.190:0/736850099 learned_addr learned my addr 172.21.15.190:0/736850099 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-05-09T06:35:43.648 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.667+0000 7f1e69440640 1 --2- 172.21.15.190:0/736850099 >> v2:172.21.15.3:3300/0 conn(0x7f1e640b1460 0x7f1e640b1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:43.648 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.667+0000 7f1e68c3f640 1 -- 172.21.15.190:0/736850099 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2115170383 0 0) 0x7f1e640cd690 con 0x7f1e6415c350 2024-05-09T06:35:43.648 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.667+0000 7f1e6a442640 1 -- 172.21.15.190:0/736850099 >> v2:172.21.15.3:3300/0 conn(0x7f1e640b1460 msgr2=0x7f1e640b1cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:35:43.648 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.667+0000 7f1e6a442640 1 --2- 172.21.15.190:0/736850099 >> v2:172.21.15.3:3300/0 conn(0x7f1e640b1460 0x7f1e640b1cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:43.648 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.667+0000 7f1e6a442640 1 -- 172.21.15.190:0/736850099 >> v1:172.21.15.190:6789/0 conn(0x7f1e6415c350 legacy=0x7f1e640d0260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:43.648 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.667+0000 7f1e6a442640 1 -- 172.21.15.190:0/736850099 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e640cd970 con 0x7f1e640d0970 2024-05-09T06:35:43.650 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.667+0000 7f1e6a442640 1 --2- 172.21.15.190:0/736850099 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1e640d0970 0x7f1e640b2210 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f1e5c009e50 tx=0x7f1e5c02a710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a7fef002848561bc server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:35:43.650 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.668+0000 7f1e68c3f640 1 -- 172.21.15.190:0/736850099 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e5c036070 con 0x7f1e640d0970 2024-05-09T06:35:43.650 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.668+0000 7f1e68c3f640 1 -- 172.21.15.190:0/736850099 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e5c031600 con 0x7f1e640d0970 2024-05-09T06:35:43.650 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.668+0000 7f1e68c3f640 1 -- 172.21.15.190:0/736850099 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e5c0319d0 con 0x7f1e640d0970 2024-05-09T06:35:43.650 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.668+0000 7f1e6bea5640 1 -- 172.21.15.190:0/736850099 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1e640d0970 msgr2=0x7f1e640b2210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:43.650 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.668+0000 7f1e6bea5640 1 --2- 172.21.15.190:0/736850099 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1e640d0970 0x7f1e640b2210 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f1e5c009e50 tx=0x7f1e5c02a710 comp rx=0 tx=0).stop 2024-05-09T06:35:43.650 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.668+0000 7f1e6bea5640 1 -- 172.21.15.190:0/736850099 shutdown_connections 2024-05-09T06:35:43.650 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.668+0000 7f1e6bea5640 1 --2- 172.21.15.190:0/736850099 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1e640d0970 0x7f1e640b2210 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:43.650 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.668+0000 7f1e6bea5640 1 --2- 172.21.15.190:0/736850099 >> v2:172.21.15.3:3300/0 conn(0x7f1e640b1460 0x7f1e640b1cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:43.651 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.668+0000 7f1e6bea5640 1 -- 172.21.15.190:0/736850099 >> 172.21.15.190:0/736850099 conn(0x7f1e640b3ac0 msgr2=0x7f1e640b3ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:35:43.651 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.668+0000 7f1e6bea5640 1 -- 172.21.15.190:0/736850099 shutdown_connections 2024-05-09T06:35:43.651 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.668+0000 7f1e6bea5640 1 -- 172.21.15.190:0/736850099 wait complete. 2024-05-09T06:35:43.653 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.669+0000 7f1e6bea5640 1 Processor -- start 2024-05-09T06:35:43.653 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.669+0000 7f1e6bea5640 1 -- start start 2024-05-09T06:35:43.653 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.670+0000 7f1e6bea5640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1e640b1460 0x7f1e641f5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:43.654 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.670+0000 7f1e6bea5640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1e640d0970 0x7f1e640c64e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:43.654 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.670+0000 7f1e6bea5640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1e6415c350 0x7f1e640c6a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:43.654 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.670+0000 7f1e6bea5640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f1e640cfba0 con 0x7f1e640d0970 2024-05-09T06:35:43.654 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.670+0000 7f1e6bea5640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f1e6415c730 con 0x7f1e640b1460 2024-05-09T06:35:43.654 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.670+0000 7f1e6bea5640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f1e640cfd20 con 0x7f1e6415c350 2024-05-09T06:35:43.654 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.670+0000 7f1e69c41640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1e640b1460 0x7f1e641f5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:43.654 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.670+0000 7f1e6a442640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1e6415c350 0x7f1e640c6a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:43.654 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.670+0000 7f1e69c41640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1e640b1460 0x7f1e641f5bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.190:36298/0 (socket says 172.21.15.190:36298) 2024-05-09T06:35:43.654 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.670+0000 7f1e69c41640 1 -- 172.21.15.190:0/4146157846 learned_addr learned my addr 172.21.15.190:0/4146157846 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-05-09T06:35:43.654 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.670+0000 7f1e69440640 1 --2- 172.21.15.190:0/4146157846 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1e640d0970 0x7f1e640c64e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:43.654 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.670+0000 7f1e6a442640 1 -- 172.21.15.190:0/4146157846 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1e640b1460 msgr2=0x7f1e641f5bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:43.654 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.670+0000 7f1e6a442640 1 --2- 172.21.15.190:0/4146157846 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1e640b1460 0x7f1e641f5bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:43.654 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.670+0000 7f1e6a442640 1 -- 172.21.15.190:0/4146157846 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1e640d0970 msgr2=0x7f1e640c64e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:43.654 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.670+0000 7f1e6a442640 1 --2- 172.21.15.190:0/4146157846 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1e640d0970 0x7f1e640c64e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:43.655 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.671+0000 7f1e6a442640 1 -- 172.21.15.190:0/4146157846 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e640c7260 con 0x7f1e6415c350 2024-05-09T06:35:43.655 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.671+0000 7f1e69c41640 1 --2- 172.21.15.190:0/4146157846 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1e640b1460 0x7f1e641f5bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T06:35:43.655 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.671+0000 7f1e6a442640 1 --2- 172.21.15.190:0/4146157846 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1e6415c350 0x7f1e640c6a20 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f1e5c030000 tx=0x7f1e5c004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:35:43.655 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.671+0000 7f1e567fc640 1 -- 172.21.15.190:0/4146157846 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e5c036050 con 0x7f1e6415c350 2024-05-09T06:35:43.655 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.671+0000 7f1e567fc640 1 -- 172.21.15.190:0/4146157846 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e5c042070 con 0x7f1e6415c350 2024-05-09T06:35:43.655 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.671+0000 7f1e567fc640 1 -- 172.21.15.190:0/4146157846 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e5c009510 con 0x7f1e6415c350 2024-05-09T06:35:43.655 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.671+0000 7f1e6bea5640 1 -- 172.21.15.190:0/4146157846 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e641f3680 con 0x7f1e6415c350 2024-05-09T06:35:43.655 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.671+0000 7f1e6bea5640 1 -- 172.21.15.190:0/4146157846 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e641f3c40 con 0x7f1e6415c350 2024-05-09T06:35:43.655 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.671+0000 7f1e567fc640 1 -- 172.21.15.190:0/4146157846 <== mon.2 v2:172.21.15.190:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f1e5c007c10 con 0x7f1e6415c350 2024-05-09T06:35:43.655 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.671+0000 7f1e567fc640 1 -- 172.21.15.190:0/4146157846 <== mon.2 v2:172.21.15.190:3301/0 5 ==== osd_map(4..4 src has 1..4) ==== 955+0+0 (secure 0 0 0) 0x7f1e5c03d450 con 0x7f1e6415c350 2024-05-09T06:35:43.655 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.672+0000 7f1e6bea5640 1 -- 172.21.15.190:0/4146157846 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e641f3480 con 0x7f1e6415c350 2024-05-09T06:35:43.656 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.674+0000 7f1e567fc640 1 -- 172.21.15.190:0/4146157846 <== mon.2 v2:172.21.15.190:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f1e5c047460 con 0x7f1e6415c350 2024-05-09T06:35:43.728 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.748+0000 7f1e6bea5640 1 -- 172.21.15.190:0/4146157846 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "osd new", "uuid": "1abbadb5-a570-4359-9643-acff3ede35a1", "id": 2} v 0) -- 0x7f1e640682e0 con 0x7f1e6415c350 2024-05-09T06:35:43.731 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.751+0000 7f1e567fc640 1 -- 172.21.15.190:0/4146157846 <== mon.2 v2:172.21.15.190:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "1abbadb5-a570-4359-9643-acff3ede35a1", "id": 2}]=0 v5) ==== 112+0+2 (secure 0 0 0) 0x7f1e5c047a50 con 0x7f1e6415c350 2024-05-09T06:35:43.732 INFO:teuthology.orchestra.run.smithi190.stdout:2 2024-05-09T06:35:43.733 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.752+0000 7f1e6bea5640 1 -- 172.21.15.190:0/4146157846 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1e6415c350 msgr2=0x7f1e640c6a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:43.733 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.752+0000 7f1e6bea5640 1 --2- 172.21.15.190:0/4146157846 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1e6415c350 0x7f1e640c6a20 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f1e5c030000 tx=0x7f1e5c004a40 comp rx=0 tx=0).stop 2024-05-09T06:35:43.733 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.752+0000 7f1e6bea5640 1 -- 172.21.15.190:0/4146157846 shutdown_connections 2024-05-09T06:35:43.733 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.752+0000 7f1e6bea5640 1 --2- 172.21.15.190:0/4146157846 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1e6415c350 0x7f1e640c6a20 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:43.733 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.752+0000 7f1e6bea5640 1 --2- 172.21.15.190:0/4146157846 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1e640d0970 0x7f1e640c64e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:43.733 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.752+0000 7f1e6bea5640 1 --2- 172.21.15.190:0/4146157846 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1e640b1460 0x7f1e641f5bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:43.733 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.752+0000 7f1e6bea5640 1 -- 172.21.15.190:0/4146157846 >> 172.21.15.190:0/4146157846 conn(0x7f1e640b3ac0 msgr2=0x7f1e6409aa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:35:43.733 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.752+0000 7f1e6bea5640 1 -- 172.21.15.190:0/4146157846 shutdown_connections 2024-05-09T06:35:43.733 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.752+0000 7f1e6bea5640 1 -- 172.21.15.190:0/4146157846 wait complete. 2024-05-09T06:35:43.743 DEBUG:teuthology.orchestra.run.smithi190:> sudo ceph --cluster ceph osd new 62ac4dc1-2371-4dd2-acd3-e2dbb11aedf5 3 2024-05-09T06:35:43.843 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.862+0000 7f629c664640 1 Processor -- start 2024-05-09T06:35:43.843 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.862+0000 7f629c664640 1 -- start start 2024-05-09T06:35:43.845 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.863+0000 7f629c664640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6294071ea0 0x7f629415fd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:43.845 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.863+0000 7f629c664640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f6294072280 0x7f6294160270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:43.845 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.863+0000 7f629c664640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f62940ceeb0 0x7f62940cf300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:43.845 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.863+0000 7f629c664640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f62941607b0 con 0x7f62940ceeb0 2024-05-09T06:35:43.845 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.863+0000 7f629c664640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f629406b2f0 con 0x7f6294072280 2024-05-09T06:35:43.845 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.863+0000 7f629c664640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f629406b5d0 con 0x7f6294071ea0 2024-05-09T06:35:43.845 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.863+0000 7f6299bff640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f6294072280 0x7f6294160270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:43.845 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.863+0000 7f629a400640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6294071ea0 0x7f629415fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:43.845 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.863+0000 7f6299bff640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f6294072280 0x7f6294160270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.190:36304/0 (socket says 172.21.15.190:36304) 2024-05-09T06:35:43.845 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.863+0000 7f6299bff640 1 -- 172.21.15.190:0/1370431121 learned_addr learned my addr 172.21.15.190:0/1370431121 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-05-09T06:35:43.845 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.863+0000 7f629ac01640 1 --2- 172.21.15.190:0/1370431121 >> v2:172.21.15.3:3300/0 conn(0x7f62940ceeb0 0x7f62940cf300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:43.845 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.863+0000 7f629a400640 1 -- 172.21.15.190:0/1370431121 >> v2:172.21.15.190:3300/0 conn(0x7f6294072280 msgr2=0x7f6294160270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:35:43.845 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.864+0000 7f629a400640 1 --2- 172.21.15.190:0/1370431121 >> v2:172.21.15.190:3300/0 conn(0x7f6294072280 0x7f6294160270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:43.845 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.864+0000 7f629a400640 1 -- 172.21.15.190:0/1370431121 >> v2:172.21.15.3:3300/0 conn(0x7f62940ceeb0 msgr2=0x7f62940cf300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:35:43.846 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.864+0000 7f629a400640 1 --2- 172.21.15.190:0/1370431121 >> v2:172.21.15.3:3300/0 conn(0x7f62940ceeb0 0x7f62940cf300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:43.846 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.864+0000 7f629a400640 1 -- 172.21.15.190:0/1370431121 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f629406b8b0 con 0x7f6294071ea0 2024-05-09T06:35:43.846 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.864+0000 7f6299bff640 1 --2- 172.21.15.190:0/1370431121 >> v2:172.21.15.190:3300/0 conn(0x7f6294072280 0x7f6294160270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T06:35:43.846 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.864+0000 7f629ac01640 1 --2- 172.21.15.190:0/1370431121 >> v2:172.21.15.3:3300/0 conn(0x7f62940ceeb0 0x7f62940cf300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:35:43.846 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.864+0000 7f629a400640 1 --2- 172.21.15.190:0/1370431121 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6294071ea0 0x7f629415fd30 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f62940d16c0 tx=0x7f628c02a5d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a626df266b6fec40 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:35:43.847 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.864+0000 7f62993fe640 1 -- 172.21.15.190:0/1370431121 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f628c031600 con 0x7f6294071ea0 2024-05-09T06:35:43.847 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.864+0000 7f62993fe640 1 -- 172.21.15.190:0/1370431121 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f628c0317a0 con 0x7f6294071ea0 2024-05-09T06:35:43.847 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.864+0000 7f62993fe640 1 -- 172.21.15.190:0/1370431121 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f628c031b70 con 0x7f6294071ea0 2024-05-09T06:35:43.847 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.865+0000 7f629c664640 1 -- 172.21.15.190:0/1370431121 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6294071ea0 msgr2=0x7f629415fd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:43.847 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.865+0000 7f629c664640 1 --2- 172.21.15.190:0/1370431121 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6294071ea0 0x7f629415fd30 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f62940d16c0 tx=0x7f628c02a5d0 comp rx=0 tx=0).stop 2024-05-09T06:35:43.847 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.865+0000 7f629c664640 1 -- 172.21.15.190:0/1370431121 shutdown_connections 2024-05-09T06:35:43.847 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.865+0000 7f629c664640 1 --2- 172.21.15.190:0/1370431121 >> v2:172.21.15.3:3300/0 conn(0x7f62940ceeb0 0x7f62940cf300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:43.847 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.865+0000 7f629c664640 1 --2- 172.21.15.190:0/1370431121 >> v2:172.21.15.190:3300/0 conn(0x7f6294072280 0x7f6294160270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:43.847 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.865+0000 7f629c664640 1 --2- 172.21.15.190:0/1370431121 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6294071ea0 0x7f629415fd30 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:43.848 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.865+0000 7f629c664640 1 -- 172.21.15.190:0/1370431121 >> 172.21.15.190:0/1370431121 conn(0x7f62940b2660 msgr2=0x7f62940b2a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:35:43.848 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.865+0000 7f629c664640 1 -- 172.21.15.190:0/1370431121 shutdown_connections 2024-05-09T06:35:43.848 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.865+0000 7f629c664640 1 -- 172.21.15.190:0/1370431121 wait complete. 2024-05-09T06:35:43.848 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.867+0000 7f629c664640 1 Processor -- start 2024-05-09T06:35:43.848 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.867+0000 7f629c664640 1 -- start start 2024-05-09T06:35:43.850 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.868+0000 7f629c664640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6294071ea0 0x7f629414b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:43.850 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.868+0000 7f629c664640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6294072280 0x7f62940a9490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:43.851 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.868+0000 7f629c664640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f62940ceeb0 0x7f6294157d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:43.851 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.868+0000 7f629c664640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f6294160a60 con 0x7f6294072280 2024-05-09T06:35:43.851 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.868+0000 7f629c664640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f629406c0b0 con 0x7f62940ceeb0 2024-05-09T06:35:43.851 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.868+0000 7f629c664640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f62940cf840 con 0x7f6294071ea0 2024-05-09T06:35:43.851 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.868+0000 7f629a400640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6294071ea0 0x7f629414b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:43.851 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.868+0000 7f629a400640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6294071ea0 0x7f629414b7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.190:44562/0 (socket says 172.21.15.190:44562) 2024-05-09T06:35:43.851 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.868+0000 7f629a400640 1 -- 172.21.15.190:0/420205932 learned_addr learned my addr 172.21.15.190:0/420205932 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-05-09T06:35:43.851 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.868+0000 7f629ac01640 1 --2- 172.21.15.190:0/420205932 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f62940ceeb0 0x7f6294157d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:43.851 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.868+0000 7f6299bff640 1 --2- 172.21.15.190:0/420205932 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6294072280 0x7f62940a9490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:43.851 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.868+0000 7f629ac01640 1 -- 172.21.15.190:0/420205932 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6294071ea0 msgr2=0x7f629414b7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:43.851 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.868+0000 7f629ac01640 1 --2- 172.21.15.190:0/420205932 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6294071ea0 0x7f629414b7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:43.851 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.868+0000 7f629ac01640 1 -- 172.21.15.190:0/420205932 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6294072280 msgr2=0x7f62940a9490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:43.851 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.868+0000 7f629ac01640 1 --2- 172.21.15.190:0/420205932 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6294072280 0x7f62940a9490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:43.852 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.868+0000 7f629ac01640 1 -- 172.21.15.190:0/420205932 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f629415e860 con 0x7f62940ceeb0 2024-05-09T06:35:43.852 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.868+0000 7f629a400640 1 --2- 172.21.15.190:0/420205932 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6294071ea0 0x7f629414b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T06:35:43.852 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.869+0000 7f629ac01640 1 --2- 172.21.15.190:0/420205932 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f62940ceeb0 0x7f6294157d00 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f628400c5f0 tx=0x7f628400cab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:35:43.852 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.869+0000 7f6292ffd640 1 -- 172.21.15.190:0/420205932 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6284015070 con 0x7f62940ceeb0 2024-05-09T06:35:43.852 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.869+0000 7f6292ffd640 1 -- 172.21.15.190:0/420205932 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6284011820 con 0x7f62940ceeb0 2024-05-09T06:35:43.852 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.869+0000 7f629c664640 1 -- 172.21.15.190:0/420205932 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f629415eb40 con 0x7f62940ceeb0 2024-05-09T06:35:43.852 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.869+0000 7f6292ffd640 1 -- 172.21.15.190:0/420205932 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6284011c10 con 0x7f62940ceeb0 2024-05-09T06:35:43.852 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.869+0000 7f629c664640 1 -- 172.21.15.190:0/420205932 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62940a99d0 con 0x7f62940ceeb0 2024-05-09T06:35:43.852 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.869+0000 7f6292ffd640 1 -- 172.21.15.190:0/420205932 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f6284018070 con 0x7f62940ceeb0 2024-05-09T06:35:43.852 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.869+0000 7f6292ffd640 1 -- 172.21.15.190:0/420205932 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1070+0+0 (secure 0 0 0) 0x7f6284022830 con 0x7f62940ceeb0 2024-05-09T06:35:43.852 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.870+0000 7f629c664640 1 -- 172.21.15.190:0/420205932 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62940682e0 con 0x7f62940ceeb0 2024-05-09T06:35:43.853 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.872+0000 7f6292ffd640 1 -- 172.21.15.190:0/420205932 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f62840223a0 con 0x7f62940ceeb0 2024-05-09T06:35:43.925 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.944+0000 7f629c664640 1 -- 172.21.15.190:0/420205932 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "osd new", "uuid": "62ac4dc1-2371-4dd2-acd3-e2dbb11aedf5", "id": 3} v 0) -- 0x7f629406c3a0 con 0x7f62940ceeb0 2024-05-09T06:35:43.930 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.948+0000 7f6292ffd640 1 -- 172.21.15.190:0/420205932 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "62ac4dc1-2371-4dd2-acd3-e2dbb11aedf5", "id": 3}]=0 v6) ==== 112+0+2 (secure 0 0 0) 0x7f6284022640 con 0x7f62940ceeb0 2024-05-09T06:35:43.931 INFO:teuthology.orchestra.run.smithi190.stdout:3 2024-05-09T06:35:43.932 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.949+0000 7f629c664640 1 -- 172.21.15.190:0/420205932 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f62940ceeb0 msgr2=0x7f6294157d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:43.932 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.949+0000 7f629c664640 1 --2- 172.21.15.190:0/420205932 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f62940ceeb0 0x7f6294157d00 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f628400c5f0 tx=0x7f628400cab0 comp rx=0 tx=0).stop 2024-05-09T06:35:43.932 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.949+0000 7f629c664640 1 -- 172.21.15.190:0/420205932 shutdown_connections 2024-05-09T06:35:43.932 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.949+0000 7f629c664640 1 --2- 172.21.15.190:0/420205932 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f62940ceeb0 0x7f6294157d00 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:43.932 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.949+0000 7f629c664640 1 --2- 172.21.15.190:0/420205932 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6294072280 0x7f62940a9490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:43.932 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.949+0000 7f629c664640 1 --2- 172.21.15.190:0/420205932 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6294071ea0 0x7f629414b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:43.932 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.949+0000 7f629c664640 1 -- 172.21.15.190:0/420205932 >> 172.21.15.190:0/420205932 conn(0x7f62940b2660 msgr2=0x7f629415a490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:35:43.932 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.949+0000 7f629c664640 1 -- 172.21.15.190:0/420205932 shutdown_connections 2024-05-09T06:35:43.932 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:43.949+0000 7f629c664640 1 -- 172.21.15.190:0/420205932 wait complete. 2024-05-09T06:35:43.935 INFO:tasks.ceph.mgr.x.smithi190.stderr:2024-05-09T06:35:43.955+0000 7fab08831040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-09T06:35:43.940 DEBUG:teuthology.orchestra.run.smithi190:> sudo ceph --cluster ceph osd new ae808398-8326-4d7c-9319-a2a0eafffb59 4 2024-05-09T06:35:43.956 INFO:tasks.ceph.mgr.y.smithi003.stderr:2024-05-09T06:35:43.975+0000 7f733de11040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-09T06:35:44.028 INFO:tasks.ceph.mgr.x.smithi190.stderr:2024-05-09T06:35:44.047+0000 7fab08831040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-09T06:35:44.040 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.060+0000 7fc5d3c76640 1 Processor -- start 2024-05-09T06:35:44.041 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.060+0000 7fc5d3c76640 1 -- start start 2024-05-09T06:35:44.049 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.061+0000 7fc5d3c76640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc5cc06ed10 0x7fc5cc0b5110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:44.049 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.061+0000 7fc5d3c76640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc5cc15ebd0 con 0x7fc5cc0b5650 2024-05-09T06:35:44.049 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.061+0000 7fc5d3c76640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc5cc15eeb0 con 0x7fc5cc147680 2024-05-09T06:35:44.050 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.061+0000 7fc5d3c76640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fc5cc15f500 con 0x7fc5cc06ed10 2024-05-09T06:35:44.050 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.061+0000 7fc5d1a12640 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7fc5cc147680 0x7fc5cc06e600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.190:51360/0 (socket says 172.21.15.190:51360) 2024-05-09T06:35:44.050 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.061+0000 7fc5d1a12640 1 -- 172.21.15.190:0/3407269025 learned_addr learned my addr 172.21.15.190:0/3407269025 (peer_addr_for_me v1:172.21.15.190:0/0) 2024-05-09T06:35:44.050 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.061+0000 7fc5d1211640 1 --2- 172.21.15.190:0/3407269025 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc5cc06ed10 0x7fc5cc0b5110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:44.050 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.061+0000 7fc5d0a10640 1 -- 172.21.15.190:0/3407269025 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2448402008 0 0) 0x7fc5cc15eeb0 con 0x7fc5cc147680 2024-05-09T06:35:44.051 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.061+0000 7fc5d0a10640 1 -- 172.21.15.190:0/3407269025 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc5b8003520 con 0x7fc5cc147680 2024-05-09T06:35:44.051 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.061+0000 7fc5d1211640 1 -- 172.21.15.190:0/3407269025 >> v1:172.21.15.190:6789/0 conn(0x7fc5cc147680 legacy=0x7fc5cc06e600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:44.051 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.061+0000 7fc5d1211640 1 -- 172.21.15.190:0/3407269025 >> v1:172.21.15.3:6789/0 conn(0x7fc5cc0b5650 legacy=0x7fc5cc072570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:44.051 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.061+0000 7fc5d1211640 1 -- 172.21.15.190:0/3407269025 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5cc15f190 con 0x7fc5cc06ed10 2024-05-09T06:35:44.051 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.062+0000 7fc5d1211640 1 --2- 172.21.15.190:0/3407269025 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc5cc06ed10 0x7fc5cc0b5110 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fc5bc0097d0 tx=0x7fc5bc02a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2836834284eda545 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:35:44.051 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.062+0000 7fc5d0a10640 1 -- 172.21.15.190:0/3407269025 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5bc037070 con 0x7fc5cc06ed10 2024-05-09T06:35:44.051 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.062+0000 7fc5d0a10640 1 -- 172.21.15.190:0/3407269025 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc5bc032440 con 0x7fc5cc06ed10 2024-05-09T06:35:44.052 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.062+0000 7fc5d0a10640 1 -- 172.21.15.190:0/3407269025 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5bc0327f0 con 0x7fc5cc06ed10 2024-05-09T06:35:44.052 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.063+0000 7fc5d3c76640 1 -- 172.21.15.190:0/3407269025 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc5cc06ed10 msgr2=0x7fc5cc0b5110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:44.052 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.063+0000 7fc5d3c76640 1 --2- 172.21.15.190:0/3407269025 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc5cc06ed10 0x7fc5cc0b5110 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fc5bc0097d0 tx=0x7fc5bc02a180 comp rx=0 tx=0).stop 2024-05-09T06:35:44.052 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.063+0000 7fc5d3c76640 1 -- 172.21.15.190:0/3407269025 shutdown_connections 2024-05-09T06:35:44.052 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.063+0000 7fc5d3c76640 1 --2- 172.21.15.190:0/3407269025 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc5cc06ed10 0x7fc5cc0b5110 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:44.052 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.063+0000 7fc5d3c76640 1 -- 172.21.15.190:0/3407269025 >> 172.21.15.190:0/3407269025 conn(0x7fc5cc0b6530 msgr2=0x7fc5cc0b6910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:35:44.052 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.063+0000 7fc5d3c76640 1 -- 172.21.15.190:0/3407269025 shutdown_connections 2024-05-09T06:35:44.052 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.063+0000 7fc5d3c76640 1 -- 172.21.15.190:0/3407269025 wait complete. 2024-05-09T06:35:44.052 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.064+0000 7fc5d3c76640 1 Processor -- start 2024-05-09T06:35:44.053 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.064+0000 7fc5d3c76640 1 -- start start 2024-05-09T06:35:44.053 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.065+0000 7fc5d3c76640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc5cc06ed10 0x7fc5cc143c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:44.053 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.065+0000 7fc5d3c76640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc5cc0b5650 0x7fc5cc15d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:44.053 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.065+0000 7fc5d3c76640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc5cc147680 0x7fc5cc15dc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:44.053 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.065+0000 7fc5d3c76640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fc5cc072c80 con 0x7fc5cc0b5650 2024-05-09T06:35:44.053 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.065+0000 7fc5d3c76640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fc5cc15e840 con 0x7fc5cc147680 2024-05-09T06:35:44.053 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.065+0000 7fc5d3c76640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fc5cc072e00 con 0x7fc5cc06ed10 2024-05-09T06:35:44.053 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.065+0000 7fc5d1a12640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc5cc06ed10 0x7fc5cc143c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:44.054 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.065+0000 7fc5d2213640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc5cc147680 0x7fc5cc15dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:44.054 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.065+0000 7fc5d1a12640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc5cc06ed10 0x7fc5cc143c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.190:44574/0 (socket says 172.21.15.190:44574) 2024-05-09T06:35:44.054 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.065+0000 7fc5d1a12640 1 -- 172.21.15.190:0/372514878 learned_addr learned my addr 172.21.15.190:0/372514878 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-05-09T06:35:44.054 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.065+0000 7fc5d1211640 1 --2- 172.21.15.190:0/372514878 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc5cc0b5650 0x7fc5cc15d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:44.054 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.065+0000 7fc5d2213640 1 -- 172.21.15.190:0/372514878 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc5cc06ed10 msgr2=0x7fc5cc143c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:44.054 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.065+0000 7fc5d2213640 1 --2- 172.21.15.190:0/372514878 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc5cc06ed10 0x7fc5cc143c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:44.054 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.065+0000 7fc5d2213640 1 -- 172.21.15.190:0/372514878 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc5cc0b5650 msgr2=0x7fc5cc15d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:44.054 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.065+0000 7fc5d2213640 1 --2- 172.21.15.190:0/372514878 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc5cc0b5650 0x7fc5cc15d730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:44.054 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.065+0000 7fc5d2213640 1 -- 172.21.15.190:0/372514878 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5cc1420c0 con 0x7fc5cc147680 2024-05-09T06:35:44.054 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.065+0000 7fc5d1a12640 1 --2- 172.21.15.190:0/372514878 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc5cc06ed10 0x7fc5cc143c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T06:35:44.055 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.065+0000 7fc5d2213640 1 --2- 172.21.15.190:0/372514878 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc5cc147680 0x7fc5cc15dc70 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fc5c400c690 tx=0x7fc5c400cb50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:35:44.055 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.065+0000 7fc5c27fc640 1 -- 172.21.15.190:0/372514878 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5c4003b70 con 0x7fc5cc147680 2024-05-09T06:35:44.055 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.066+0000 7fc5c27fc640 1 -- 172.21.15.190:0/372514878 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc5c4003d10 con 0x7fc5cc147680 2024-05-09T06:35:44.055 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.066+0000 7fc5d3c76640 1 -- 172.21.15.190:0/372514878 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5cc1423a0 con 0x7fc5cc147680 2024-05-09T06:35:44.055 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.066+0000 7fc5c27fc640 1 -- 172.21.15.190:0/372514878 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5c4005310 con 0x7fc5cc147680 2024-05-09T06:35:44.055 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.066+0000 7fc5d3c76640 1 -- 172.21.15.190:0/372514878 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5cc142960 con 0x7fc5cc147680 2024-05-09T06:35:44.055 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.066+0000 7fc5c27fc640 1 -- 172.21.15.190:0/372514878 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fc5c40054b0 con 0x7fc5cc147680 2024-05-09T06:35:44.055 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.066+0000 7fc5c27fc640 1 -- 172.21.15.190:0/372514878 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(6..6 src has 1..6) ==== 1185+0+0 (secure 0 0 0) 0x7fc5c40129c0 con 0x7fc5cc147680 2024-05-09T06:35:44.055 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.067+0000 7fc5d3c76640 1 -- 172.21.15.190:0/372514878 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5cc0682e0 con 0x7fc5cc147680 2024-05-09T06:35:44.056 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.070+0000 7fc5c27fc640 1 -- 172.21.15.190:0/372514878 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fc5c4012c60 con 0x7fc5cc147680 2024-05-09T06:35:44.056 INFO:tasks.ceph.mgr.y.smithi003.stderr:2024-05-09T06:35:44.066+0000 7f733de11040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-09T06:35:44.075 INFO:tasks.ceph.mgr.x.smithi190.stderr:2024-05-09T06:35:44.094+0000 7fab08831040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-09T06:35:44.092 INFO:tasks.ceph.mgr.y.smithi003.stderr:2024-05-09T06:35:44.111+0000 7f733de11040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-09T06:35:44.124 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.143+0000 7fc5d3c76640 1 -- 172.21.15.190:0/372514878 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "osd new", "uuid": "ae808398-8326-4d7c-9319-a2a0eafffb59", "id": 4} v 0) -- 0x7fc5cc0b6d20 con 0x7fc5cc147680 2024-05-09T06:35:44.127 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.147+0000 7fc5c27fc640 1 -- 172.21.15.190:0/372514878 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "ae808398-8326-4d7c-9319-a2a0eafffb59", "id": 4}]=0 v7) ==== 112+0+2 (secure 0 0 0) 0x7fc5c4012470 con 0x7fc5cc147680 2024-05-09T06:35:44.128 INFO:teuthology.orchestra.run.smithi190.stdout:4 2024-05-09T06:35:44.129 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.148+0000 7fc5d3c76640 1 -- 172.21.15.190:0/372514878 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc5cc147680 msgr2=0x7fc5cc15dc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:44.129 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.148+0000 7fc5d3c76640 1 --2- 172.21.15.190:0/372514878 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc5cc147680 0x7fc5cc15dc70 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fc5c400c690 tx=0x7fc5c400cb50 comp rx=0 tx=0).stop 2024-05-09T06:35:44.129 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.148+0000 7fc5d3c76640 1 -- 172.21.15.190:0/372514878 shutdown_connections 2024-05-09T06:35:44.129 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.148+0000 7fc5d3c76640 1 --2- 172.21.15.190:0/372514878 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc5cc147680 0x7fc5cc15dc70 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:44.129 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.148+0000 7fc5d3c76640 1 --2- 172.21.15.190:0/372514878 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc5cc0b5650 0x7fc5cc15d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:44.129 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.148+0000 7fc5d3c76640 1 --2- 172.21.15.190:0/372514878 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc5cc06ed10 0x7fc5cc143c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:44.129 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.148+0000 7fc5d3c76640 1 -- 172.21.15.190:0/372514878 >> 172.21.15.190:0/372514878 conn(0x7fc5cc0b6530 msgr2=0x7fc5cc06e5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:35:44.129 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.148+0000 7fc5d3c76640 1 -- 172.21.15.190:0/372514878 shutdown_connections 2024-05-09T06:35:44.129 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.148+0000 7fc5d3c76640 1 -- 172.21.15.190:0/372514878 wait complete. 2024-05-09T06:35:44.139 DEBUG:teuthology.orchestra.run.smithi190:> sudo ceph --cluster ceph osd new 918adf50-c440-48ac-9a8d-4837562d7275 5 2024-05-09T06:35:44.239 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.258+0000 7f7cdfc53640 1 Processor -- start 2024-05-09T06:35:44.239 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.258+0000 7f7cdfc53640 1 -- start start 2024-05-09T06:35:44.240 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.259+0000 7f7cdfc53640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7cd815e290 0x7f7cd806f050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:44.240 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.259+0000 7f7cdfc53640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f7cd80afd80 0x7f7cd806b250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:44.240 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.259+0000 7f7cdfc53640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f7cd806b790 0x7f7cd8161e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:44.240 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.259+0000 7f7cdfc53640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f7cd80b01e0 con 0x7f7cd806b790 2024-05-09T06:35:44.240 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.259+0000 7f7cdfc53640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f7cd8072230 con 0x7f7cd80afd80 2024-05-09T06:35:44.240 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.259+0000 7f7cdfc53640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f7cd8072510 con 0x7f7cd815e290 2024-05-09T06:35:44.241 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.259+0000 7f7cdd9ef640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7cd815e290 0x7f7cd806f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:44.241 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.259+0000 7f7cdd1ee640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f7cd80afd80 0x7f7cd806b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:44.241 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.259+0000 7f7cdd1ee640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f7cd80afd80 0x7f7cd806b250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.190:36320/0 (socket says 172.21.15.190:36320) 2024-05-09T06:35:44.241 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.259+0000 7f7cdd9ef640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7cd815e290 0x7f7cd806f050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.190:44584/0 (socket says 172.21.15.190:44584) 2024-05-09T06:35:44.241 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.259+0000 7f7cdd1ee640 1 -- 172.21.15.190:0/3618907206 learned_addr learned my addr 172.21.15.190:0/3618907206 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-05-09T06:35:44.241 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.259+0000 7f7cde1f0640 1 --2- 172.21.15.190:0/3618907206 >> v2:172.21.15.3:3300/0 conn(0x7f7cd806b790 0x7f7cd8161e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:44.241 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.259+0000 7f7cdd9ef640 1 -- 172.21.15.190:0/3618907206 >> v2:172.21.15.190:3300/0 conn(0x7f7cd80afd80 msgr2=0x7f7cd806b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:35:44.241 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.259+0000 7f7cdd9ef640 1 --2- 172.21.15.190:0/3618907206 >> v2:172.21.15.190:3300/0 conn(0x7f7cd80afd80 0x7f7cd806b250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:44.241 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.259+0000 7f7cdd9ef640 1 -- 172.21.15.190:0/3618907206 >> v2:172.21.15.3:3300/0 conn(0x7f7cd806b790 msgr2=0x7f7cd8161e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:35:44.241 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.259+0000 7f7cdd9ef640 1 --2- 172.21.15.190:0/3618907206 >> v2:172.21.15.3:3300/0 conn(0x7f7cd806b790 0x7f7cd8161e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:44.241 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.259+0000 7f7cdd9ef640 1 -- 172.21.15.190:0/3618907206 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7cd80727f0 con 0x7f7cd815e290 2024-05-09T06:35:44.241 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.259+0000 7f7cdd1ee640 1 --2- 172.21.15.190:0/3618907206 >> v2:172.21.15.190:3300/0 conn(0x7f7cd80afd80 0x7f7cd806b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T06:35:44.243 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.260+0000 7f7cdd9ef640 1 --2- 172.21.15.190:0/3618907206 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7cd815e290 0x7f7cd806f050 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f7cd80b03c0 tx=0x7f7cd402a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=56b8cb76a189d34e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:35:44.243 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.260+0000 7f7cdc9ed640 1 -- 172.21.15.190:0/3618907206 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cd4037070 con 0x7f7cd815e290 2024-05-09T06:35:44.243 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.260+0000 7f7cdc9ed640 1 -- 172.21.15.190:0/3618907206 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7cd402ae50 con 0x7f7cd815e290 2024-05-09T06:35:44.243 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.260+0000 7f7cdc9ed640 1 -- 172.21.15.190:0/3618907206 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cd4032660 con 0x7f7cd815e290 2024-05-09T06:35:44.243 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.261+0000 7f7cdfc53640 1 -- 172.21.15.190:0/3618907206 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7cd815e290 msgr2=0x7f7cd806f050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:44.243 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.261+0000 7f7cdfc53640 1 --2- 172.21.15.190:0/3618907206 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7cd815e290 0x7f7cd806f050 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f7cd80b03c0 tx=0x7f7cd402a100 comp rx=0 tx=0).stop 2024-05-09T06:35:44.243 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.261+0000 7f7cdfc53640 1 -- 172.21.15.190:0/3618907206 shutdown_connections 2024-05-09T06:35:44.243 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.261+0000 7f7cdfc53640 1 --2- 172.21.15.190:0/3618907206 >> v2:172.21.15.3:3300/0 conn(0x7f7cd806b790 0x7f7cd8161e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:44.243 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.261+0000 7f7cdfc53640 1 --2- 172.21.15.190:0/3618907206 >> v2:172.21.15.190:3300/0 conn(0x7f7cd80afd80 0x7f7cd806b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:44.243 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.261+0000 7f7cdfc53640 1 --2- 172.21.15.190:0/3618907206 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7cd815e290 0x7f7cd806f050 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:44.244 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.261+0000 7f7cdfc53640 1 -- 172.21.15.190:0/3618907206 >> 172.21.15.190:0/3618907206 conn(0x7f7cd80b2c30 msgr2=0x7f7cd80b3010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:35:44.244 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.261+0000 7f7cdfc53640 1 -- 172.21.15.190:0/3618907206 shutdown_connections 2024-05-09T06:35:44.244 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.261+0000 7f7cdfc53640 1 -- 172.21.15.190:0/3618907206 wait complete. 2024-05-09T06:35:44.244 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.262+0000 7f7cdfc53640 1 Processor -- start 2024-05-09T06:35:44.244 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.262+0000 7f7cdfc53640 1 -- start start 2024-05-09T06:35:44.246 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.263+0000 7f7cdfc53640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7cd806b790 0x7f7cd8160860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:44.246 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.263+0000 7f7cdfc53640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7cd80afd80 0x7f7cd81f8190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:44.246 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.263+0000 7f7cdfc53640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7cd815e290 0x7f7cd814e120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:44.246 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.263+0000 7f7cdfc53640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f7cd815e670 con 0x7f7cd806b790 2024-05-09T06:35:44.246 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.263+0000 7f7cdfc53640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f7cd8072db0 con 0x7f7cd815e290 2024-05-09T06:35:44.246 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.263+0000 7f7cdfc53640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f7cd8162380 con 0x7f7cd80afd80 2024-05-09T06:35:44.246 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.263+0000 7f7cdd1ee640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7cd80afd80 0x7f7cd81f8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:44.246 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.263+0000 7f7cde1f0640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7cd815e290 0x7f7cd814e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:44.247 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.263+0000 7f7cdd1ee640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7cd80afd80 0x7f7cd81f8190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.190:44598/0 (socket says 172.21.15.190:44598) 2024-05-09T06:35:44.247 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.263+0000 7f7cde1f0640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7cd815e290 0x7f7cd814e120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.190:36330/0 (socket says 172.21.15.190:36330) 2024-05-09T06:35:44.247 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.263+0000 7f7cdd1ee640 1 -- 172.21.15.190:0/2253736094 learned_addr learned my addr 172.21.15.190:0/2253736094 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-05-09T06:35:44.247 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.263+0000 7f7cdd9ef640 1 --2- 172.21.15.190:0/2253736094 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7cd806b790 0x7f7cd8160860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:44.247 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.263+0000 7f7cde1f0640 1 -- 172.21.15.190:0/2253736094 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7cd80afd80 msgr2=0x7f7cd81f8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:44.247 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.263+0000 7f7cde1f0640 1 --2- 172.21.15.190:0/2253736094 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7cd80afd80 0x7f7cd81f8190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:44.247 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.263+0000 7f7cde1f0640 1 -- 172.21.15.190:0/2253736094 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7cd806b790 msgr2=0x7f7cd8160860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:44.247 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.263+0000 7f7cde1f0640 1 --2- 172.21.15.190:0/2253736094 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7cd806b790 0x7f7cd8160860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:44.247 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.263+0000 7f7cde1f0640 1 -- 172.21.15.190:0/2253736094 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7cd8160eb0 con 0x7f7cd815e290 2024-05-09T06:35:44.247 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.263+0000 7f7cdd1ee640 1 --2- 172.21.15.190:0/2253736094 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7cd80afd80 0x7f7cd81f8190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T06:35:44.247 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.263+0000 7f7cde1f0640 1 --2- 172.21.15.190:0/2253736094 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7cd815e290 0x7f7cd814e120 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f7cd0007da0 tx=0x7f7cd000e4b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:35:44.247 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.264+0000 7f7cce7fc640 1 -- 172.21.15.190:0/2253736094 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cd0019070 con 0x7f7cd815e290 2024-05-09T06:35:44.247 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.264+0000 7f7cce7fc640 1 -- 172.21.15.190:0/2253736094 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7cd0003990 con 0x7f7cd815e290 2024-05-09T06:35:44.247 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.264+0000 7f7cce7fc640 1 -- 172.21.15.190:0/2253736094 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cd0003d80 con 0x7f7cd815e290 2024-05-09T06:35:44.248 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.264+0000 7f7cdfc53640 1 -- 172.21.15.190:0/2253736094 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7cd8161190 con 0x7f7cd815e290 2024-05-09T06:35:44.248 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.264+0000 7f7cdfc53640 1 -- 172.21.15.190:0/2253736094 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7cd81528b0 con 0x7f7cd815e290 2024-05-09T06:35:44.248 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.264+0000 7f7cce7fc640 1 -- 172.21.15.190:0/2253736094 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f7cd001c070 con 0x7f7cd815e290 2024-05-09T06:35:44.248 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.264+0000 7f7cce7fc640 1 -- 172.21.15.190:0/2253736094 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(7..7 src has 1..7) ==== 1300+0+0 (secure 0 0 0) 0x7f7cd00045b0 con 0x7f7cd815e290 2024-05-09T06:35:44.248 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.265+0000 7f7cdfc53640 1 -- 172.21.15.190:0/2253736094 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7cd8152ab0 con 0x7f7cd815e290 2024-05-09T06:35:44.249 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.268+0000 7f7cce7fc640 1 -- 172.21.15.190:0/2253736094 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f7cd8152ab0 con 0x7f7cd815e290 2024-05-09T06:35:44.255 INFO:tasks.ceph.mgr.x.smithi190.stderr:2024-05-09T06:35:44.272+0000 7fab08831040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-09T06:35:44.267 INFO:tasks.ceph.mgr.y.smithi003.stderr:2024-05-09T06:35:44.286+0000 7f733de11040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-09T06:35:44.305 INFO:tasks.ceph.mgr.x.smithi190.stderr:2024-05-09T06:35:44.324+0000 7fab08831040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-09T06:35:44.315 INFO:tasks.ceph.mgr.y.smithi003.stderr:2024-05-09T06:35:44.335+0000 7f733de11040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-09T06:35:44.322 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.341+0000 7f7cdfc53640 1 -- 172.21.15.190:0/2253736094 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "osd new", "uuid": "918adf50-c440-48ac-9a8d-4837562d7275", "id": 5} v 0) -- 0x7f7cd80682e0 con 0x7f7cd815e290 2024-05-09T06:35:44.326 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.345+0000 7f7cce7fc640 1 -- 172.21.15.190:0/2253736094 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "918adf50-c440-48ac-9a8d-4837562d7275", "id": 5}]=0 v8) ==== 112+0+2 (secure 0 0 0) 0x7f7cd0023020 con 0x7f7cd815e290 2024-05-09T06:35:44.327 INFO:teuthology.orchestra.run.smithi190.stdout:5 2024-05-09T06:35:44.329 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.346+0000 7f7cdfc53640 1 -- 172.21.15.190:0/2253736094 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7cd815e290 msgr2=0x7f7cd814e120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:44.329 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.346+0000 7f7cdfc53640 1 --2- 172.21.15.190:0/2253736094 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7cd815e290 0x7f7cd814e120 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f7cd0007da0 tx=0x7f7cd000e4b0 comp rx=0 tx=0).stop 2024-05-09T06:35:44.329 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.346+0000 7f7cdfc53640 1 -- 172.21.15.190:0/2253736094 shutdown_connections 2024-05-09T06:35:44.329 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.346+0000 7f7cdfc53640 1 --2- 172.21.15.190:0/2253736094 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7cd815e290 0x7f7cd814e120 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:44.330 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.346+0000 7f7cdfc53640 1 --2- 172.21.15.190:0/2253736094 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7cd80afd80 0x7f7cd81f8190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:44.330 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.346+0000 7f7cdfc53640 1 --2- 172.21.15.190:0/2253736094 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7cd806b790 0x7f7cd8160860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:44.330 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.346+0000 7f7cdfc53640 1 -- 172.21.15.190:0/2253736094 >> 172.21.15.190:0/2253736094 conn(0x7f7cd80b2c30 msgr2=0x7f7cd80ae980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:35:44.330 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.346+0000 7f7cdfc53640 1 -- 172.21.15.190:0/2253736094 shutdown_connections 2024-05-09T06:35:44.331 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.346+0000 7f7cdfc53640 1 -- 172.21.15.190:0/2253736094 wait complete. 2024-05-09T06:35:44.338 DEBUG:teuthology.orchestra.run.smithi190:> sudo ceph --cluster ceph osd new ac2882bc-2a7c-463c-8f6c-76ff49828adc 6 2024-05-09T06:35:44.361 INFO:tasks.ceph.mgr.x.smithi190.stderr:2024-05-09T06:35:44.380+0000 7fab08831040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-09T06:35:44.371 INFO:tasks.ceph.mgr.y.smithi003.stderr:2024-05-09T06:35:44.390+0000 7f733de11040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-09T06:35:44.441 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.460+0000 7f64cc56a640 1 Processor -- start 2024-05-09T06:35:44.441 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.460+0000 7f64cc56a640 1 -- start start 2024-05-09T06:35:44.443 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.461+0000 7f64cc56a640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f64c4072020 0x7f64c4072400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:44.443 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.461+0000 7f64cc56a640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f64c4072940 con 0x7f64c40cebc0 2024-05-09T06:35:44.443 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.461+0000 7f64cc56a640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f64c40b49c0 con 0x7f64c40ce760 2024-05-09T06:35:44.443 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.461+0000 7f64cc56a640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f64c40cf050 con 0x7f64c4072020 2024-05-09T06:35:44.443 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.461+0000 7f64ca306640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f64c4072020 0x7f64c4072400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:44.443 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.461+0000 7f64ca306640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f64c4072020 0x7f64c4072400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.190:36340/0 (socket says 172.21.15.190:36340) 2024-05-09T06:35:44.443 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.461+0000 7f64c9b05640 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7f64c40ce760 0x7f64c40b3e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.190:51364/0 (socket says 172.21.15.190:51364) 2024-05-09T06:35:44.443 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.461+0000 7f64ca306640 1 -- 172.21.15.190:0/1477045461 learned_addr learned my addr 172.21.15.190:0/1477045461 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-05-09T06:35:44.443 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.462+0000 7f64ca306640 1 -- 172.21.15.190:0/1477045461 >> v1:172.21.15.190:6789/0 conn(0x7f64c40ce760 legacy=0x7f64c40b3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:44.444 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.462+0000 7f64ca306640 1 -- 172.21.15.190:0/1477045461 >> v1:172.21.15.3:6789/0 conn(0x7f64c40cebc0 legacy=0x7f64c406ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:44.444 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.462+0000 7f64c9304640 1 -- 172.21.15.190:0/1477045461 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1179477525 0 0) 0x7f64c40b49c0 con 0x7f64c40ce760 2024-05-09T06:35:44.444 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.462+0000 7f64ca306640 1 -- 172.21.15.190:0/1477045461 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64c40b4ca0 con 0x7f64c4072020 2024-05-09T06:35:44.445 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.462+0000 7f64ca306640 1 --2- 172.21.15.190:0/1477045461 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f64c4072020 0x7f64c4072400 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f64b8009870 tx=0x7f64b802a300 comp rx=0 tx=0).ready entity=mon.1 client_cookie=59d945aa2775c58c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:35:44.445 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.463+0000 7f64c9304640 1 -- 172.21.15.190:0/1477045461 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64b8037070 con 0x7f64c4072020 2024-05-09T06:35:44.445 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.463+0000 7f64c9304640 1 -- 172.21.15.190:0/1477045461 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f64b802ae50 con 0x7f64c4072020 2024-05-09T06:35:44.445 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.463+0000 7f64c9304640 1 -- 172.21.15.190:0/1477045461 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64b8032860 con 0x7f64c4072020 2024-05-09T06:35:44.445 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.463+0000 7f64cc56a640 1 -- 172.21.15.190:0/1477045461 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f64c4072020 msgr2=0x7f64c4072400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:44.446 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.463+0000 7f64cc56a640 1 --2- 172.21.15.190:0/1477045461 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f64c4072020 0x7f64c4072400 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f64b8009870 tx=0x7f64b802a300 comp rx=0 tx=0).stop 2024-05-09T06:35:44.446 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.463+0000 7f64cc56a640 1 -- 172.21.15.190:0/1477045461 shutdown_connections 2024-05-09T06:35:44.446 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.463+0000 7f64cc56a640 1 --2- 172.21.15.190:0/1477045461 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f64c4072020 0x7f64c4072400 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:44.446 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.463+0000 7f64cc56a640 1 -- 172.21.15.190:0/1477045461 >> 172.21.15.190:0/1477045461 conn(0x7f64c40b2b50 msgr2=0x7f64c40b2f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:35:44.446 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.463+0000 7f64cc56a640 1 -- 172.21.15.190:0/1477045461 shutdown_connections 2024-05-09T06:35:44.446 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.463+0000 7f64cc56a640 1 -- 172.21.15.190:0/1477045461 wait complete. 2024-05-09T06:35:44.446 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.464+0000 7f64cc56a640 1 Processor -- start 2024-05-09T06:35:44.446 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.465+0000 7f64cc56a640 1 -- start start 2024-05-09T06:35:44.448 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.465+0000 7f64cc56a640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f64c4072020 0x7f64c413fa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:44.449 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.465+0000 7f64cc56a640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f64c40ce760 0x7f64c4142fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:44.449 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.465+0000 7f64cc56a640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f64c40cebc0 0x7f64c41434e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:44.449 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.465+0000 7f64cc56a640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f64c4073240 con 0x7f64c40ce760 2024-05-09T06:35:44.449 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.465+0000 7f64cc56a640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f64c406e6d0 con 0x7f64c40cebc0 2024-05-09T06:35:44.449 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.465+0000 7f64cc56a640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f64c4144cd0 con 0x7f64c4072020 2024-05-09T06:35:44.449 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.465+0000 7f64ca306640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f64c4072020 0x7f64c413fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:44.449 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.465+0000 7f64cab07640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f64c40cebc0 0x7f64c41434e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:44.449 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.465+0000 7f64ca306640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f64c4072020 0x7f64c413fa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.190:44606/0 (socket says 172.21.15.190:44606) 2024-05-09T06:35:44.449 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.465+0000 7f64ca306640 1 -- 172.21.15.190:0/1931797196 learned_addr learned my addr 172.21.15.190:0/1931797196 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-05-09T06:35:44.449 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.465+0000 7f64cab07640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f64c40cebc0 0x7f64c41434e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.190:36352/0 (socket says 172.21.15.190:36352) 2024-05-09T06:35:44.449 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.465+0000 7f64c9b05640 1 --2- 172.21.15.190:0/1931797196 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f64c40ce760 0x7f64c4142fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:44.449 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.466+0000 7f64ca306640 1 -- 172.21.15.190:0/1931797196 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f64c40cebc0 msgr2=0x7f64c41434e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:44.450 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.466+0000 7f64ca306640 1 --2- 172.21.15.190:0/1931797196 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f64c40cebc0 0x7f64c41434e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:44.450 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.466+0000 7f64ca306640 1 -- 172.21.15.190:0/1931797196 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f64c40ce760 msgr2=0x7f64c4142fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:44.450 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.466+0000 7f64ca306640 1 --2- 172.21.15.190:0/1931797196 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f64c40ce760 0x7f64c4142fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:44.450 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.466+0000 7f64ca306640 1 -- 172.21.15.190:0/1931797196 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64c40adaf0 con 0x7f64c4072020 2024-05-09T06:35:44.450 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.466+0000 7f64cab07640 1 --2- 172.21.15.190:0/1931797196 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f64c40cebc0 0x7f64c41434e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T06:35:44.450 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.466+0000 7f64ca306640 1 --2- 172.21.15.190:0/1931797196 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f64c4072020 0x7f64c413fa70 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f64b8009870 tx=0x7f64b80046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:35:44.450 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.466+0000 7f64b6ffd640 1 -- 172.21.15.190:0/1931797196 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64b8042070 con 0x7f64c4072020 2024-05-09T06:35:44.450 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.466+0000 7f64b6ffd640 1 -- 172.21.15.190:0/1931797196 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f64b803e870 con 0x7f64c4072020 2024-05-09T06:35:44.450 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.466+0000 7f64cc56a640 1 -- 172.21.15.190:0/1931797196 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64c40addd0 con 0x7f64c4072020 2024-05-09T06:35:44.450 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.466+0000 7f64b6ffd640 1 -- 172.21.15.190:0/1931797196 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64b8037040 con 0x7f64c4072020 2024-05-09T06:35:44.450 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.466+0000 7f64cc56a640 1 -- 172.21.15.190:0/1931797196 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f64c40ae390 con 0x7f64c4072020 2024-05-09T06:35:44.450 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.466+0000 7f64b6ffd640 1 -- 172.21.15.190:0/1931797196 <== mon.2 v2:172.21.15.190:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f64b8045070 con 0x7f64c4072020 2024-05-09T06:35:44.450 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.466+0000 7f64b6ffd640 1 -- 172.21.15.190:0/1931797196 <== mon.2 v2:172.21.15.190:3301/0 5 ==== osd_map(8..8 src has 1..8) ==== 1415+0+0 (secure 0 0 0) 0x7f64b8047b30 con 0x7f64c4072020 2024-05-09T06:35:44.450 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.467+0000 7f64cc56a640 1 -- 172.21.15.190:0/1931797196 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64c40682e0 con 0x7f64c4072020 2024-05-09T06:35:44.451 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.469+0000 7f64b6ffd640 1 -- 172.21.15.190:0/1931797196 <== mon.2 v2:172.21.15.190:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f64b8032e20 con 0x7f64c4072020 2024-05-09T06:35:44.479 INFO:tasks.ceph.mgr.x.smithi190.stderr:2024-05-09T06:35:44.498+0000 7fab08831040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-09T06:35:44.488 INFO:tasks.ceph.mgr.y.smithi003.stderr:2024-05-09T06:35:44.508+0000 7f733de11040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-09T06:35:44.521 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.540+0000 7f64cc56a640 1 -- 172.21.15.190:0/1931797196 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "osd new", "uuid": "ac2882bc-2a7c-463c-8f6c-76ff49828adc", "id": 6} v 0) -- 0x7f64c4072d70 con 0x7f64c4072020 2024-05-09T06:35:44.524 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.544+0000 7f64b6ffd640 1 -- 172.21.15.190:0/1931797196 <== mon.2 v2:172.21.15.190:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "ac2882bc-2a7c-463c-8f6c-76ff49828adc", "id": 6}]=0 v9) ==== 112+0+2 (secure 0 0 0) 0x7f64b803ed30 con 0x7f64c4072020 2024-05-09T06:35:44.524 INFO:teuthology.orchestra.run.smithi190.stdout:6 2024-05-09T06:35:44.526 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.544+0000 7f64cc56a640 1 -- 172.21.15.190:0/1931797196 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f64c4072020 msgr2=0x7f64c413fa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:44.526 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.544+0000 7f64cc56a640 1 --2- 172.21.15.190:0/1931797196 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f64c4072020 0x7f64c413fa70 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f64b8009870 tx=0x7f64b80046c0 comp rx=0 tx=0).stop 2024-05-09T06:35:44.526 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.545+0000 7f64cc56a640 1 -- 172.21.15.190:0/1931797196 shutdown_connections 2024-05-09T06:35:44.526 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.545+0000 7f64cc56a640 1 --2- 172.21.15.190:0/1931797196 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f64c40cebc0 0x7f64c41434e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:44.526 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.545+0000 7f64cc56a640 1 --2- 172.21.15.190:0/1931797196 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f64c40ce760 0x7f64c4142fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:44.526 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.545+0000 7f64cc56a640 1 --2- 172.21.15.190:0/1931797196 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f64c4072020 0x7f64c413fa70 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:44.526 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.545+0000 7f64cc56a640 1 -- 172.21.15.190:0/1931797196 >> 172.21.15.190:0/1931797196 conn(0x7f64c40b2b50 msgr2=0x7f64c40b0c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:35:44.526 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.545+0000 7f64cc56a640 1 -- 172.21.15.190:0/1931797196 shutdown_connections 2024-05-09T06:35:44.526 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.545+0000 7f64cc56a640 1 -- 172.21.15.190:0/1931797196 wait complete. 2024-05-09T06:35:44.536 DEBUG:teuthology.orchestra.run.smithi190:> sudo ceph --cluster ceph osd new dd4e767a-4730-4c70-8bd5-3f088140a5a1 7 2024-05-09T06:35:44.677 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.696+0000 7f60f2e15640 1 Processor -- start 2024-05-09T06:35:44.677 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.696+0000 7f60f2e15640 1 -- start start 2024-05-09T06:35:44.679 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.697+0000 7f60f2e15640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f60ec1589b0 0x7f60ec15f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:44.679 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.697+0000 7f60f2e15640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f60ec0b3640 0x7f60ec0b3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:44.679 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.697+0000 7f60f2e15640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f60ec161f30 0x7f60ec1623c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:44.679 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.697+0000 7f60f2e15640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f60ec15f8e0 con 0x7f60ec1589b0 2024-05-09T06:35:44.679 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.697+0000 7f60f2e15640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f60ec06ddf0 con 0x7f60ec161f30 2024-05-09T06:35:44.679 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.697+0000 7f60f2e15640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f60ec06e0d0 con 0x7f60ec0b3640 2024-05-09T06:35:44.679 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.697+0000 7f60ebfff640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f60ec0b3640 0x7f60ec0b3ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:44.680 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.697+0000 7f60ebfff640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f60ec0b3640 0x7f60ec0b3ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.190:44622/0 (socket says 172.21.15.190:44622) 2024-05-09T06:35:44.680 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.697+0000 7f60ebfff640 1 -- 172.21.15.190:0/2406422766 learned_addr learned my addr 172.21.15.190:0/2406422766 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-05-09T06:35:44.680 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.697+0000 7f60f13b2640 1 --2- 172.21.15.190:0/2406422766 >> v2:172.21.15.190:3300/0 conn(0x7f60ec161f30 0x7f60ec1623c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:44.680 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.697+0000 7f60f0bb1640 1 --2- 172.21.15.190:0/2406422766 >> v2:172.21.15.3:3300/0 conn(0x7f60ec1589b0 0x7f60ec15f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:44.680 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.697+0000 7f60ebfff640 1 -- 172.21.15.190:0/2406422766 >> v2:172.21.15.190:3300/0 conn(0x7f60ec161f30 msgr2=0x7f60ec1623c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:35:44.680 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.697+0000 7f60ebfff640 1 --2- 172.21.15.190:0/2406422766 >> v2:172.21.15.190:3300/0 conn(0x7f60ec161f30 0x7f60ec1623c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:44.680 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.697+0000 7f60ebfff640 1 -- 172.21.15.190:0/2406422766 >> v2:172.21.15.3:3300/0 conn(0x7f60ec1589b0 msgr2=0x7f60ec15f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:35:44.680 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.697+0000 7f60ebfff640 1 --2- 172.21.15.190:0/2406422766 >> v2:172.21.15.3:3300/0 conn(0x7f60ec1589b0 0x7f60ec15f3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:44.680 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.697+0000 7f60ebfff640 1 -- 172.21.15.190:0/2406422766 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60ec06e3b0 con 0x7f60ec0b3640 2024-05-09T06:35:44.680 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.698+0000 7f60ebfff640 1 --2- 172.21.15.190:0/2406422766 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f60ec0b3640 0x7f60ec0b3ac0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f60ec0b5ce0 tx=0x7f60d802a140 comp rx=0 tx=0).ready entity=mon.2 client_cookie=779840387ae13cc0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:35:44.680 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.698+0000 7f60eb7fe640 1 -- 172.21.15.190:0/2406422766 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60d8037070 con 0x7f60ec0b3640 2024-05-09T06:35:44.680 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.698+0000 7f60eb7fe640 1 -- 172.21.15.190:0/2406422766 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60d8032440 con 0x7f60ec0b3640 2024-05-09T06:35:44.680 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.698+0000 7f60eb7fe640 1 -- 172.21.15.190:0/2406422766 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60d8032810 con 0x7f60ec0b3640 2024-05-09T06:35:44.684 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.699+0000 7f60f2e15640 1 -- 172.21.15.190:0/2406422766 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f60ec0b3640 msgr2=0x7f60ec0b3ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:44.684 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.699+0000 7f60f2e15640 1 --2- 172.21.15.190:0/2406422766 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f60ec0b3640 0x7f60ec0b3ac0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f60ec0b5ce0 tx=0x7f60d802a140 comp rx=0 tx=0).stop 2024-05-09T06:35:44.684 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.699+0000 7f60f2e15640 1 -- 172.21.15.190:0/2406422766 shutdown_connections 2024-05-09T06:35:44.684 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.699+0000 7f60f2e15640 1 --2- 172.21.15.190:0/2406422766 >> v2:172.21.15.190:3300/0 conn(0x7f60ec161f30 0x7f60ec1623c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:44.684 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.699+0000 7f60f2e15640 1 --2- 172.21.15.190:0/2406422766 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f60ec0b3640 0x7f60ec0b3ac0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:44.684 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.699+0000 7f60f2e15640 1 --2- 172.21.15.190:0/2406422766 >> v2:172.21.15.3:3300/0 conn(0x7f60ec1589b0 0x7f60ec15f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:44.684 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.699+0000 7f60f2e15640 1 -- 172.21.15.190:0/2406422766 >> 172.21.15.190:0/2406422766 conn(0x7f60ec0d06f0 msgr2=0x7f60ec0d0ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:35:44.685 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.699+0000 7f60f2e15640 1 -- 172.21.15.190:0/2406422766 shutdown_connections 2024-05-09T06:35:44.685 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.699+0000 7f60f2e15640 1 -- 172.21.15.190:0/2406422766 wait complete. 2024-05-09T06:35:44.685 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.700+0000 7f60f2e15640 1 Processor -- start 2024-05-09T06:35:44.685 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.700+0000 7f60f2e15640 1 -- start start 2024-05-09T06:35:44.685 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.701+0000 7f60f2e15640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f60ec0b3640 0x7f60ec145160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:44.685 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.701+0000 7f60f2e15640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f60ec1589b0 0x7f60ec157140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:44.685 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.701+0000 7f60f2e15640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f60ec161f30 0x7f60ec157680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:44.685 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.701+0000 7f60f2e15640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f60ec06da80 con 0x7f60ec0b3640 2024-05-09T06:35:44.685 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.701+0000 7f60f2e15640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f60ec158d90 con 0x7f60ec1589b0 2024-05-09T06:35:44.685 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.701+0000 7f60f2e15640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f60ec0b4340 con 0x7f60ec161f30 2024-05-09T06:35:44.685 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.701+0000 7f60f13b2640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f60ec161f30 0x7f60ec157680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:44.685 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.701+0000 7f60ebfff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f60ec1589b0 0x7f60ec157140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:44.685 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.701+0000 7f60f13b2640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f60ec161f30 0x7f60ec157680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.190:44628/0 (socket says 172.21.15.190:44628) 2024-05-09T06:35:44.685 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.701+0000 7f60f13b2640 1 -- 172.21.15.190:0/277529002 learned_addr learned my addr 172.21.15.190:0/277529002 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-05-09T06:35:44.686 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.701+0000 7f60f0bb1640 1 --2- 172.21.15.190:0/277529002 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f60ec0b3640 0x7f60ec145160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:44.686 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.701+0000 7f60f13b2640 1 -- 172.21.15.190:0/277529002 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f60ec1589b0 msgr2=0x7f60ec157140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:44.686 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.701+0000 7f60f13b2640 1 --2- 172.21.15.190:0/277529002 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f60ec1589b0 0x7f60ec157140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:44.686 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.701+0000 7f60f13b2640 1 -- 172.21.15.190:0/277529002 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f60ec0b3640 msgr2=0x7f60ec145160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:44.686 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.701+0000 7f60f13b2640 1 --2- 172.21.15.190:0/277529002 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f60ec0b3640 0x7f60ec145160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:44.686 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.701+0000 7f60f13b2640 1 -- 172.21.15.190:0/277529002 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60ec1456a0 con 0x7f60ec161f30 2024-05-09T06:35:44.686 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.701+0000 7f60ebfff640 1 --2- 172.21.15.190:0/277529002 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f60ec1589b0 0x7f60ec157140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T06:35:44.686 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.701+0000 7f60f0bb1640 1 --2- 172.21.15.190:0/277529002 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f60ec0b3640 0x7f60ec145160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T06:35:44.686 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.701+0000 7f60f13b2640 1 --2- 172.21.15.190:0/277529002 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f60ec161f30 0x7f60ec157680 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f60e4009d30 tx=0x7f60e400c490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:35:44.686 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.702+0000 7f60e97fa640 1 -- 172.21.15.190:0/277529002 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60e4018020 con 0x7f60ec161f30 2024-05-09T06:35:44.686 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.702+0000 7f60e97fa640 1 -- 172.21.15.190:0/277529002 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60e4014480 con 0x7f60ec161f30 2024-05-09T06:35:44.686 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.702+0000 7f60e97fa640 1 -- 172.21.15.190:0/277529002 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60e4014870 con 0x7f60ec161f30 2024-05-09T06:35:44.686 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.702+0000 7f60f2e15640 1 -- 172.21.15.190:0/277529002 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60ec1e7fc0 con 0x7f60ec161f30 2024-05-09T06:35:44.686 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.702+0000 7f60f2e15640 1 -- 172.21.15.190:0/277529002 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f60ec1e8580 con 0x7f60ec161f30 2024-05-09T06:35:44.687 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.702+0000 7f60e97fa640 1 -- 172.21.15.190:0/277529002 <== mon.2 v2:172.21.15.190:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f60e4005d10 con 0x7f60ec161f30 2024-05-09T06:35:44.687 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.702+0000 7f60e97fa640 1 -- 172.21.15.190:0/277529002 <== mon.2 v2:172.21.15.190:3301/0 5 ==== osd_map(9..9 src has 1..9) ==== 1530+0+0 (secure 0 0 0) 0x7f60e4013a40 con 0x7f60ec161f30 2024-05-09T06:35:44.687 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.703+0000 7f60f2e15640 1 -- 172.21.15.190:0/277529002 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60ec0682e0 con 0x7f60ec161f30 2024-05-09T06:35:44.687 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.705+0000 7f60e97fa640 1 -- 172.21.15.190:0/277529002 <== mon.2 v2:172.21.15.190:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f60e4022030 con 0x7f60ec161f30 2024-05-09T06:35:44.759 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.778+0000 7f60f2e15640 1 -- 172.21.15.190:0/277529002 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "osd new", "uuid": "dd4e767a-4730-4c70-8bd5-3f088140a5a1", "id": 7} v 0) -- 0x7f60ec067280 con 0x7f60ec161f30 2024-05-09T06:35:44.765 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.784+0000 7f60e97fa640 1 -- 172.21.15.190:0/277529002 <== mon.2 v2:172.21.15.190:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "dd4e767a-4730-4c70-8bd5-3f088140a5a1", "id": 7}]=0 v10) ==== 112+0+2 (secure 0 0 0) 0x7f60ec1456a0 con 0x7f60ec161f30 2024-05-09T06:35:44.765 INFO:teuthology.orchestra.run.smithi190.stdout:7 2024-05-09T06:35:44.766 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.785+0000 7f60f2e15640 1 -- 172.21.15.190:0/277529002 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f60ec161f30 msgr2=0x7f60ec157680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:44.766 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.785+0000 7f60f2e15640 1 --2- 172.21.15.190:0/277529002 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f60ec161f30 0x7f60ec157680 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f60e4009d30 tx=0x7f60e400c490 comp rx=0 tx=0).stop 2024-05-09T06:35:44.767 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.785+0000 7f60f2e15640 1 -- 172.21.15.190:0/277529002 shutdown_connections 2024-05-09T06:35:44.767 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.785+0000 7f60f2e15640 1 --2- 172.21.15.190:0/277529002 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f60ec161f30 0x7f60ec157680 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:44.767 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.785+0000 7f60f2e15640 1 --2- 172.21.15.190:0/277529002 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f60ec1589b0 0x7f60ec157140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:44.767 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.785+0000 7f60f2e15640 1 --2- 172.21.15.190:0/277529002 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f60ec0b3640 0x7f60ec145160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:44.767 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.785+0000 7f60f2e15640 1 -- 172.21.15.190:0/277529002 >> 172.21.15.190:0/277529002 conn(0x7f60ec0d06f0 msgr2=0x7f60ec15fae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:35:44.767 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.785+0000 7f60f2e15640 1 -- 172.21.15.190:0/277529002 shutdown_connections 2024-05-09T06:35:44.767 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:35:44.785+0000 7f60f2e15640 1 -- 172.21.15.190:0/277529002 wait complete. 2024-05-09T06:35:44.777 INFO:tasks.ceph.osd.0:Restarting daemon 2024-05-09T06:35:44.777 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-05-09T06:35:44.779 INFO:tasks.ceph.osd.0:Started 2024-05-09T06:35:44.779 INFO:tasks.ceph.osd.1:Restarting daemon 2024-05-09T06:35:44.780 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-05-09T06:35:44.782 INFO:tasks.ceph.osd.1:Started 2024-05-09T06:35:44.782 INFO:tasks.ceph.osd.2:Restarting daemon 2024-05-09T06:35:44.782 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-05-09T06:35:44.784 INFO:tasks.ceph.osd.2:Started 2024-05-09T06:35:44.784 INFO:tasks.ceph.osd.3:Restarting daemon 2024-05-09T06:35:44.784 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-05-09T06:35:44.787 INFO:tasks.ceph.osd.3:Started 2024-05-09T06:35:44.787 INFO:tasks.ceph.osd.4:Restarting daemon 2024-05-09T06:35:44.787 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-05-09T06:35:44.789 INFO:tasks.ceph.osd.4:Started 2024-05-09T06:35:44.789 INFO:tasks.ceph.osd.5:Restarting daemon 2024-05-09T06:35:44.789 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-05-09T06:35:44.791 INFO:tasks.ceph.osd.5:Started 2024-05-09T06:35:44.791 INFO:tasks.ceph.osd.6:Restarting daemon 2024-05-09T06:35:44.791 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-05-09T06:35:44.793 INFO:tasks.ceph.osd.6:Started 2024-05-09T06:35:44.793 INFO:tasks.ceph.osd.7:Restarting daemon 2024-05-09T06:35:44.793 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-05-09T06:35:44.796 INFO:tasks.ceph.osd.7:Started 2024-05-09T06:35:44.796 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T06:35:44.897 INFO:tasks.ceph.mgr.x.smithi190.stderr:2024-05-09T06:35:44.916+0000 7fab08831040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-09T06:35:44.943 INFO:tasks.ceph.mgr.y.smithi003.stderr:2024-05-09T06:35:44.962+0000 7f733de11040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-09T06:35:44.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:35:44.967+0000 7f1045e8c640 1 Processor -- start 2024-05-09T06:35:44.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:35:44.967+0000 7f1045e8c640 1 -- start start 2024-05-09T06:35:44.949 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:35:44.968+0000 7f1045e8c640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f10401625a0 0x7f104006c360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:44.949 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:35:44.968+0000 7f1045e8c640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f104015e510 0x7f10400b19d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:44.949 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:35:44.968+0000 7f1045e8c640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f104006c940 con 0x7f104015e130 2024-05-09T06:35:44.949 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:35:44.968+0000 7f1045e8c640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f104006cc20 con 0x7f104015e510 2024-05-09T06:35:44.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:35:44.968+0000 7f1045e8c640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f104006cf00 con 0x7f10401625a0 2024-05-09T06:35:44.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:35:44.968+0000 7f103ffff640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f104015e510 0x7f10400b19d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:44.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:35:44.968+0000 7f103f7fe640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f104015e130 0x7f1040161e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:60788/0 (socket says 172.21.15.3:60788) 2024-05-09T06:35:44.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:35:44.968+0000 7f103ffff640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f104015e510 0x7f10400b19d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:42140/0 (socket says 172.21.15.3:42140) 2024-05-09T06:35:44.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:35:44.968+0000 7f103f7fe640 1 -- 172.21.15.3:0/715874835 learned_addr learned my addr 172.21.15.3:0/715874835 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-05-09T06:35:44.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:35:44.968+0000 7f1036ffd640 1 --2- 172.21.15.3:0/715874835 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f10401625a0 0x7f104006c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:44.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:35:44.968+0000 7f103ffff640 1 -- 172.21.15.3:0/715874835 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f10401625a0 msgr2=0x7f104006c360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:35:44.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:35:44.968+0000 7f103ffff640 1 --2- 172.21.15.3:0/715874835 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f10401625a0 0x7f104006c360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:44.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:35:44.968+0000 7f103ffff640 1 -- 172.21.15.3:0/715874835 >> v1:172.21.15.3:6789/0 conn(0x7f104015e130 legacy=0x7f1040161e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:44.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:35:44.968+0000 7f103ffff640 1 -- 172.21.15.3:0/715874835 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f104006d1e0 con 0x7f104015e510 2024-05-09T06:35:44.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:35:44.968+0000 7f103effd640 1 -- 172.21.15.3:0/715874835 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 6675618 0 0) 0x7f104006c940 con 0x7f104015e130 2024-05-09T06:35:44.951 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:35:44.969+0000 7f103ffff640 1 --2- 172.21.15.3:0/715874835 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f104015e510 0x7f10400b19d0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f1038009d90 tx=0x7f103802a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72ec934bc7c43619 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:35:44.951 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:35:44.969+0000 7f103effd640 1 -- 172.21.15.3:0/715874835 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1038007740 con 0x7f104015e510 2024-05-09T06:35:44.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:35:44.969+0000 7f103effd640 1 -- 172.21.15.3:0/715874835 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10380078e0 con 0x7f104015e510 2024-05-09T06:35:44.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:35:44.969+0000 7f103effd640 1 -- 172.21.15.3:0/715874835 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1038007cb0 con 0x7f104015e510 2024-05-09T06:35:44.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:35:44.970+0000 7f1045e8c640 1 -- 172.21.15.3:0/715874835 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f104015e510 msgr2=0x7f10400b19d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:44.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:35:44.970+0000 7f1045e8c640 1 --2- 172.21.15.3:0/715874835 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f104015e510 0x7f10400b19d0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f1038009d90 tx=0x7f103802a660 comp rx=0 tx=0).stop 2024-05-09T06:35:44.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:35:44.970+0000 7f1045e8c640 1 -- 172.21.15.3:0/715874835 shutdown_connections 2024-05-09T06:35:44.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:35:44.970+0000 7f1045e8c640 1 --2- 172.21.15.3:0/715874835 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f104015e510 0x7f10400b19d0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:44.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:35:44.970+0000 7f1045e8c640 1 --2- 172.21.15.3:0/715874835 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f10401625a0 0x7f104006c360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:44.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:35:44.970+0000 7f1045e8c640 1 -- 172.21.15.3:0/715874835 >> 172.21.15.3:0/715874835 conn(0x7f10400b3a80 msgr2=0x7f10400b3e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:35:44.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:35:44.970+0000 7f1045e8c640 1 -- 172.21.15.3:0/715874835 shutdown_connections 2024-05-09T06:35:44.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:35:44.970+0000 7f1045e8c640 1 -- 172.21.15.3:0/715874835 wait complete. 2024-05-09T06:35:44.960 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:35:44.971+0000 7f1045e8c640 1 Processor -- start 2024-05-09T06:35:44.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:35:44.971+0000 7f1045e8c640 1 -- start start 2024-05-09T06:35:44.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:35:44.972+0000 7f1045e8c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f104015e130 0x7f104014e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:44.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:35:44.972+0000 7f1045e8c640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f104015e510 0x7f10400b4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:44.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:35:44.972+0000 7f1045e8c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f10401625a0 0x7f10401f7510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:44.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:35:44.972+0000 7f1045e8c640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f10400b2090 con 0x7f104015e130 2024-05-09T06:35:44.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:35:44.972+0000 7f1045e8c640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f10400b1f10 con 0x7f10401625a0 2024-05-09T06:35:44.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:35:44.972+0000 7f1045e8c640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f10400b2210 con 0x7f104015e510 2024-05-09T06:35:44.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:35:44.972+0000 7f103f7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f104015e130 0x7f104014e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:44.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:35:44.972+0000 7f103f7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f104015e130 0x7f104014e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:42146/0 (socket says 172.21.15.3:42146) 2024-05-09T06:35:44.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:35:44.972+0000 7f103f7fe640 1 -- 172.21.15.3:0/85540084 learned_addr learned my addr 172.21.15.3:0/85540084 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:35:44.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:35:44.972+0000 7f1036ffd640 1 --2- 172.21.15.3:0/85540084 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f104015e510 0x7f10400b4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:44.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:35:44.972+0000 7f103ffff640 1 --2- 172.21.15.3:0/85540084 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f10401625a0 0x7f10401f7510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:44.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:35:44.973+0000 7f103f7fe640 1 -- 172.21.15.3:0/85540084 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f104015e510 msgr2=0x7f10400b4de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:44.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:35:44.973+0000 7f103f7fe640 1 --2- 172.21.15.3:0/85540084 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f104015e510 0x7f10400b4de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:44.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:35:44.973+0000 7f103f7fe640 1 -- 172.21.15.3:0/85540084 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f10401625a0 msgr2=0x7f10401f7510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:44.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:35:44.973+0000 7f103f7fe640 1 --2- 172.21.15.3:0/85540084 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f10401625a0 0x7f10401f7510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:44.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:35:44.973+0000 7f103f7fe640 1 -- 172.21.15.3:0/85540084 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10400b73c0 con 0x7f104015e130 2024-05-09T06:35:44.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:35:44.973+0000 7f103f7fe640 1 --2- 172.21.15.3:0/85540084 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f104015e130 0x7f104014e580 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f103000c4d0 tx=0x7f103000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:35:44.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:35:44.973+0000 7f103cff9640 1 -- 172.21.15.3:0/85540084 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1030018070 con 0x7f104015e130 2024-05-09T06:35:44.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:35:44.973+0000 7f103cff9640 1 -- 172.21.15.3:0/85540084 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1030007500 con 0x7f104015e130 2024-05-09T06:35:44.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:35:44.973+0000 7f103cff9640 1 -- 172.21.15.3:0/85540084 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1030004a20 con 0x7f104015e130 2024-05-09T06:35:44.963 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:35:44.973+0000 7f1045e8c640 1 -- 172.21.15.3:0/85540084 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10400b76a0 con 0x7f104015e130 2024-05-09T06:35:44.963 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:35:44.973+0000 7f1045e8c640 1 -- 172.21.15.3:0/85540084 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10400b7c60 con 0x7f104015e130 2024-05-09T06:35:44.963 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:35:44.973+0000 7f103cff9640 1 -- 172.21.15.3:0/85540084 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f103001b070 con 0x7f104015e130 2024-05-09T06:35:44.963 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:35:44.973+0000 7f103cff9640 1 -- 172.21.15.3:0/85540084 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7f1030012ad0 con 0x7f104015e130 2024-05-09T06:35:44.963 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:35:44.974+0000 7f1045e8c640 1 -- 172.21.15.3:0/85540084 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10400b7e60 con 0x7f104015e130 2024-05-09T06:35:44.963 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:35:44.976+0000 7f103cff9640 1 -- 172.21.15.3:0/85540084 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f10400b7e60 con 0x7f104015e130 2024-05-09T06:35:45.020 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:35:45.039+0000 7f1045e8c640 1 -- 172.21.15.3:0/85540084 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f104004f660 con 0x7f104015e130 2024-05-09T06:35:45.020 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:35:45.040+0000 7f103cff9640 1 -- 172.21.15.3:0/85540084 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7f103001d440 con 0x7f104015e130 2024-05-09T06:35:45.021 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-09T06:35:45.021 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":10,"fsid":"16ed737d-2665-431a-8ac6-ef242be3975a","created":"2024-05-09T06:35:42.541368+0000","modified":"2024-05-09T06:35:44.780586+0000","last_up_change":"0.000000","last_in_change":"2024-05-09T06:35:44.780586+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"fae3a8db-4826-4f3c-b33a-12c09bb5df6a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"646e882d-4ce4-4720-8c84-f71df563a35e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"1abbadb5-a570-4359-9643-acff3ede35a1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"62ac4dc1-2371-4dd2-acd3-e2dbb11aedf5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"ae808398-8326-4d7c-9319-a2a0eafffb59","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"918adf50-c440-48ac-9a8d-4837562d7275","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"ac2882bc-2a7c-463c-8f6c-76ff49828adc","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"dd4e767a-4730-4c70-8bd5-3f088140a5a1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T06:35:45.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:35:45.040+0000 7f1045e8c640 1 -- 172.21.15.3:0/85540084 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f104015e130 msgr2=0x7f104014e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:45.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:35:45.040+0000 7f1045e8c640 1 --2- 172.21.15.3:0/85540084 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f104015e130 0x7f104014e580 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f103000c4d0 tx=0x7f103000c990 comp rx=0 tx=0).stop 2024-05-09T06:35:45.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:35:45.040+0000 7f1045e8c640 1 -- 172.21.15.3:0/85540084 shutdown_connections 2024-05-09T06:35:45.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:35:45.040+0000 7f1045e8c640 1 --2- 172.21.15.3:0/85540084 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f10401625a0 0x7f10401f7510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:45.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:35:45.040+0000 7f1045e8c640 1 --2- 172.21.15.3:0/85540084 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f104015e510 0x7f10400b4de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:45.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:35:45.040+0000 7f1045e8c640 1 --2- 172.21.15.3:0/85540084 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f104015e130 0x7f104014e580 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:45.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:35:45.040+0000 7f1045e8c640 1 -- 172.21.15.3:0/85540084 >> 172.21.15.3:0/85540084 conn(0x7f10400b3a80 msgr2=0x7f104015d460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:35:45.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:35:45.041+0000 7f1045e8c640 1 -- 172.21.15.3:0/85540084 shutdown_connections 2024-05-09T06:35:45.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:35:45.041+0000 7f1045e8c640 1 -- 172.21.15.3:0/85540084 wait complete. 2024-05-09T06:35:45.032 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-05-09T06:35:45.032 INFO:tasks.ceph:Waiting for OSDs to come up 2024-05-09T06:35:45.158 INFO:tasks.ceph.mgr.x.smithi190.stderr:2024-05-09T06:35:45.177+0000 7fab08831040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-09T06:35:45.211 INFO:tasks.ceph.mgr.x.smithi190.stderr:2024-05-09T06:35:45.230+0000 7fab08831040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-09T06:35:45.217 INFO:tasks.ceph.mgr.y.smithi003.stderr:2024-05-09T06:35:45.236+0000 7f733de11040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-09T06:35:45.272 INFO:tasks.ceph.mgr.y.smithi003.stderr:2024-05-09T06:35:45.291+0000 7f733de11040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-09T06:35:45.327 INFO:tasks.ceph.mgr.x.smithi190.stderr:2024-05-09T06:35:45.346+0000 7fab08831040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-09T06:35:45.388 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-05-09T06:35:45.407+0000 7f055b00d640 -1 Falling back to public interface 2024-05-09T06:35:45.391 INFO:tasks.ceph.osd.1.smithi003.stderr:2024-05-09T06:35:45.410+0000 7f18cbdcb640 -1 Falling back to public interface 2024-05-09T06:35:45.394 INFO:tasks.ceph.mgr.y.smithi003.stderr:2024-05-09T06:35:45.413+0000 7f733de11040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-09T06:35:45.398 INFO:tasks.ceph.osd.2.smithi003.stderr:2024-05-09T06:35:45.417+0000 7f5404616640 -1 Falling back to public interface 2024-05-09T06:35:45.403 INFO:tasks.ceph.osd.0.smithi003.stderr:2024-05-09T06:35:45.422+0000 7fb3041a5640 -1 Falling back to public interface 2024-05-09T06:35:45.411 INFO:tasks.ceph.osd.3.smithi003.stderr:2024-05-09T06:35:45.431+0000 7f92e19c4640 -1 Falling back to public interface 2024-05-09T06:35:45.418 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-05-09T06:35:45.438+0000 7f169700e640 -1 Falling back to public interface 2024-05-09T06:35:45.419 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-05-09T06:35:45.438+0000 7fd8e180f640 -1 Falling back to public interface 2024-05-09T06:35:45.419 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-05-09T06:35:45.438+0000 7fd8e0218640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6811/0: (98) Address already in use 2024-05-09T06:35:45.427 INFO:tasks.ceph.osd.6.smithi190.stderr:2024-05-09T06:35:45.446+0000 7fb207553640 -1 Falling back to public interface 2024-05-09T06:35:45.635 INFO:tasks.ceph.mgr.x.smithi190.stderr:2024-05-09T06:35:45.654+0000 7fab08831040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-09T06:35:45.684 INFO:tasks.ceph.mgr.x.smithi190.stderr:2024-05-09T06:35:45.703+0000 7fab08831040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-09T06:35:45.708 INFO:tasks.ceph.mgr.y.smithi003.stderr:2024-05-09T06:35:45.727+0000 7f733de11040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-09T06:35:45.758 INFO:tasks.ceph.mgr.y.smithi003.stderr:2024-05-09T06:35:45.777+0000 7f733de11040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-09T06:35:45.838 DEBUG:teuthology.orchestra.run.smithi003:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-05-09T06:35:45.841 INFO:tasks.ceph.mgr.x.smithi190.stderr:2024-05-09T06:35:45.861+0000 7fab08831040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-09T06:35:45.923 INFO:tasks.ceph.mgr.y.smithi003.stderr:2024-05-09T06:35:45.942+0000 7f733de11040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-09T06:35:45.933 INFO:teuthology.misc.health.smithi003.stderr:2024-05-09T06:35:45.952+0000 7fa6e1577640 1 Processor -- start 2024-05-09T06:35:45.933 INFO:teuthology.misc.health.smithi003.stderr:2024-05-09T06:35:45.952+0000 7fa6e1577640 1 -- start start 2024-05-09T06:35:45.934 INFO:teuthology.misc.health.smithi003.stderr:2024-05-09T06:35:45.953+0000 7fa6e1577640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa6dc0b4f80 0x7fa6dc0b5360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:45.934 INFO:teuthology.misc.health.smithi003.stderr:2024-05-09T06:35:45.953+0000 7fa6e1577640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fa6dc15fed0 0x7fa6dc1602e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:45.934 INFO:teuthology.misc.health.smithi003.stderr:2024-05-09T06:35:45.953+0000 7fa6e1577640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa6dc0b58a0 con 0x7fa6dc15e9b0 2024-05-09T06:35:45.935 INFO:teuthology.misc.health.smithi003.stderr:2024-05-09T06:35:45.953+0000 7fa6e1577640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fa6dc15fa40 con 0x7fa6dc0b4f80 2024-05-09T06:35:45.935 INFO:teuthology.misc.health.smithi003.stderr:2024-05-09T06:35:45.953+0000 7fa6e1577640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7fa6dc15ee30 con 0x7fa6dc15fed0 2024-05-09T06:35:45.935 INFO:teuthology.misc.health.smithi003.stderr:2024-05-09T06:35:45.953+0000 7fa6daffd640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa6dc0b4f80 0x7fa6dc0b5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:45.935 INFO:teuthology.misc.health.smithi003.stderr:2024-05-09T06:35:45.953+0000 7fa6daffd640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa6dc0b4f80 0x7fa6dc0b5360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:42152/0 (socket says 172.21.15.3:42152) 2024-05-09T06:35:45.935 INFO:teuthology.misc.health.smithi003.stderr:2024-05-09T06:35:45.953+0000 7fa6daffd640 1 -- 172.21.15.3:0/183770224 learned_addr learned my addr 172.21.15.3:0/183770224 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:35:45.935 INFO:teuthology.misc.health.smithi003.stderr:2024-05-09T06:35:45.953+0000 7fa6da7fc640 1 --1- 172.21.15.3:0/183770224 >> v1:172.21.15.3:6789/0 conn(0x7fa6dc15e9b0 0x7fa6dc06d400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:60790/0 (socket says 172.21.15.3:60790) 2024-05-09T06:35:45.935 INFO:teuthology.misc.health.smithi003.stderr:2024-05-09T06:35:45.953+0000 7fa6db7fe640 1 --2- 172.21.15.3:0/183770224 >> v2:172.21.15.190:3300/0 conn(0x7fa6dc15fed0 0x7fa6dc1602e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:45.935 INFO:teuthology.misc.health.smithi003.stderr:2024-05-09T06:35:45.953+0000 7fa6daffd640 1 -- 172.21.15.3:0/183770224 >> v2:172.21.15.190:3300/0 conn(0x7fa6dc15fed0 msgr2=0x7fa6dc1602e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:35:45.935 INFO:teuthology.misc.health.smithi003.stderr:2024-05-09T06:35:45.953+0000 7fa6daffd640 1 --2- 172.21.15.3:0/183770224 >> v2:172.21.15.190:3300/0 conn(0x7fa6dc15fed0 0x7fa6dc1602e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:45.935 INFO:teuthology.misc.health.smithi003.stderr:2024-05-09T06:35:45.953+0000 7fa6daffd640 1 -- 172.21.15.3:0/183770224 >> v1:172.21.15.3:6789/0 conn(0x7fa6dc15e9b0 legacy=0x7fa6dc06d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:45.935 INFO:teuthology.misc.health.smithi003.stderr:2024-05-09T06:35:45.953+0000 7fa6daffd640 1 -- 172.21.15.3:0/183770224 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6dc15f110 con 0x7fa6dc0b4f80 2024-05-09T06:35:45.935 INFO:teuthology.misc.health.smithi003.stderr:2024-05-09T06:35:45.953+0000 7fa6d9ffb640 1 -- 172.21.15.3:0/183770224 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3942031109 0 0) 0x7fa6dc0b58a0 con 0x7fa6dc15e9b0 2024-05-09T06:35:45.935 INFO:teuthology.misc.health.smithi003.stderr:2024-05-09T06:35:45.953+0000 7fa6db7fe640 1 --2- 172.21.15.3:0/183770224 >> v2:172.21.15.190:3300/0 conn(0x7fa6dc15fed0 0x7fa6dc1602e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:35:45.937 INFO:teuthology.misc.health.smithi003.stderr:2024-05-09T06:35:45.954+0000 7fa6daffd640 1 --2- 172.21.15.3:0/183770224 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa6dc0b4f80 0x7fa6dc0b5360 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fa6dc073350 tx=0x7fa6cc02a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc413a38eba9617b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:35:45.937 INFO:teuthology.misc.health.smithi003.stderr:2024-05-09T06:35:45.954+0000 7fa6d9ffb640 1 -- 172.21.15.3:0/183770224 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6cc037070 con 0x7fa6dc0b4f80 2024-05-09T06:35:45.937 INFO:teuthology.misc.health.smithi003.stderr:2024-05-09T06:35:45.954+0000 7fa6d9ffb640 1 -- 172.21.15.3:0/183770224 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6cc02ae50 con 0x7fa6dc0b4f80 2024-05-09T06:35:45.937 INFO:teuthology.misc.health.smithi003.stderr:2024-05-09T06:35:45.954+0000 7fa6d9ffb640 1 -- 172.21.15.3:0/183770224 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6cc032860 con 0x7fa6dc0b4f80 2024-05-09T06:35:45.937 INFO:teuthology.misc.health.smithi003.stderr:2024-05-09T06:35:45.955+0000 7fa6e1577640 1 -- 172.21.15.3:0/183770224 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa6dc0b4f80 msgr2=0x7fa6dc0b5360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:45.937 INFO:teuthology.misc.health.smithi003.stderr:2024-05-09T06:35:45.955+0000 7fa6e1577640 1 --2- 172.21.15.3:0/183770224 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa6dc0b4f80 0x7fa6dc0b5360 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fa6dc073350 tx=0x7fa6cc02a300 comp rx=0 tx=0).stop 2024-05-09T06:35:45.937 INFO:teuthology.misc.health.smithi003.stderr:2024-05-09T06:35:45.955+0000 7fa6e1577640 1 -- 172.21.15.3:0/183770224 shutdown_connections 2024-05-09T06:35:45.937 INFO:teuthology.misc.health.smithi003.stderr:2024-05-09T06:35:45.955+0000 7fa6e1577640 1 --2- 172.21.15.3:0/183770224 >> v2:172.21.15.190:3300/0 conn(0x7fa6dc15fed0 0x7fa6dc1602e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:45.937 INFO:teuthology.misc.health.smithi003.stderr:2024-05-09T06:35:45.955+0000 7fa6e1577640 1 --2- 172.21.15.3:0/183770224 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa6dc0b4f80 0x7fa6dc0b5360 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:45.937 INFO:teuthology.misc.health.smithi003.stderr:2024-05-09T06:35:45.955+0000 7fa6e1577640 1 -- 172.21.15.3:0/183770224 >> 172.21.15.3:0/183770224 conn(0x7fa6dc06f4e0 msgr2=0x7fa6dc06f8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:35:45.937 INFO:teuthology.misc.health.smithi003.stderr:2024-05-09T06:35:45.955+0000 7fa6e1577640 1 -- 172.21.15.3:0/183770224 shutdown_connections 2024-05-09T06:35:45.938 INFO:teuthology.misc.health.smithi003.stderr:2024-05-09T06:35:45.955+0000 7fa6e1577640 1 -- 172.21.15.3:0/183770224 wait complete. 2024-05-09T06:35:45.938 INFO:teuthology.misc.health.smithi003.stderr:2024-05-09T06:35:45.956+0000 7fa6e1577640 1 Processor -- start 2024-05-09T06:35:45.938 INFO:teuthology.misc.health.smithi003.stderr:2024-05-09T06:35:45.956+0000 7fa6e1577640 1 -- start start 2024-05-09T06:35:45.940 INFO:teuthology.misc.health.smithi003.stderr:2024-05-09T06:35:45.957+0000 7fa6e1577640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa6dc0b4f80 0x7fa6dc1f3d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:45.940 INFO:teuthology.misc.health.smithi003.stderr:2024-05-09T06:35:45.957+0000 7fa6e1577640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa6dc15e9b0 0x7fa6dc15dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:45.940 INFO:teuthology.misc.health.smithi003.stderr:2024-05-09T06:35:45.957+0000 7fa6e1577640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa6dc15fed0 0x7fa6dc0cf450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:45.940 INFO:teuthology.misc.health.smithi003.stderr:2024-05-09T06:35:45.957+0000 7fa6e1577640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fa6dc06db10 con 0x7fa6dc0b4f80 2024-05-09T06:35:45.940 INFO:teuthology.misc.health.smithi003.stderr:2024-05-09T06:35:45.957+0000 7fa6e1577640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fa6dc15f6d0 con 0x7fa6dc15fed0 2024-05-09T06:35:45.940 INFO:teuthology.misc.health.smithi003.stderr:2024-05-09T06:35:45.957+0000 7fa6e1577640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fa6dc147fb0 con 0x7fa6dc15e9b0 2024-05-09T06:35:45.940 INFO:teuthology.misc.health.smithi003.stderr:2024-05-09T06:35:45.957+0000 7fa6daffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa6dc0b4f80 0x7fa6dc1f3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:45.940 INFO:teuthology.misc.health.smithi003.stderr:2024-05-09T06:35:45.957+0000 7fa6daffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa6dc0b4f80 0x7fa6dc1f3d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:42166/0 (socket says 172.21.15.3:42166) 2024-05-09T06:35:45.940 INFO:teuthology.misc.health.smithi003.stderr:2024-05-09T06:35:45.957+0000 7fa6daffd640 1 -- 172.21.15.3:0/1202092856 learned_addr learned my addr 172.21.15.3:0/1202092856 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:35:45.940 INFO:teuthology.misc.health.smithi003.stderr:2024-05-09T06:35:45.957+0000 7fa6db7fe640 1 --2- 172.21.15.3:0/1202092856 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa6dc15fed0 0x7fa6dc0cf450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:45.940 INFO:teuthology.misc.health.smithi003.stderr:2024-05-09T06:35:45.957+0000 7fa6da7fc640 1 --2- 172.21.15.3:0/1202092856 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa6dc15e9b0 0x7fa6dc15dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:45.940 INFO:teuthology.misc.health.smithi003.stderr:2024-05-09T06:35:45.957+0000 7fa6daffd640 1 -- 172.21.15.3:0/1202092856 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa6dc15e9b0 msgr2=0x7fa6dc15dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:45.941 INFO:teuthology.misc.health.smithi003.stderr:2024-05-09T06:35:45.957+0000 7fa6daffd640 1 --2- 172.21.15.3:0/1202092856 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa6dc15e9b0 0x7fa6dc15dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:45.941 INFO:teuthology.misc.health.smithi003.stderr:2024-05-09T06:35:45.957+0000 7fa6daffd640 1 -- 172.21.15.3:0/1202092856 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa6dc15fed0 msgr2=0x7fa6dc0cf450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:45.941 INFO:teuthology.misc.health.smithi003.stderr:2024-05-09T06:35:45.957+0000 7fa6daffd640 1 --2- 172.21.15.3:0/1202092856 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa6dc15fed0 0x7fa6dc0cf450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:45.941 INFO:teuthology.misc.health.smithi003.stderr:2024-05-09T06:35:45.957+0000 7fa6daffd640 1 -- 172.21.15.3:0/1202092856 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6dc0cfdc0 con 0x7fa6dc0b4f80 2024-05-09T06:35:45.941 INFO:teuthology.misc.health.smithi003.stderr:2024-05-09T06:35:45.958+0000 7fa6daffd640 1 --2- 172.21.15.3:0/1202092856 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa6dc0b4f80 0x7fa6dc1f3d30 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fa6cc000c00 tx=0x7fa6cc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:35:45.941 INFO:teuthology.misc.health.smithi003.stderr:2024-05-09T06:35:45.958+0000 7fa6c37fe640 1 -- 172.21.15.3:0/1202092856 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6cc037040 con 0x7fa6dc0b4f80 2024-05-09T06:35:45.941 INFO:teuthology.misc.health.smithi003.stderr:2024-05-09T06:35:45.958+0000 7fa6c37fe640 1 -- 172.21.15.3:0/1202092856 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6cc004030 con 0x7fa6dc0b4f80 2024-05-09T06:35:45.941 INFO:teuthology.misc.health.smithi003.stderr:2024-05-09T06:35:45.958+0000 7fa6e1577640 1 -- 172.21.15.3:0/1202092856 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6dc0cc3f0 con 0x7fa6dc0b4f80 2024-05-09T06:35:45.941 INFO:teuthology.misc.health.smithi003.stderr:2024-05-09T06:35:45.958+0000 7fa6c37fe640 1 -- 172.21.15.3:0/1202092856 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6cc004400 con 0x7fa6dc0b4f80 2024-05-09T06:35:45.941 INFO:teuthology.misc.health.smithi003.stderr:2024-05-09T06:35:45.958+0000 7fa6e1577640 1 -- 172.21.15.3:0/1202092856 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6dc0cc9b0 con 0x7fa6dc0b4f80 2024-05-09T06:35:45.941 INFO:teuthology.misc.health.smithi003.stderr:2024-05-09T06:35:45.958+0000 7fa6c37fe640 1 -- 172.21.15.3:0/1202092856 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fa6cc03a070 con 0x7fa6dc0b4f80 2024-05-09T06:35:45.941 INFO:teuthology.misc.health.smithi003.stderr:2024-05-09T06:35:45.958+0000 7fa6c37fe640 1 -- 172.21.15.3:0/1202092856 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7fa6cc003d80 con 0x7fa6dc0b4f80 2024-05-09T06:35:45.941 INFO:teuthology.misc.health.smithi003.stderr:2024-05-09T06:35:45.959+0000 7fa6e1577640 1 -- 172.21.15.3:0/1202092856 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6dc15e290 con 0x7fa6dc0b4f80 2024-05-09T06:35:45.942 INFO:teuthology.misc.health.smithi003.stderr:2024-05-09T06:35:45.961+0000 7fa6c37fe640 1 -- 172.21.15.3:0/1202092856 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fa6cc041020 con 0x7fa6dc0b4f80 2024-05-09T06:35:45.983 INFO:tasks.ceph.mgr.x.smithi190.stderr:2024-05-09T06:35:46.000+0000 7fab08831040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-09T06:35:46.005 INFO:teuthology.misc.health.smithi003.stderr:2024-05-09T06:35:46.024+0000 7fa6e1577640 1 -- 172.21.15.3:0/1202092856 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa6dc04f660 con 0x7fa6dc0b4f80 2024-05-09T06:35:46.006 INFO:teuthology.misc.health.smithi003.stderr:2024-05-09T06:35:46.025+0000 7fa6c37fe640 1 -- 172.21.15.3:0/1202092856 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7fa6cc0036b0 con 0x7fa6dc0b4f80 2024-05-09T06:35:46.006 INFO:teuthology.misc.health.smithi003.stdout: 2024-05-09T06:35:46.006 INFO:teuthology.misc.health.smithi003.stdout:{"epoch":10,"fsid":"16ed737d-2665-431a-8ac6-ef242be3975a","created":"2024-05-09T06:35:42.541368+0000","modified":"2024-05-09T06:35:44.780586+0000","last_up_change":"0.000000","last_in_change":"2024-05-09T06:35:44.780586+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"fae3a8db-4826-4f3c-b33a-12c09bb5df6a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"646e882d-4ce4-4720-8c84-f71df563a35e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"1abbadb5-a570-4359-9643-acff3ede35a1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"62ac4dc1-2371-4dd2-acd3-e2dbb11aedf5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"ae808398-8326-4d7c-9319-a2a0eafffb59","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"918adf50-c440-48ac-9a8d-4837562d7275","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"ac2882bc-2a7c-463c-8f6c-76ff49828adc","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"dd4e767a-4730-4c70-8bd5-3f088140a5a1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T06:35:46.007 INFO:teuthology.misc.health.smithi003.stderr:2024-05-09T06:35:46.026+0000 7fa6e1577640 1 -- 172.21.15.3:0/1202092856 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa6dc0b4f80 msgr2=0x7fa6dc1f3d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:46.007 INFO:teuthology.misc.health.smithi003.stderr:2024-05-09T06:35:46.026+0000 7fa6e1577640 1 --2- 172.21.15.3:0/1202092856 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa6dc0b4f80 0x7fa6dc1f3d30 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fa6cc000c00 tx=0x7fa6cc0046c0 comp rx=0 tx=0).stop 2024-05-09T06:35:46.007 INFO:teuthology.misc.health.smithi003.stderr:2024-05-09T06:35:46.026+0000 7fa6e1577640 1 -- 172.21.15.3:0/1202092856 shutdown_connections 2024-05-09T06:35:46.007 INFO:teuthology.misc.health.smithi003.stderr:2024-05-09T06:35:46.026+0000 7fa6e1577640 1 --2- 172.21.15.3:0/1202092856 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa6dc15fed0 0x7fa6dc0cf450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:46.007 INFO:teuthology.misc.health.smithi003.stderr:2024-05-09T06:35:46.026+0000 7fa6e1577640 1 --2- 172.21.15.3:0/1202092856 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa6dc15e9b0 0x7fa6dc15dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:46.007 INFO:teuthology.misc.health.smithi003.stderr:2024-05-09T06:35:46.026+0000 7fa6e1577640 1 --2- 172.21.15.3:0/1202092856 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa6dc0b4f80 0x7fa6dc1f3d30 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:46.007 INFO:teuthology.misc.health.smithi003.stderr:2024-05-09T06:35:46.026+0000 7fa6e1577640 1 -- 172.21.15.3:0/1202092856 >> 172.21.15.3:0/1202092856 conn(0x7fa6dc06f4e0 msgr2=0x7fa6dc146e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:35:46.007 INFO:teuthology.misc.health.smithi003.stderr:2024-05-09T06:35:46.026+0000 7fa6e1577640 1 -- 172.21.15.3:0/1202092856 shutdown_connections 2024-05-09T06:35:46.008 INFO:teuthology.misc.health.smithi003.stderr:2024-05-09T06:35:46.026+0000 7fa6e1577640 1 -- 172.21.15.3:0/1202092856 wait complete. 2024-05-09T06:35:46.016 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-05-09T06:35:46.031 INFO:tasks.ceph.mgr.x.smithi190.stderr:2024-05-09T06:35:46.051+0000 7fab08831040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-09T06:35:46.070 INFO:tasks.ceph.mgr.y.smithi003.stderr:2024-05-09T06:35:46.089+0000 7f733de11040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-09T06:35:46.100 INFO:tasks.ceph.mgr.x.smithi190.stderr:2024-05-09T06:35:46.119+0000 7fab08831040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-09T06:35:46.122 INFO:tasks.ceph.mgr.y.smithi003.stderr:2024-05-09T06:35:46.141+0000 7f733de11040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-09T06:35:46.146 INFO:tasks.ceph.mgr.x.smithi190.stderr:2024-05-09T06:35:46.165+0000 7fab08831040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-09T06:35:46.196 INFO:tasks.ceph.mgr.y.smithi003.stderr:2024-05-09T06:35:46.215+0000 7f733de11040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-09T06:35:46.243 INFO:tasks.ceph.mgr.y.smithi003.stderr:2024-05-09T06:35:46.262+0000 7f733de11040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-09T06:35:46.626 INFO:tasks.ceph.mgr.x.smithi190.stderr:2024-05-09T06:35:46.645+0000 7fab08831040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-09T06:35:46.741 INFO:tasks.ceph.mgr.x.smithi190.stderr:2024-05-09T06:35:46.761+0000 7fab08831040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-09T06:35:46.745 INFO:tasks.ceph.mgr.y.smithi003.stderr:2024-05-09T06:35:46.765+0000 7f733de11040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-09T06:35:46.871 INFO:tasks.ceph.mgr.y.smithi003.stderr:2024-05-09T06:35:46.889+0000 7f733de11040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-09T06:35:47.758 INFO:tasks.ceph.mgr.x.smithi190.stderr:2024-05-09T06:35:47.777+0000 7faaa8ae6640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-05-09T06:35:47.758 INFO:tasks.ceph.mgr.x.smithi190.stderr:2024-05-09T06:35:47.778+0000 7faaa8ae6640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-05-09T06:35:48.866 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-05-09T06:35:48.884+0000 7f055b00d640 -1 osd.4 0 log_to_monitors true 2024-05-09T06:35:48.867 INFO:tasks.ceph.osd.0.smithi003.stderr:2024-05-09T06:35:48.887+0000 7fb3041a5640 -1 osd.0 0 log_to_monitors true 2024-05-09T06:35:48.907 INFO:tasks.ceph.osd.2.smithi003.stderr:2024-05-09T06:35:48.927+0000 7f5404616640 -1 osd.2 0 log_to_monitors true 2024-05-09T06:35:48.925 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-05-09T06:35:48.945+0000 7fd8e180f640 -1 osd.5 0 log_to_monitors true 2024-05-09T06:35:48.929 INFO:tasks.ceph.osd.3.smithi003.stderr:2024-05-09T06:35:48.949+0000 7f92e19c4640 -1 osd.3 0 log_to_monitors true 2024-05-09T06:35:48.935 INFO:tasks.ceph.osd.1.smithi003.stderr:2024-05-09T06:35:48.955+0000 7f18cbdcb640 -1 osd.1 0 log_to_monitors true 2024-05-09T06:35:48.940 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-05-09T06:35:48.960+0000 7f169700e640 -1 osd.7 0 log_to_monitors true 2024-05-09T06:35:48.947 INFO:tasks.ceph.osd.6.smithi190.stderr:2024-05-09T06:35:48.967+0000 7fb207553640 -1 osd.6 0 log_to_monitors true 2024-05-09T06:35:52.754 INFO:tasks.ceph.osd.6.smithi190.stderr:2024-05-09T06:35:52.773+0000 7fb202cc6640 -1 osd.6 0 waiting for initial osdmap 2024-05-09T06:35:52.764 INFO:tasks.ceph.osd.6.smithi190.stderr:2024-05-09T06:35:52.783+0000 7fb1fb2f8640 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-09T06:35:52.771 INFO:tasks.ceph.osd.1.smithi003.stderr:2024-05-09T06:35:52.790+0000 7f18c8d76640 -1 osd.1 0 waiting for initial osdmap 2024-05-09T06:35:52.783 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-05-09T06:35:52.803+0000 7f16921bc640 -1 osd.7 0 waiting for initial osdmap 2024-05-09T06:35:52.784 INFO:tasks.ceph.osd.1.smithi003.stderr:2024-05-09T06:35:52.804+0000 7f18bfb70640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-09T06:35:52.799 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-05-09T06:35:52.819+0000 7f168a7ee640 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-09T06:35:52.802 INFO:tasks.ceph.osd.0.smithi003.stderr:2024-05-09T06:35:52.821+0000 7fb2ff918640 -1 osd.0 0 waiting for initial osdmap 2024-05-09T06:35:52.811 INFO:tasks.ceph.osd.0.smithi003.stderr:2024-05-09T06:35:52.831+0000 7fb2f7f4a640 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-09T06:35:52.817 INFO:tasks.ceph.osd.3.smithi003.stderr:2024-05-09T06:35:52.836+0000 7f92dd137640 -1 osd.3 0 waiting for initial osdmap 2024-05-09T06:35:52.823 DEBUG:teuthology.orchestra.run.smithi003:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-05-09T06:35:52.828 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-05-09T06:35:52.847+0000 7f0556175640 -1 osd.4 0 waiting for initial osdmap 2024-05-09T06:35:52.828 INFO:tasks.ceph.osd.3.smithi003.stderr:2024-05-09T06:35:52.848+0000 7f92d5769640 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-09T06:35:52.840 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-05-09T06:35:52.860+0000 7f054e7a7640 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-09T06:35:52.857 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-05-09T06:35:52.876+0000 7fd8dc9dc640 -1 osd.5 0 waiting for initial osdmap 2024-05-09T06:35:52.868 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-05-09T06:35:52.887+0000 7fd8d500e640 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-09T06:35:52.880 INFO:tasks.ceph.osd.2.smithi003.stderr:2024-05-09T06:35:52.900+0000 7f53ff77d640 -1 osd.2 0 waiting for initial osdmap 2024-05-09T06:35:52.891 INFO:tasks.ceph.osd.2.smithi003.stderr:2024-05-09T06:35:52.911+0000 7f53f7daf640 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-09T06:35:52.928 INFO:teuthology.misc.health.smithi003.stderr:2024-05-09T06:35:52.947+0000 7fc998609640 1 Processor -- start 2024-05-09T06:35:52.928 INFO:teuthology.misc.health.smithi003.stderr:2024-05-09T06:35:52.947+0000 7fc998609640 1 -- start start 2024-05-09T06:35:52.930 INFO:teuthology.misc.health.smithi003.stderr:2024-05-09T06:35:52.948+0000 7fc998609640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fc990147d80 0x7fc99006c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:52.931 INFO:teuthology.misc.health.smithi003.stderr:2024-05-09T06:35:52.948+0000 7fc998609640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc99006cbe0 0x7fc9900b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:52.931 INFO:teuthology.misc.health.smithi003.stderr:2024-05-09T06:35:52.948+0000 7fc998609640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fc9900b54c0 0x7fc990161f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:52.931 INFO:teuthology.misc.health.smithi003.stderr:2024-05-09T06:35:52.948+0000 7fc998609640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fc990148160 con 0x7fc990147d80 2024-05-09T06:35:52.931 INFO:teuthology.misc.health.smithi003.stderr:2024-05-09T06:35:52.948+0000 7fc998609640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7fc9900b5910 con 0x7fc9900b54c0 2024-05-09T06:35:52.931 INFO:teuthology.misc.health.smithi003.stderr:2024-05-09T06:35:52.948+0000 7fc998609640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fc99006d1c0 con 0x7fc99006cbe0 2024-05-09T06:35:52.931 INFO:teuthology.misc.health.smithi003.stderr:2024-05-09T06:35:52.949+0000 7fc9963a5640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fc990147d80 0x7fc99006c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:52.931 INFO:teuthology.misc.health.smithi003.stderr:2024-05-09T06:35:52.949+0000 7fc9963a5640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fc990147d80 0x7fc99006c6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40378/0 (socket says 172.21.15.3:40378) 2024-05-09T06:35:52.931 INFO:teuthology.misc.health.smithi003.stderr:2024-05-09T06:35:52.949+0000 7fc9963a5640 1 -- 172.21.15.3:0/2327821472 learned_addr learned my addr 172.21.15.3:0/2327821472 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:35:52.931 INFO:teuthology.misc.health.smithi003.stderr:2024-05-09T06:35:52.949+0000 7fc995ba4640 1 --2- 172.21.15.3:0/2327821472 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc99006cbe0 0x7fc9900b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:52.931 INFO:teuthology.misc.health.smithi003.stderr:2024-05-09T06:35:52.949+0000 7fc996ba6640 1 --2- 172.21.15.3:0/2327821472 >> v2:172.21.15.190:3300/0 conn(0x7fc9900b54c0 0x7fc990161f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:52.931 INFO:teuthology.misc.health.smithi003.stderr:2024-05-09T06:35:52.949+0000 7fc9963a5640 1 -- 172.21.15.3:0/2327821472 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc99006cbe0 msgr2=0x7fc9900b4f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:35:52.931 INFO:teuthology.misc.health.smithi003.stderr:2024-05-09T06:35:52.949+0000 7fc9963a5640 1 --2- 172.21.15.3:0/2327821472 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc99006cbe0 0x7fc9900b4f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:52.931 INFO:teuthology.misc.health.smithi003.stderr:2024-05-09T06:35:52.949+0000 7fc9963a5640 1 -- 172.21.15.3:0/2327821472 >> v2:172.21.15.190:3300/0 conn(0x7fc9900b54c0 msgr2=0x7fc990161f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:35:52.931 INFO:teuthology.misc.health.smithi003.stderr:2024-05-09T06:35:52.949+0000 7fc9963a5640 1 --2- 172.21.15.3:0/2327821472 >> v2:172.21.15.190:3300/0 conn(0x7fc9900b54c0 0x7fc990161f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:52.931 INFO:teuthology.misc.health.smithi003.stderr:2024-05-09T06:35:52.949+0000 7fc9963a5640 1 -- 172.21.15.3:0/2327821472 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc99006d4a0 con 0x7fc990147d80 2024-05-09T06:35:52.932 INFO:teuthology.misc.health.smithi003.stderr:2024-05-09T06:35:52.950+0000 7fc9963a5640 1 --2- 172.21.15.3:0/2327821472 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc990147d80 0x7fc99006c6a0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fc990073350 tx=0x7fc98c02a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31b435ee15b11714 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:35:52.932 INFO:teuthology.misc.health.smithi003.stderr:2024-05-09T06:35:52.950+0000 7fc9953a3640 1 -- 172.21.15.3:0/2327821472 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc98c037070 con 0x7fc990147d80 2024-05-09T06:35:52.932 INFO:teuthology.misc.health.smithi003.stderr:2024-05-09T06:35:52.950+0000 7fc9953a3640 1 -- 172.21.15.3:0/2327821472 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc98c032440 con 0x7fc990147d80 2024-05-09T06:35:52.932 INFO:teuthology.misc.health.smithi003.stderr:2024-05-09T06:35:52.950+0000 7fc9953a3640 1 -- 172.21.15.3:0/2327821472 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc98c032810 con 0x7fc990147d80 2024-05-09T06:35:52.935 INFO:teuthology.misc.health.smithi003.stderr:2024-05-09T06:35:52.950+0000 7fc998609640 1 -- 172.21.15.3:0/2327821472 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc990147d80 msgr2=0x7fc99006c6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:52.935 INFO:teuthology.misc.health.smithi003.stderr:2024-05-09T06:35:52.950+0000 7fc998609640 1 --2- 172.21.15.3:0/2327821472 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc990147d80 0x7fc99006c6a0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fc990073350 tx=0x7fc98c02a140 comp rx=0 tx=0).stop 2024-05-09T06:35:52.935 INFO:teuthology.misc.health.smithi003.stderr:2024-05-09T06:35:52.950+0000 7fc998609640 1 -- 172.21.15.3:0/2327821472 shutdown_connections 2024-05-09T06:35:52.935 INFO:teuthology.misc.health.smithi003.stderr:2024-05-09T06:35:52.950+0000 7fc998609640 1 --2- 172.21.15.3:0/2327821472 >> v2:172.21.15.190:3300/0 conn(0x7fc9900b54c0 0x7fc990161f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:52.935 INFO:teuthology.misc.health.smithi003.stderr:2024-05-09T06:35:52.950+0000 7fc998609640 1 --2- 172.21.15.3:0/2327821472 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc99006cbe0 0x7fc9900b4f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:52.936 INFO:teuthology.misc.health.smithi003.stderr:2024-05-09T06:35:52.950+0000 7fc998609640 1 --2- 172.21.15.3:0/2327821472 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc990147d80 0x7fc99006c6a0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:52.936 INFO:teuthology.misc.health.smithi003.stderr:2024-05-09T06:35:52.950+0000 7fc998609640 1 -- 172.21.15.3:0/2327821472 >> 172.21.15.3:0/2327821472 conn(0x7fc99006f4e0 msgr2=0x7fc99006f8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:35:52.936 INFO:teuthology.misc.health.smithi003.stderr:2024-05-09T06:35:52.950+0000 7fc998609640 1 -- 172.21.15.3:0/2327821472 shutdown_connections 2024-05-09T06:35:52.936 INFO:teuthology.misc.health.smithi003.stderr:2024-05-09T06:35:52.950+0000 7fc998609640 1 -- 172.21.15.3:0/2327821472 wait complete. 2024-05-09T06:35:52.936 INFO:teuthology.misc.health.smithi003.stderr:2024-05-09T06:35:52.952+0000 7fc998609640 1 Processor -- start 2024-05-09T06:35:52.936 INFO:teuthology.misc.health.smithi003.stderr:2024-05-09T06:35:52.952+0000 7fc998609640 1 -- start start 2024-05-09T06:35:52.936 INFO:teuthology.misc.health.smithi003.stderr:2024-05-09T06:35:52.952+0000 7fc998609640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc99006cbe0 0x7fc99013ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:52.936 INFO:teuthology.misc.health.smithi003.stderr:2024-05-09T06:35:52.952+0000 7fc998609640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc9900b54c0 0x7fc9901408b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:52.936 INFO:teuthology.misc.health.smithi003.stderr:2024-05-09T06:35:52.952+0000 7fc998609640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc990147d80 0x7fc990140df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:52.936 INFO:teuthology.misc.health.smithi003.stderr:2024-05-09T06:35:52.952+0000 7fc998609640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fc990070e70 con 0x7fc99006cbe0 2024-05-09T06:35:52.936 INFO:teuthology.misc.health.smithi003.stderr:2024-05-09T06:35:52.952+0000 7fc998609640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fc99006da60 con 0x7fc9900b54c0 2024-05-09T06:35:52.936 INFO:teuthology.misc.health.smithi003.stderr:2024-05-09T06:35:52.952+0000 7fc998609640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fc99015e990 con 0x7fc990147d80 2024-05-09T06:35:52.936 INFO:teuthology.misc.health.smithi003.stderr:2024-05-09T06:35:52.952+0000 7fc9963a5640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc99006cbe0 0x7fc99013ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:52.936 INFO:teuthology.misc.health.smithi003.stderr:2024-05-09T06:35:52.952+0000 7fc9963a5640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc99006cbe0 0x7fc99013ce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40382/0 (socket says 172.21.15.3:40382) 2024-05-09T06:35:52.937 INFO:teuthology.misc.health.smithi003.stderr:2024-05-09T06:35:52.952+0000 7fc9963a5640 1 -- 172.21.15.3:0/2354201622 learned_addr learned my addr 172.21.15.3:0/2354201622 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:35:52.937 INFO:teuthology.misc.health.smithi003.stderr:2024-05-09T06:35:52.953+0000 7fc995ba4640 1 --2- 172.21.15.3:0/2354201622 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc9900b54c0 0x7fc9901408b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:52.937 INFO:teuthology.misc.health.smithi003.stderr:2024-05-09T06:35:52.953+0000 7fc996ba6640 1 --2- 172.21.15.3:0/2354201622 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc990147d80 0x7fc990140df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:52.937 INFO:teuthology.misc.health.smithi003.stderr:2024-05-09T06:35:52.953+0000 7fc9963a5640 1 -- 172.21.15.3:0/2354201622 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc990147d80 msgr2=0x7fc990140df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:52.937 INFO:teuthology.misc.health.smithi003.stderr:2024-05-09T06:35:52.953+0000 7fc9963a5640 1 --2- 172.21.15.3:0/2354201622 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc990147d80 0x7fc990140df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:52.937 INFO:teuthology.misc.health.smithi003.stderr:2024-05-09T06:35:52.953+0000 7fc9963a5640 1 -- 172.21.15.3:0/2354201622 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc9900b54c0 msgr2=0x7fc9901408b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:52.937 INFO:teuthology.misc.health.smithi003.stderr:2024-05-09T06:35:52.953+0000 7fc9963a5640 1 --2- 172.21.15.3:0/2354201622 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc9900b54c0 0x7fc9901408b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:52.937 INFO:teuthology.misc.health.smithi003.stderr:2024-05-09T06:35:52.953+0000 7fc9963a5640 1 -- 172.21.15.3:0/2354201622 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc99015e1d0 con 0x7fc99006cbe0 2024-05-09T06:35:52.937 INFO:teuthology.misc.health.smithi003.stderr:2024-05-09T06:35:52.953+0000 7fc9963a5640 1 --2- 172.21.15.3:0/2354201622 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc99006cbe0 0x7fc99013ce60 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fc98c02a180 tx=0x7fc98c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:35:52.937 INFO:teuthology.misc.health.smithi003.stderr:2024-05-09T06:35:52.953+0000 7fc986ffd640 1 -- 172.21.15.3:0/2354201622 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc98c041020 con 0x7fc99006cbe0 2024-05-09T06:35:52.937 INFO:teuthology.misc.health.smithi003.stderr:2024-05-09T06:35:52.953+0000 7fc986ffd640 1 -- 172.21.15.3:0/2354201622 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc98c03d620 con 0x7fc99006cbe0 2024-05-09T06:35:52.937 INFO:teuthology.misc.health.smithi003.stderr:2024-05-09T06:35:52.953+0000 7fc986ffd640 1 -- 172.21.15.3:0/2354201622 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc98c037040 con 0x7fc99006cbe0 2024-05-09T06:35:52.937 INFO:teuthology.misc.health.smithi003.stderr:2024-05-09T06:35:52.953+0000 7fc998609640 1 -- 172.21.15.3:0/2354201622 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc990141420 con 0x7fc99006cbe0 2024-05-09T06:35:52.937 INFO:teuthology.misc.health.smithi003.stderr:2024-05-09T06:35:52.953+0000 7fc998609640 1 -- 172.21.15.3:0/2354201622 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9901419e0 con 0x7fc99006cbe0 2024-05-09T06:35:52.937 INFO:teuthology.misc.health.smithi003.stderr:2024-05-09T06:35:52.954+0000 7fc998609640 1 -- 172.21.15.3:0/2354201622 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc99006dd40 con 0x7fc99006cbe0 2024-05-09T06:35:52.938 INFO:teuthology.misc.health.smithi003.stderr:2024-05-09T06:35:52.955+0000 7fc986ffd640 1 -- 172.21.15.3:0/2354201622 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 4) ==== 90442+0+0 (secure 0 0 0) 0x7fc98c044020 con 0x7fc99006cbe0 2024-05-09T06:35:52.938 INFO:teuthology.misc.health.smithi003.stderr:2024-05-09T06:35:52.955+0000 7fc986ffd640 1 --2- 172.21.15.3:0/2354201622 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fc97406bda0 0x7fc97406e260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:35:52.938 INFO:teuthology.misc.health.smithi003.stderr:2024-05-09T06:35:52.955+0000 7fc986ffd640 1 -- 172.21.15.3:0/2354201622 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(12..12 src has 1..12) ==== 2231+0+0 (secure 0 0 0) 0x7fc98c0aa4a0 con 0x7fc99006cbe0 2024-05-09T06:35:52.938 INFO:teuthology.misc.health.smithi003.stderr:2024-05-09T06:35:52.955+0000 7fc995ba4640 1 --2- 172.21.15.3:0/2354201622 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fc97406bda0 0x7fc97406e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:35:52.938 INFO:teuthology.misc.health.smithi003.stderr:2024-05-09T06:35:52.956+0000 7fc995ba4640 1 --2- 172.21.15.3:0/2354201622 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fc97406bda0 0x7fc97406e260 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fc990140630 tx=0x7fc980009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:35:52.940 INFO:teuthology.misc.health.smithi003.stderr:2024-05-09T06:35:52.960+0000 7fc986ffd640 1 -- 172.21.15.3:0/2354201622 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fc99006dd40 con 0x7fc99006cbe0 2024-05-09T06:35:53.126 INFO:teuthology.misc.health.smithi003.stderr:2024-05-09T06:35:53.145+0000 7fc998609640 1 -- 172.21.15.3:0/2354201622 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc990162df0 con 0x7fc99006cbe0 2024-05-09T06:35:53.126 INFO:teuthology.misc.health.smithi003.stderr:2024-05-09T06:35:53.145+0000 7fc986ffd640 1 -- 172.21.15.3:0/2354201622 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v12) ==== 74+0+7058 (secure 0 0 0) 0x7fc98c04b030 con 0x7fc99006cbe0 2024-05-09T06:35:53.126 INFO:teuthology.misc.health.smithi003.stdout: 2024-05-09T06:35:53.127 INFO:teuthology.misc.health.smithi003.stdout:{"epoch":12,"fsid":"16ed737d-2665-431a-8ac6-ef242be3975a","created":"2024-05-09T06:35:42.541368+0000","modified":"2024-05-09T06:35:50.783179+0000","last_up_change":"0.000000","last_in_change":"2024-05-09T06:35:44.780586+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"fae3a8db-4826-4f3c-b33a-12c09bb5df6a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"646e882d-4ce4-4720-8c84-f71df563a35e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"1abbadb5-a570-4359-9643-acff3ede35a1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"62ac4dc1-2371-4dd2-acd3-e2dbb11aedf5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"ae808398-8326-4d7c-9319-a2a0eafffb59","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"918adf50-c440-48ac-9a8d-4837562d7275","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"ac2882bc-2a7c-463c-8f6c-76ff49828adc","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"dd4e767a-4730-4c70-8bd5-3f088140a5a1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T06:35:53.128 INFO:teuthology.misc.health.smithi003.stderr:2024-05-09T06:35:53.147+0000 7fc998609640 1 -- 172.21.15.3:0/2354201622 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fc97406bda0 msgr2=0x7fc97406e260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:53.128 INFO:teuthology.misc.health.smithi003.stderr:2024-05-09T06:35:53.147+0000 7fc998609640 1 --2- 172.21.15.3:0/2354201622 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fc97406bda0 0x7fc97406e260 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fc990140630 tx=0x7fc980009190 comp rx=0 tx=0).stop 2024-05-09T06:35:53.128 INFO:teuthology.misc.health.smithi003.stderr:2024-05-09T06:35:53.147+0000 7fc998609640 1 -- 172.21.15.3:0/2354201622 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc99006cbe0 msgr2=0x7fc99013ce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:35:53.128 INFO:teuthology.misc.health.smithi003.stderr:2024-05-09T06:35:53.147+0000 7fc998609640 1 --2- 172.21.15.3:0/2354201622 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc99006cbe0 0x7fc99013ce60 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fc98c02a180 tx=0x7fc98c0046c0 comp rx=0 tx=0).stop 2024-05-09T06:35:53.129 INFO:teuthology.misc.health.smithi003.stderr:2024-05-09T06:35:53.147+0000 7fc998609640 1 -- 172.21.15.3:0/2354201622 shutdown_connections 2024-05-09T06:35:53.129 INFO:teuthology.misc.health.smithi003.stderr:2024-05-09T06:35:53.147+0000 7fc998609640 1 --2- 172.21.15.3:0/2354201622 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fc97406bda0 0x7fc97406e260 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:53.129 INFO:teuthology.misc.health.smithi003.stderr:2024-05-09T06:35:53.147+0000 7fc998609640 1 --2- 172.21.15.3:0/2354201622 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc990147d80 0x7fc990140df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:53.129 INFO:teuthology.misc.health.smithi003.stderr:2024-05-09T06:35:53.147+0000 7fc998609640 1 --2- 172.21.15.3:0/2354201622 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc9900b54c0 0x7fc9901408b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:53.129 INFO:teuthology.misc.health.smithi003.stderr:2024-05-09T06:35:53.147+0000 7fc998609640 1 --2- 172.21.15.3:0/2354201622 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc99006cbe0 0x7fc99013ce60 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:35:53.129 INFO:teuthology.misc.health.smithi003.stderr:2024-05-09T06:35:53.147+0000 7fc998609640 1 -- 172.21.15.3:0/2354201622 >> 172.21.15.3:0/2354201622 conn(0x7fc99006f4e0 msgr2=0x7fc990143bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:35:53.129 INFO:teuthology.misc.health.smithi003.stderr:2024-05-09T06:35:53.147+0000 7fc998609640 1 -- 172.21.15.3:0/2354201622 shutdown_connections 2024-05-09T06:35:53.129 INFO:teuthology.misc.health.smithi003.stderr:2024-05-09T06:35:53.147+0000 7fc998609640 1 -- 172.21.15.3:0/2354201622 wait complete. 2024-05-09T06:35:53.138 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-05-09T06:35:59.945 DEBUG:teuthology.orchestra.run.smithi003:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-05-09T06:36:00.051 INFO:teuthology.misc.health.smithi003.stderr:2024-05-09T06:36:00.071+0000 7fbefd67d640 1 Processor -- start 2024-05-09T06:36:00.052 INFO:teuthology.misc.health.smithi003.stderr:2024-05-09T06:36:00.071+0000 7fbefd67d640 1 -- start start 2024-05-09T06:36:00.053 INFO:teuthology.misc.health.smithi003.stderr:2024-05-09T06:36:00.071+0000 7fbefd67d640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fbef815e6a0 0x7fbef81623c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:00.053 INFO:teuthology.misc.health.smithi003.stderr:2024-05-09T06:36:00.071+0000 7fbefd67d640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbef815ea80 0x7fbef8162900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:00.053 INFO:teuthology.misc.health.smithi003.stderr:2024-05-09T06:36:00.071+0000 7fbefd67d640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fbef80d4060 0x7fbef80d44e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:00.053 INFO:teuthology.misc.health.smithi003.stderr:2024-05-09T06:36:00.071+0000 7fbefd67d640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fbef80b5980 con 0x7fbef815e6a0 2024-05-09T06:36:00.053 INFO:teuthology.misc.health.smithi003.stderr:2024-05-09T06:36:00.071+0000 7fbefd67d640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7fbef80d4a20 con 0x7fbef80d4060 2024-05-09T06:36:00.053 INFO:teuthology.misc.health.smithi003.stderr:2024-05-09T06:36:00.071+0000 7fbefd67d640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fbef8163280 con 0x7fbef815ea80 2024-05-09T06:36:00.053 INFO:teuthology.misc.health.smithi003.stderr:2024-05-09T06:36:00.072+0000 7fbef6ffd640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fbef815e6a0 0x7fbef81623c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:00.053 INFO:teuthology.misc.health.smithi003.stderr:2024-05-09T06:36:00.072+0000 7fbef6ffd640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fbef815e6a0 0x7fbef81623c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40390/0 (socket says 172.21.15.3:40390) 2024-05-09T06:36:00.053 INFO:teuthology.misc.health.smithi003.stderr:2024-05-09T06:36:00.072+0000 7fbef6ffd640 1 -- 172.21.15.3:0/2991312767 learned_addr learned my addr 172.21.15.3:0/2991312767 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:36:00.053 INFO:teuthology.misc.health.smithi003.stderr:2024-05-09T06:36:00.072+0000 7fbef6ffd640 1 -- 172.21.15.3:0/2991312767 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbef815ea80 msgr2=0x7fbef8162900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:00.054 INFO:teuthology.misc.health.smithi003.stderr:2024-05-09T06:36:00.072+0000 7fbef6ffd640 1 --2- 172.21.15.3:0/2991312767 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbef815ea80 0x7fbef8162900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:00.054 INFO:teuthology.misc.health.smithi003.stderr:2024-05-09T06:36:00.072+0000 7fbef6ffd640 1 -- 172.21.15.3:0/2991312767 >> v2:172.21.15.190:3300/0 conn(0x7fbef80d4060 msgr2=0x7fbef80d44e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:00.054 INFO:teuthology.misc.health.smithi003.stderr:2024-05-09T06:36:00.072+0000 7fbef77fe640 1 --2- 172.21.15.3:0/2991312767 >> v2:172.21.15.190:3300/0 conn(0x7fbef80d4060 0x7fbef80d44e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:00.054 INFO:teuthology.misc.health.smithi003.stderr:2024-05-09T06:36:00.072+0000 7fbef6ffd640 1 --2- 172.21.15.3:0/2991312767 >> v2:172.21.15.190:3300/0 conn(0x7fbef80d4060 0x7fbef80d44e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:00.054 INFO:teuthology.misc.health.smithi003.stderr:2024-05-09T06:36:00.072+0000 7fbef6ffd640 1 -- 172.21.15.3:0/2991312767 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbef8162fa0 con 0x7fbef815e6a0 2024-05-09T06:36:00.055 INFO:teuthology.misc.health.smithi003.stderr:2024-05-09T06:36:00.073+0000 7fbef6ffd640 1 --2- 172.21.15.3:0/2991312767 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbef815e6a0 0x7fbef81623c0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fbef808dae0 tx=0x7fbee802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e18b15df5e81859 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:00.055 INFO:teuthology.misc.health.smithi003.stderr:2024-05-09T06:36:00.073+0000 7fbef5ffb640 1 -- 172.21.15.3:0/2991312767 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbee8037070 con 0x7fbef815e6a0 2024-05-09T06:36:00.055 INFO:teuthology.misc.health.smithi003.stderr:2024-05-09T06:36:00.073+0000 7fbef5ffb640 1 -- 172.21.15.3:0/2991312767 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbee8032440 con 0x7fbef815e6a0 2024-05-09T06:36:00.055 INFO:teuthology.misc.health.smithi003.stderr:2024-05-09T06:36:00.073+0000 7fbef5ffb640 1 -- 172.21.15.3:0/2991312767 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbee8032810 con 0x7fbef815e6a0 2024-05-09T06:36:00.055 INFO:teuthology.misc.health.smithi003.stderr:2024-05-09T06:36:00.073+0000 7fbefd67d640 1 -- 172.21.15.3:0/2991312767 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbef815e6a0 msgr2=0x7fbef81623c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:00.055 INFO:teuthology.misc.health.smithi003.stderr:2024-05-09T06:36:00.073+0000 7fbefd67d640 1 --2- 172.21.15.3:0/2991312767 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbef815e6a0 0x7fbef81623c0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fbef808dae0 tx=0x7fbee802a1e0 comp rx=0 tx=0).stop 2024-05-09T06:36:00.055 INFO:teuthology.misc.health.smithi003.stderr:2024-05-09T06:36:00.073+0000 7fbefd67d640 1 -- 172.21.15.3:0/2991312767 shutdown_connections 2024-05-09T06:36:00.056 INFO:teuthology.misc.health.smithi003.stderr:2024-05-09T06:36:00.073+0000 7fbefd67d640 1 --2- 172.21.15.3:0/2991312767 >> v2:172.21.15.190:3300/0 conn(0x7fbef80d4060 0x7fbef80d44e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:00.056 INFO:teuthology.misc.health.smithi003.stderr:2024-05-09T06:36:00.073+0000 7fbefd67d640 1 --2- 172.21.15.3:0/2991312767 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbef815ea80 0x7fbef8162900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:00.056 INFO:teuthology.misc.health.smithi003.stderr:2024-05-09T06:36:00.073+0000 7fbefd67d640 1 --2- 172.21.15.3:0/2991312767 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbef815e6a0 0x7fbef81623c0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:00.056 INFO:teuthology.misc.health.smithi003.stderr:2024-05-09T06:36:00.073+0000 7fbefd67d640 1 -- 172.21.15.3:0/2991312767 >> 172.21.15.3:0/2991312767 conn(0x7fbef808dd40 msgr2=0x7fbef809d660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:00.056 INFO:teuthology.misc.health.smithi003.stderr:2024-05-09T06:36:00.074+0000 7fbefd67d640 1 -- 172.21.15.3:0/2991312767 shutdown_connections 2024-05-09T06:36:00.056 INFO:teuthology.misc.health.smithi003.stderr:2024-05-09T06:36:00.074+0000 7fbefd67d640 1 -- 172.21.15.3:0/2991312767 wait complete. 2024-05-09T06:36:00.058 INFO:teuthology.misc.health.smithi003.stderr:2024-05-09T06:36:00.075+0000 7fbefd67d640 1 Processor -- start 2024-05-09T06:36:00.059 INFO:teuthology.misc.health.smithi003.stderr:2024-05-09T06:36:00.075+0000 7fbefd67d640 1 -- start start 2024-05-09T06:36:00.059 INFO:teuthology.misc.health.smithi003.stderr:2024-05-09T06:36:00.076+0000 7fbefd67d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbef80d4060 0x7fbef815f990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:00.059 INFO:teuthology.misc.health.smithi003.stderr:2024-05-09T06:36:00.076+0000 7fbefd67d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbef815e6a0 0x7fbef8159510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:00.059 INFO:teuthology.misc.health.smithi003.stderr:2024-05-09T06:36:00.076+0000 7fbefd67d640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbef815ea80 0x7fbef81533d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:00.059 INFO:teuthology.misc.health.smithi003.stderr:2024-05-09T06:36:00.076+0000 7fbefd67d640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fbef80cf8c0 con 0x7fbef815e6a0 2024-05-09T06:36:00.059 INFO:teuthology.misc.health.smithi003.stderr:2024-05-09T06:36:00.076+0000 7fbefd67d640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fbef80cf5e0 con 0x7fbef80d4060 2024-05-09T06:36:00.059 INFO:teuthology.misc.health.smithi003.stderr:2024-05-09T06:36:00.076+0000 7fbefd67d640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fbef80d28e0 con 0x7fbef815ea80 2024-05-09T06:36:00.059 INFO:teuthology.misc.health.smithi003.stderr:2024-05-09T06:36:00.076+0000 7fbef67fc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbef815e6a0 0x7fbef8159510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:00.059 INFO:teuthology.misc.health.smithi003.stderr:2024-05-09T06:36:00.076+0000 7fbef67fc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbef815e6a0 0x7fbef8159510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40400/0 (socket says 172.21.15.3:40400) 2024-05-09T06:36:00.059 INFO:teuthology.misc.health.smithi003.stderr:2024-05-09T06:36:00.076+0000 7fbef67fc640 1 -- 172.21.15.3:0/1730687355 learned_addr learned my addr 172.21.15.3:0/1730687355 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:36:00.059 INFO:teuthology.misc.health.smithi003.stderr:2024-05-09T06:36:00.076+0000 7fbef6ffd640 1 --2- 172.21.15.3:0/1730687355 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbef80d4060 0x7fbef815f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:00.059 INFO:teuthology.misc.health.smithi003.stderr:2024-05-09T06:36:00.076+0000 7fbef77fe640 1 --2- 172.21.15.3:0/1730687355 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbef815ea80 0x7fbef81533d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:00.059 INFO:teuthology.misc.health.smithi003.stderr:2024-05-09T06:36:00.076+0000 7fbef67fc640 1 -- 172.21.15.3:0/1730687355 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbef815ea80 msgr2=0x7fbef81533d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:00.059 INFO:teuthology.misc.health.smithi003.stderr:2024-05-09T06:36:00.076+0000 7fbef67fc640 1 --2- 172.21.15.3:0/1730687355 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbef815ea80 0x7fbef81533d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:00.060 INFO:teuthology.misc.health.smithi003.stderr:2024-05-09T06:36:00.076+0000 7fbef67fc640 1 -- 172.21.15.3:0/1730687355 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbef80d4060 msgr2=0x7fbef815f990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:00.060 INFO:teuthology.misc.health.smithi003.stderr:2024-05-09T06:36:00.076+0000 7fbef67fc640 1 --2- 172.21.15.3:0/1730687355 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbef80d4060 0x7fbef815f990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:00.060 INFO:teuthology.misc.health.smithi003.stderr:2024-05-09T06:36:00.076+0000 7fbef67fc640 1 -- 172.21.15.3:0/1730687355 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbef815f0d0 con 0x7fbef815e6a0 2024-05-09T06:36:00.060 INFO:teuthology.misc.health.smithi003.stderr:2024-05-09T06:36:00.076+0000 7fbef67fc640 1 --2- 172.21.15.3:0/1730687355 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbef815e6a0 0x7fbef8159510 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fbeec00b410 tx=0x7fbeec00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:00.060 INFO:teuthology.misc.health.smithi003.stderr:2024-05-09T06:36:00.077+0000 7fbedf7fe640 1 -- 172.21.15.3:0/1730687355 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbeec015020 con 0x7fbef815e6a0 2024-05-09T06:36:00.060 INFO:teuthology.misc.health.smithi003.stderr:2024-05-09T06:36:00.077+0000 7fbedf7fe640 1 -- 172.21.15.3:0/1730687355 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbeec011850 con 0x7fbef815e6a0 2024-05-09T06:36:00.060 INFO:teuthology.misc.health.smithi003.stderr:2024-05-09T06:36:00.077+0000 7fbefd67d640 1 -- 172.21.15.3:0/1730687355 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbef815f3b0 con 0x7fbef815e6a0 2024-05-09T06:36:00.060 INFO:teuthology.misc.health.smithi003.stderr:2024-05-09T06:36:00.077+0000 7fbedf7fe640 1 -- 172.21.15.3:0/1730687355 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbeec011c40 con 0x7fbef815e6a0 2024-05-09T06:36:00.060 INFO:teuthology.misc.health.smithi003.stderr:2024-05-09T06:36:00.077+0000 7fbefd67d640 1 -- 172.21.15.3:0/1730687355 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbef8153910 con 0x7fbef815e6a0 2024-05-09T06:36:00.060 INFO:teuthology.misc.health.smithi003.stderr:2024-05-09T06:36:00.078+0000 7fbefd67d640 1 -- 172.21.15.3:0/1730687355 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbef80b5580 con 0x7fbef815e6a0 2024-05-09T06:36:00.061 INFO:teuthology.misc.health.smithi003.stderr:2024-05-09T06:36:00.078+0000 7fbedf7fe640 1 -- 172.21.15.3:0/1730687355 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fbeec018020 con 0x7fbef815e6a0 2024-05-09T06:36:00.061 INFO:teuthology.misc.health.smithi003.stderr:2024-05-09T06:36:00.079+0000 7fbedf7fe640 1 --2- 172.21.15.3:0/1730687355 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fbee406be30 0x7fbee406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:00.061 INFO:teuthology.misc.health.smithi003.stderr:2024-05-09T06:36:00.079+0000 7fbedf7fe640 1 -- 172.21.15.3:0/1730687355 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7fbeec08a970 con 0x7fbef815e6a0 2024-05-09T06:36:00.061 INFO:teuthology.misc.health.smithi003.stderr:2024-05-09T06:36:00.079+0000 7fbef6ffd640 1 --2- 172.21.15.3:0/1730687355 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fbee406be30 0x7fbee406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:00.061 INFO:teuthology.misc.health.smithi003.stderr:2024-05-09T06:36:00.080+0000 7fbef6ffd640 1 --2- 172.21.15.3:0/1730687355 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fbee406be30 0x7fbee406e2f0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fbee8034000 tx=0x7fbee802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:00.064 INFO:teuthology.misc.health.smithi003.stderr:2024-05-09T06:36:00.083+0000 7fbedf7fe640 1 -- 172.21.15.3:0/1730687355 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fbef80b5580 con 0x7fbef815e6a0 2024-05-09T06:36:00.248 INFO:teuthology.misc.health.smithi003.stderr:2024-05-09T06:36:00.267+0000 7fbefd67d640 1 -- 172.21.15.3:0/1730687355 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbef804f660 con 0x7fbef815e6a0 2024-05-09T06:36:00.249 INFO:teuthology.misc.health.smithi003.stderr:2024-05-09T06:36:00.268+0000 7fbedf7fe640 1 -- 172.21.15.3:0/1730687355 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) ==== 74+0+12800 (secure 0 0 0) 0x7fbeec01f020 con 0x7fbef815e6a0 2024-05-09T06:36:00.249 INFO:teuthology.misc.health.smithi003.stdout: 2024-05-09T06:36:00.249 INFO:teuthology.misc.health.smithi003.stdout:{"epoch":16,"fsid":"16ed737d-2665-431a-8ac6-ef242be3975a","created":"2024-05-09T06:35:42.541368+0000","modified":"2024-05-09T06:35:56.885471+0000","last_up_change":"2024-05-09T06:35:53.778274+0000","last_in_change":"2024-05-09T06:35:44.780586+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T06:35:54.819999+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fae3a8db-4826-4f3c-b33a-12c09bb5df6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":1897849827},{"type":"v1","addr":"172.21.15.3:6817","nonce":1897849827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":1897849827},{"type":"v1","addr":"172.21.15.3:6819","nonce":1897849827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":1897849827},{"type":"v1","addr":"172.21.15.3:6823","nonce":1897849827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":1897849827},{"type":"v1","addr":"172.21.15.3:6821","nonce":1897849827}]},"public_addr":"172.21.15.3:6817/1897849827","cluster_addr":"172.21.15.3:6819/1897849827","heartbeat_back_addr":"172.21.15.3:6823/1897849827","heartbeat_front_addr":"172.21.15.3:6821/1897849827","state":["exists","up"]},{"osd":1,"uuid":"646e882d-4ce4-4720-8c84-f71df563a35e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":926805498},{"type":"v1","addr":"172.21.15.3:6801","nonce":926805498}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":926805498},{"type":"v1","addr":"172.21.15.3:6803","nonce":926805498}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":926805498},{"type":"v1","addr":"172.21.15.3:6807","nonce":926805498}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":926805498},{"type":"v1","addr":"172.21.15.3:6805","nonce":926805498}]},"public_addr":"172.21.15.3:6801/926805498","cluster_addr":"172.21.15.3:6803/926805498","heartbeat_back_addr":"172.21.15.3:6807/926805498","heartbeat_front_addr":"172.21.15.3:6805/926805498","state":["exists","up"]},{"osd":2,"uuid":"1abbadb5-a570-4359-9643-acff3ede35a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":801295451},{"type":"v1","addr":"172.21.15.3:6809","nonce":801295451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":801295451},{"type":"v1","addr":"172.21.15.3:6811","nonce":801295451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":801295451},{"type":"v1","addr":"172.21.15.3:6815","nonce":801295451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":801295451},{"type":"v1","addr":"172.21.15.3:6813","nonce":801295451}]},"public_addr":"172.21.15.3:6809/801295451","cluster_addr":"172.21.15.3:6811/801295451","heartbeat_back_addr":"172.21.15.3:6815/801295451","heartbeat_front_addr":"172.21.15.3:6813/801295451","state":["exists","up"]},{"osd":3,"uuid":"62ac4dc1-2371-4dd2-acd3-e2dbb11aedf5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":1903389857},{"type":"v1","addr":"172.21.15.3:6825","nonce":1903389857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":1903389857},{"type":"v1","addr":"172.21.15.3:6827","nonce":1903389857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":1903389857},{"type":"v1","addr":"172.21.15.3:6831","nonce":1903389857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":1903389857},{"type":"v1","addr":"172.21.15.3:6829","nonce":1903389857}]},"public_addr":"172.21.15.3:6825/1903389857","cluster_addr":"172.21.15.3:6827/1903389857","heartbeat_back_addr":"172.21.15.3:6831/1903389857","heartbeat_front_addr":"172.21.15.3:6829/1903389857","state":["exists","up"]},{"osd":4,"uuid":"ae808398-8326-4d7c-9319-a2a0eafffb59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":2143624576},{"type":"v1","addr":"172.21.15.190:6801","nonce":2143624576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":2143624576},{"type":"v1","addr":"172.21.15.190:6803","nonce":2143624576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":2143624576},{"type":"v1","addr":"172.21.15.190:6807","nonce":2143624576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":2143624576},{"type":"v1","addr":"172.21.15.190:6805","nonce":2143624576}]},"public_addr":"172.21.15.190:6801/2143624576","cluster_addr":"172.21.15.190:6803/2143624576","heartbeat_back_addr":"172.21.15.190:6807/2143624576","heartbeat_front_addr":"172.21.15.190:6805/2143624576","state":["exists","up"]},{"osd":5,"uuid":"918adf50-c440-48ac-9a8d-4837562d7275","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":1766923126},{"type":"v1","addr":"172.21.15.190:6814","nonce":1766923126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":1766923126},{"type":"v1","addr":"172.21.15.190:6818","nonce":1766923126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":1766923126},{"type":"v1","addr":"172.21.15.190:6823","nonce":1766923126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":1766923126},{"type":"v1","addr":"172.21.15.190:6821","nonce":1766923126}]},"public_addr":"172.21.15.190:6814/1766923126","cluster_addr":"172.21.15.190:6818/1766923126","heartbeat_back_addr":"172.21.15.190:6823/1766923126","heartbeat_front_addr":"172.21.15.190:6821/1766923126","state":["exists","up"]},{"osd":6,"uuid":"ac2882bc-2a7c-463c-8f6c-76ff49828adc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":342414248},{"type":"v1","addr":"172.21.15.190:6825","nonce":342414248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":342414248},{"type":"v1","addr":"172.21.15.190:6827","nonce":342414248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":342414248},{"type":"v1","addr":"172.21.15.190:6831","nonce":342414248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":342414248},{"type":"v1","addr":"172.21.15.190:6829","nonce":342414248}]},"public_addr":"172.21.15.190:6825/342414248","cluster_addr":"172.21.15.190:6827/342414248","heartbeat_back_addr":"172.21.15.190:6831/342414248","heartbeat_front_addr":"172.21.15.190:6829/342414248","state":["exists","up"]},{"osd":7,"uuid":"dd4e767a-4730-4c70-8bd5-3f088140a5a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":537849891},{"type":"v1","addr":"172.21.15.190:6809","nonce":537849891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":537849891},{"type":"v1","addr":"172.21.15.190:6811","nonce":537849891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6817","nonce":537849891},{"type":"v1","addr":"172.21.15.190:6819","nonce":537849891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6813","nonce":537849891},{"type":"v1","addr":"172.21.15.190:6815","nonce":537849891}]},"public_addr":"172.21.15.190:6809/537849891","cluster_addr":"172.21.15.190:6811/537849891","heartbeat_back_addr":"172.21.15.190:6819/537849891","heartbeat_front_addr":"172.21.15.190:6815/537849891","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.889784+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.974002+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.973549+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.955396+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.928471+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.902418+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.958696+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.926474+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T06:36:00.251 INFO:teuthology.misc.health.smithi003.stderr:2024-05-09T06:36:00.270+0000 7fbefd67d640 1 -- 172.21.15.3:0/1730687355 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fbee406be30 msgr2=0x7fbee406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:00.251 INFO:teuthology.misc.health.smithi003.stderr:2024-05-09T06:36:00.270+0000 7fbefd67d640 1 --2- 172.21.15.3:0/1730687355 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fbee406be30 0x7fbee406e2f0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fbee8034000 tx=0x7fbee802e000 comp rx=0 tx=0).stop 2024-05-09T06:36:00.251 INFO:teuthology.misc.health.smithi003.stderr:2024-05-09T06:36:00.270+0000 7fbefd67d640 1 -- 172.21.15.3:0/1730687355 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbef815e6a0 msgr2=0x7fbef8159510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:00.252 INFO:teuthology.misc.health.smithi003.stderr:2024-05-09T06:36:00.270+0000 7fbefd67d640 1 --2- 172.21.15.3:0/1730687355 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbef815e6a0 0x7fbef8159510 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fbeec00b410 tx=0x7fbeec00b8d0 comp rx=0 tx=0).stop 2024-05-09T06:36:00.252 INFO:teuthology.misc.health.smithi003.stderr:2024-05-09T06:36:00.270+0000 7fbefd67d640 1 -- 172.21.15.3:0/1730687355 shutdown_connections 2024-05-09T06:36:00.252 INFO:teuthology.misc.health.smithi003.stderr:2024-05-09T06:36:00.270+0000 7fbefd67d640 1 --2- 172.21.15.3:0/1730687355 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fbee406be30 0x7fbee406e2f0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:00.252 INFO:teuthology.misc.health.smithi003.stderr:2024-05-09T06:36:00.270+0000 7fbefd67d640 1 --2- 172.21.15.3:0/1730687355 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbef815ea80 0x7fbef81533d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:00.252 INFO:teuthology.misc.health.smithi003.stderr:2024-05-09T06:36:00.270+0000 7fbefd67d640 1 --2- 172.21.15.3:0/1730687355 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbef815e6a0 0x7fbef8159510 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:00.252 INFO:teuthology.misc.health.smithi003.stderr:2024-05-09T06:36:00.270+0000 7fbefd67d640 1 --2- 172.21.15.3:0/1730687355 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbef80d4060 0x7fbef815f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:00.252 INFO:teuthology.misc.health.smithi003.stderr:2024-05-09T06:36:00.270+0000 7fbefd67d640 1 -- 172.21.15.3:0/1730687355 >> 172.21.15.3:0/1730687355 conn(0x7fbef808dd40 msgr2=0x7fbef815c6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:00.252 INFO:teuthology.misc.health.smithi003.stderr:2024-05-09T06:36:00.270+0000 7fbefd67d640 1 -- 172.21.15.3:0/1730687355 shutdown_connections 2024-05-09T06:36:00.252 INFO:teuthology.misc.health.smithi003.stderr:2024-05-09T06:36:00.270+0000 7fbefd67d640 1 -- 172.21.15.3:0/1730687355 wait complete. 2024-05-09T06:36:00.261 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-05-09T06:36:00.261 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-05-09T06:36:00.261 INFO:tasks.ceph.mds.a:Restarting daemon 2024-05-09T06:36:00.261 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-05-09T06:36:00.264 INFO:tasks.ceph.mds.a:Started 2024-05-09T06:36:00.264 INFO:tasks.ceph.mds.c:Restarting daemon 2024-05-09T06:36:00.264 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-05-09T06:36:00.266 INFO:tasks.ceph.mds.c:Started 2024-05-09T06:36:00.266 INFO:tasks.ceph.mds.b:Restarting daemon 2024-05-09T06:36:00.267 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-05-09T06:36:00.269 INFO:tasks.ceph.mds.b:Started 2024-05-09T06:36:00.269 INFO:tasks.ceph.mds.d:Restarting daemon 2024-05-09T06:36:00.270 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-05-09T06:36:00.272 INFO:tasks.ceph.mds.d:Started 2024-05-09T06:36:00.272 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-05-09T06:36:00.272 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T06:36:00.342 INFO:tasks.ceph.mds.a.smithi003.stdout:starting mds.a at 2024-05-09T06:36:00.342 INFO:tasks.ceph.mds.c.smithi003.stdout:starting mds.c at 2024-05-09T06:36:00.357 INFO:tasks.ceph.mds.b.smithi190.stdout:starting mds.b at 2024-05-09T06:36:00.357 INFO:tasks.ceph.mds.d.smithi190.stdout:starting mds.d at 2024-05-09T06:36:00.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:00.402+0000 7f75ba14c640 1 Processor -- start 2024-05-09T06:36:00.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:00.402+0000 7f75ba14c640 1 -- start start 2024-05-09T06:36:00.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:00.403+0000 7f75ba14c640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f75b415c310 0x7f75b40b4570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:00.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:00.403+0000 7f75ba14c640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f75b40cd5b0 0x7f75b40d0010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:00.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:00.403+0000 7f75ba14c640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f75b40cda10 con 0x7f75b40d0550 2024-05-09T06:36:00.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:00.403+0000 7f75ba14c640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f75b40b4de0 con 0x7f75b40cd5b0 2024-05-09T06:36:00.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:00.403+0000 7f75ba14c640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f75b40cdcb0 con 0x7f75b415c310 2024-05-09T06:36:00.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:00.403+0000 7f75b2ffd640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f75b40cd5b0 0x7f75b40d0010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:00.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:00.403+0000 7f75b2ffd640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f75b40cd5b0 0x7f75b40d0010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40438/0 (socket says 172.21.15.3:40438) 2024-05-09T06:36:00.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:00.403+0000 7f75b2ffd640 1 -- 172.21.15.3:0/215181945 learned_addr learned my addr 172.21.15.3:0/215181945 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:36:00.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:00.403+0000 7f75b37fe640 1 --2- 172.21.15.3:0/215181945 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f75b415c310 0x7f75b40b4570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:00.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:00.403+0000 7f75b2ffd640 1 -- 172.21.15.3:0/215181945 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f75b415c310 msgr2=0x7f75b40b4570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:00.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:00.403+0000 7f75b2ffd640 1 --2- 172.21.15.3:0/215181945 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f75b415c310 0x7f75b40b4570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:00.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:00.403+0000 7f75b2ffd640 1 -- 172.21.15.3:0/215181945 >> v1:172.21.15.190:6789/0 conn(0x7f75b40d0550 legacy=0x7f75b409aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:00.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:00.403+0000 7f75b2ffd640 1 -- 172.21.15.3:0/215181945 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75b40b5030 con 0x7f75b40cd5b0 2024-05-09T06:36:00.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:00.404+0000 7f75b2ffd640 1 --2- 172.21.15.3:0/215181945 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f75b40cd5b0 0x7f75b40d0010 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f75a8009870 tx=0x7f75a802a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2af5638a7cdf54e6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:00.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:00.404+0000 7f75b27fc640 1 -- 172.21.15.3:0/215181945 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75a8037070 con 0x7f75b40cd5b0 2024-05-09T06:36:00.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:00.404+0000 7f75b27fc640 1 -- 172.21.15.3:0/215181945 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f75a8032440 con 0x7f75b40cd5b0 2024-05-09T06:36:00.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:00.404+0000 7f75b27fc640 1 -- 172.21.15.3:0/215181945 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75a8032810 con 0x7f75b40cd5b0 2024-05-09T06:36:00.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:00.405+0000 7f75ba14c640 1 -- 172.21.15.3:0/215181945 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f75b40cd5b0 msgr2=0x7f75b40d0010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:00.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:00.405+0000 7f75ba14c640 1 --2- 172.21.15.3:0/215181945 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f75b40cd5b0 0x7f75b40d0010 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f75a8009870 tx=0x7f75a802a1c0 comp rx=0 tx=0).stop 2024-05-09T06:36:00.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:00.405+0000 7f75ba14c640 1 -- 172.21.15.3:0/215181945 shutdown_connections 2024-05-09T06:36:00.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:00.405+0000 7f75ba14c640 1 --2- 172.21.15.3:0/215181945 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f75b40cd5b0 0x7f75b40d0010 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:00.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:00.405+0000 7f75ba14c640 1 --2- 172.21.15.3:0/215181945 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f75b415c310 0x7f75b40b4570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:00.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:00.405+0000 7f75ba14c640 1 -- 172.21.15.3:0/215181945 >> 172.21.15.3:0/215181945 conn(0x7f75b40b3a80 msgr2=0x7f75b40b3e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:00.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:00.405+0000 7f75ba14c640 1 -- 172.21.15.3:0/215181945 shutdown_connections 2024-05-09T06:36:00.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:00.405+0000 7f75ba14c640 1 -- 172.21.15.3:0/215181945 wait complete. 2024-05-09T06:36:00.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:00.406+0000 7f75ba14c640 1 Processor -- start 2024-05-09T06:36:00.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:00.406+0000 7f75ba14c640 1 -- start start 2024-05-09T06:36:00.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:00.407+0000 7f75ba14c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f75b40cd5b0 0x7f75b40c7b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:00.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:00.407+0000 7f75ba14c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f75b40d0550 0x7f75b41f7760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:00.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:00.407+0000 7f75ba14c640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f75b415c310 0x7f75b4159dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:00.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:00.407+0000 7f75ba14c640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f75b40b6b20 con 0x7f75b40cd5b0 2024-05-09T06:36:00.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:00.407+0000 7f75ba14c640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f75b40b55f0 con 0x7f75b40d0550 2024-05-09T06:36:00.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:00.407+0000 7f75ba14c640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f75b40b6ca0 con 0x7f75b415c310 2024-05-09T06:36:00.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:00.407+0000 7f75b37fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f75b40cd5b0 0x7f75b40c7b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:00.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:00.407+0000 7f75b37fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f75b40cd5b0 0x7f75b40c7b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40448/0 (socket says 172.21.15.3:40448) 2024-05-09T06:36:00.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:00.407+0000 7f75b37fe640 1 -- 172.21.15.3:0/700564355 learned_addr learned my addr 172.21.15.3:0/700564355 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:36:00.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:00.407+0000 7f75b2ffd640 1 --2- 172.21.15.3:0/700564355 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f75b40d0550 0x7f75b41f7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:00.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:00.407+0000 7f75b3fff640 1 --2- 172.21.15.3:0/700564355 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f75b415c310 0x7f75b4159dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:00.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:00.407+0000 7f75b37fe640 1 -- 172.21.15.3:0/700564355 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f75b415c310 msgr2=0x7f75b4159dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:00.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:00.407+0000 7f75b37fe640 1 --2- 172.21.15.3:0/700564355 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f75b415c310 0x7f75b4159dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:00.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:00.407+0000 7f75b37fe640 1 -- 172.21.15.3:0/700564355 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f75b40d0550 msgr2=0x7f75b41f7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:00.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:00.407+0000 7f75b37fe640 1 --2- 172.21.15.3:0/700564355 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f75b40d0550 0x7f75b41f7760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:00.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:00.407+0000 7f75b37fe640 1 -- 172.21.15.3:0/700564355 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75b41f4170 con 0x7f75b40cd5b0 2024-05-09T06:36:00.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:00.408+0000 7f75b37fe640 1 --2- 172.21.15.3:0/700564355 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f75b40cd5b0 0x7f75b40c7b90 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f75a400e9b0 tx=0x7f75a400ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:00.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:00.408+0000 7f759bfff640 1 -- 172.21.15.3:0/700564355 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75a4017070 con 0x7f75b40cd5b0 2024-05-09T06:36:00.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:00.408+0000 7f759bfff640 1 -- 172.21.15.3:0/700564355 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f75a400c9a0 con 0x7f75b40cd5b0 2024-05-09T06:36:00.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:00.408+0000 7f75ba14c640 1 -- 172.21.15.3:0/700564355 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75b41f7ca0 con 0x7f75b40cd5b0 2024-05-09T06:36:00.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:00.408+0000 7f759bfff640 1 -- 172.21.15.3:0/700564355 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75a400cd90 con 0x7f75b40cd5b0 2024-05-09T06:36:00.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:00.408+0000 7f75ba14c640 1 -- 172.21.15.3:0/700564355 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75b41f8500 con 0x7f75b40cd5b0 2024-05-09T06:36:00.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:00.409+0000 7f75ba14c640 1 -- 172.21.15.3:0/700564355 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75b404f660 con 0x7f75b40cd5b0 2024-05-09T06:36:00.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:00.409+0000 7f759bfff640 1 -- 172.21.15.3:0/700564355 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f75a401a070 con 0x7f75b40cd5b0 2024-05-09T06:36:00.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:00.410+0000 7f759bfff640 1 --2- 172.21.15.3:0/700564355 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f75a006be30 0x7f75a006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:00.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:00.410+0000 7f759bfff640 1 -- 172.21.15.3:0/700564355 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f75a408aa10 con 0x7f75b40cd5b0 2024-05-09T06:36:00.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:00.410+0000 7f75b2ffd640 1 --2- 172.21.15.3:0/700564355 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f75a006be30 0x7f75a006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:00.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:00.411+0000 7f75b2ffd640 1 --2- 172.21.15.3:0/700564355 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f75a006be30 0x7f75a006e2f0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f75a8034000 tx=0x7f75a802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:00.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:00.414+0000 7f759bfff640 1 -- 172.21.15.3:0/700564355 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f75b41f4170 con 0x7f75b40cd5b0 2024-05-09T06:36:00.578 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:00.598+0000 7f75ba14c640 1 -- 172.21.15.3:0/700564355 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f75b415b960 con 0x7f75b40cd5b0 2024-05-09T06:36:00.579 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:00.598+0000 7f759bfff640 1 -- 172.21.15.3:0/700564355 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) ==== 74+0+12800 (secure 0 0 0) 0x7f75a4059150 con 0x7f75b40cd5b0 2024-05-09T06:36:00.579 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-09T06:36:00.579 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":16,"fsid":"16ed737d-2665-431a-8ac6-ef242be3975a","created":"2024-05-09T06:35:42.541368+0000","modified":"2024-05-09T06:35:56.885471+0000","last_up_change":"2024-05-09T06:35:53.778274+0000","last_in_change":"2024-05-09T06:35:44.780586+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T06:35:54.819999+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fae3a8db-4826-4f3c-b33a-12c09bb5df6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":1897849827},{"type":"v1","addr":"172.21.15.3:6817","nonce":1897849827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":1897849827},{"type":"v1","addr":"172.21.15.3:6819","nonce":1897849827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":1897849827},{"type":"v1","addr":"172.21.15.3:6823","nonce":1897849827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":1897849827},{"type":"v1","addr":"172.21.15.3:6821","nonce":1897849827}]},"public_addr":"172.21.15.3:6817/1897849827","cluster_addr":"172.21.15.3:6819/1897849827","heartbeat_back_addr":"172.21.15.3:6823/1897849827","heartbeat_front_addr":"172.21.15.3:6821/1897849827","state":["exists","up"]},{"osd":1,"uuid":"646e882d-4ce4-4720-8c84-f71df563a35e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":926805498},{"type":"v1","addr":"172.21.15.3:6801","nonce":926805498}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":926805498},{"type":"v1","addr":"172.21.15.3:6803","nonce":926805498}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":926805498},{"type":"v1","addr":"172.21.15.3:6807","nonce":926805498}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":926805498},{"type":"v1","addr":"172.21.15.3:6805","nonce":926805498}]},"public_addr":"172.21.15.3:6801/926805498","cluster_addr":"172.21.15.3:6803/926805498","heartbeat_back_addr":"172.21.15.3:6807/926805498","heartbeat_front_addr":"172.21.15.3:6805/926805498","state":["exists","up"]},{"osd":2,"uuid":"1abbadb5-a570-4359-9643-acff3ede35a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":801295451},{"type":"v1","addr":"172.21.15.3:6809","nonce":801295451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":801295451},{"type":"v1","addr":"172.21.15.3:6811","nonce":801295451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":801295451},{"type":"v1","addr":"172.21.15.3:6815","nonce":801295451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":801295451},{"type":"v1","addr":"172.21.15.3:6813","nonce":801295451}]},"public_addr":"172.21.15.3:6809/801295451","cluster_addr":"172.21.15.3:6811/801295451","heartbeat_back_addr":"172.21.15.3:6815/801295451","heartbeat_front_addr":"172.21.15.3:6813/801295451","state":["exists","up"]},{"osd":3,"uuid":"62ac4dc1-2371-4dd2-acd3-e2dbb11aedf5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":1903389857},{"type":"v1","addr":"172.21.15.3:6825","nonce":1903389857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":1903389857},{"type":"v1","addr":"172.21.15.3:6827","nonce":1903389857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":1903389857},{"type":"v1","addr":"172.21.15.3:6831","nonce":1903389857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":1903389857},{"type":"v1","addr":"172.21.15.3:6829","nonce":1903389857}]},"public_addr":"172.21.15.3:6825/1903389857","cluster_addr":"172.21.15.3:6827/1903389857","heartbeat_back_addr":"172.21.15.3:6831/1903389857","heartbeat_front_addr":"172.21.15.3:6829/1903389857","state":["exists","up"]},{"osd":4,"uuid":"ae808398-8326-4d7c-9319-a2a0eafffb59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":2143624576},{"type":"v1","addr":"172.21.15.190:6801","nonce":2143624576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":2143624576},{"type":"v1","addr":"172.21.15.190:6803","nonce":2143624576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":2143624576},{"type":"v1","addr":"172.21.15.190:6807","nonce":2143624576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":2143624576},{"type":"v1","addr":"172.21.15.190:6805","nonce":2143624576}]},"public_addr":"172.21.15.190:6801/2143624576","cluster_addr":"172.21.15.190:6803/2143624576","heartbeat_back_addr":"172.21.15.190:6807/2143624576","heartbeat_front_addr":"172.21.15.190:6805/2143624576","state":["exists","up"]},{"osd":5,"uuid":"918adf50-c440-48ac-9a8d-4837562d7275","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":1766923126},{"type":"v1","addr":"172.21.15.190:6814","nonce":1766923126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":1766923126},{"type":"v1","addr":"172.21.15.190:6818","nonce":1766923126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":1766923126},{"type":"v1","addr":"172.21.15.190:6823","nonce":1766923126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":1766923126},{"type":"v1","addr":"172.21.15.190:6821","nonce":1766923126}]},"public_addr":"172.21.15.190:6814/1766923126","cluster_addr":"172.21.15.190:6818/1766923126","heartbeat_back_addr":"172.21.15.190:6823/1766923126","heartbeat_front_addr":"172.21.15.190:6821/1766923126","state":["exists","up"]},{"osd":6,"uuid":"ac2882bc-2a7c-463c-8f6c-76ff49828adc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":342414248},{"type":"v1","addr":"172.21.15.190:6825","nonce":342414248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":342414248},{"type":"v1","addr":"172.21.15.190:6827","nonce":342414248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":342414248},{"type":"v1","addr":"172.21.15.190:6831","nonce":342414248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":342414248},{"type":"v1","addr":"172.21.15.190:6829","nonce":342414248}]},"public_addr":"172.21.15.190:6825/342414248","cluster_addr":"172.21.15.190:6827/342414248","heartbeat_back_addr":"172.21.15.190:6831/342414248","heartbeat_front_addr":"172.21.15.190:6829/342414248","state":["exists","up"]},{"osd":7,"uuid":"dd4e767a-4730-4c70-8bd5-3f088140a5a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":537849891},{"type":"v1","addr":"172.21.15.190:6809","nonce":537849891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":537849891},{"type":"v1","addr":"172.21.15.190:6811","nonce":537849891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6817","nonce":537849891},{"type":"v1","addr":"172.21.15.190:6819","nonce":537849891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6813","nonce":537849891},{"type":"v1","addr":"172.21.15.190:6815","nonce":537849891}]},"public_addr":"172.21.15.190:6809/537849891","cluster_addr":"172.21.15.190:6811/537849891","heartbeat_back_addr":"172.21.15.190:6819/537849891","heartbeat_front_addr":"172.21.15.190:6815/537849891","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.889784+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.974002+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.973549+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.955396+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.928471+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.902418+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.958696+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.926474+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T06:36:00.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:00.600+0000 7f75ba14c640 1 -- 172.21.15.3:0/700564355 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f75a006be30 msgr2=0x7f75a006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:00.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:00.600+0000 7f75ba14c640 1 --2- 172.21.15.3:0/700564355 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f75a006be30 0x7f75a006e2f0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f75a8034000 tx=0x7f75a802e000 comp rx=0 tx=0).stop 2024-05-09T06:36:00.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:00.600+0000 7f75ba14c640 1 -- 172.21.15.3:0/700564355 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f75b40cd5b0 msgr2=0x7f75b40c7b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:00.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:00.600+0000 7f75ba14c640 1 --2- 172.21.15.3:0/700564355 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f75b40cd5b0 0x7f75b40c7b90 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f75a400e9b0 tx=0x7f75a400ee70 comp rx=0 tx=0).stop 2024-05-09T06:36:00.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:00.600+0000 7f75ba14c640 1 -- 172.21.15.3:0/700564355 shutdown_connections 2024-05-09T06:36:00.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:00.600+0000 7f75ba14c640 1 --2- 172.21.15.3:0/700564355 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f75a006be30 0x7f75a006e2f0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:00.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:00.600+0000 7f75ba14c640 1 --2- 172.21.15.3:0/700564355 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f75b415c310 0x7f75b4159dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:00.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:00.600+0000 7f75ba14c640 1 --2- 172.21.15.3:0/700564355 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f75b40d0550 0x7f75b41f7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:00.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:00.600+0000 7f75ba14c640 1 --2- 172.21.15.3:0/700564355 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f75b40cd5b0 0x7f75b40c7b90 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:00.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:00.600+0000 7f75ba14c640 1 -- 172.21.15.3:0/700564355 >> 172.21.15.3:0/700564355 conn(0x7f75b40b3a80 msgr2=0x7f75b40b5830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:00.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:00.601+0000 7f75ba14c640 1 -- 172.21.15.3:0/700564355 shutdown_connections 2024-05-09T06:36:00.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:00.601+0000 7f75ba14c640 1 -- 172.21.15.3:0/700564355 wait complete. 2024-05-09T06:36:00.592 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-09T06:35:54.819999+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-09T06:36:00.593 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-09T06:36:00.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:00.718+0000 7fc232ff4640 1 Processor -- start 2024-05-09T06:36:00.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:00.719+0000 7fc232ff4640 1 -- start start 2024-05-09T06:36:00.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:00.719+0000 7fc232ff4640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fc22c0d1300 0x7fc22c0d2800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:00.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:00.719+0000 7fc232ff4640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc22c15e9a0 con 0x7fc22c0d4c40 2024-05-09T06:36:00.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:00.719+0000 7fc232ff4640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc22c0d31c0 con 0x7fc22c15e5c0 2024-05-09T06:36:00.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:00.719+0000 7fc232ff4640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7fc22c0d2ee0 con 0x7fc22c0d1300 2024-05-09T06:36:00.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:00.720+0000 7fc231591640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fc22c0d4c40 0x7fc22c0b47e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:33000/0 (socket says 172.21.15.3:33000) 2024-05-09T06:36:00.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:00.720+0000 7fc231591640 1 -- 172.21.15.3:0/1684389708 learned_addr learned my addr 172.21.15.3:0/1684389708 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-05-09T06:36:00.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:00.720+0000 7fc22bfff640 1 --2- 172.21.15.3:0/1684389708 >> v2:172.21.15.190:3300/0 conn(0x7fc22c0d1300 0x7fc22c0d2800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:00.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:00.720+0000 7fc22b7fe640 1 -- 172.21.15.3:0/1684389708 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 198411459 0 0) 0x7fc22c15e9a0 con 0x7fc22c0d4c40 2024-05-09T06:36:00.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:00.720+0000 7fc22b7fe640 1 -- 172.21.15.3:0/1684389708 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc214003520 con 0x7fc22c0d4c40 2024-05-09T06:36:00.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:00.720+0000 7fc22b7fe640 1 -- 172.21.15.3:0/1684389708 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2970819644 0 0) 0x7fc214003520 con 0x7fc22c0d4c40 2024-05-09T06:36:00.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:00.720+0000 7fc22bfff640 1 -- 172.21.15.3:0/1684389708 >> v1:172.21.15.190:6789/0 conn(0x7fc22c15e5c0 legacy=0x7fc22c0d0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:00.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:00.720+0000 7fc22bfff640 1 -- 172.21.15.3:0/1684389708 >> v1:172.21.15.3:6789/0 conn(0x7fc22c0d4c40 legacy=0x7fc22c0b47e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:00.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:00.720+0000 7fc22bfff640 1 -- 172.21.15.3:0/1684389708 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc22c0b3d00 con 0x7fc22c0d1300 2024-05-09T06:36:00.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:00.721+0000 7fc22bfff640 1 --2- 172.21.15.3:0/1684389708 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc22c0d1300 0x7fc22c0d2800 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fc2200097a0 tx=0x7fc22002a0d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bd4e00dd230067e6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:00.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:00.721+0000 7fc22b7fe640 1 -- 172.21.15.3:0/1684389708 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc220037070 con 0x7fc22c0d1300 2024-05-09T06:36:00.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:00.721+0000 7fc22b7fe640 1 -- 172.21.15.3:0/1684389708 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc22002ab60 con 0x7fc22c0d1300 2024-05-09T06:36:00.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:00.721+0000 7fc22b7fe640 1 -- 172.21.15.3:0/1684389708 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2240019f0 con 0x7fc22c0d1300 2024-05-09T06:36:00.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:00.721+0000 7fc232ff4640 1 -- 172.21.15.3:0/1684389708 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc22c0d1300 msgr2=0x7fc22c0d2800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:00.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:00.721+0000 7fc232ff4640 1 --2- 172.21.15.3:0/1684389708 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc22c0d1300 0x7fc22c0d2800 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fc2200097a0 tx=0x7fc22002a0d0 comp rx=0 tx=0).stop 2024-05-09T06:36:00.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:00.722+0000 7fc232ff4640 1 -- 172.21.15.3:0/1684389708 shutdown_connections 2024-05-09T06:36:00.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:00.722+0000 7fc232ff4640 1 --2- 172.21.15.3:0/1684389708 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc22c0d1300 0x7fc22c0d2800 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:00.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:00.722+0000 7fc232ff4640 1 -- 172.21.15.3:0/1684389708 >> 172.21.15.3:0/1684389708 conn(0x7fc22c09d500 msgr2=0x7fc22c0d4220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:00.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:00.722+0000 7fc232ff4640 1 -- 172.21.15.3:0/1684389708 shutdown_connections 2024-05-09T06:36:00.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:00.722+0000 7fc232ff4640 1 -- 172.21.15.3:0/1684389708 wait complete. 2024-05-09T06:36:00.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:00.723+0000 7fc232ff4640 1 Processor -- start 2024-05-09T06:36:00.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:00.723+0000 7fc232ff4640 1 -- start start 2024-05-09T06:36:00.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:00.724+0000 7fc232ff4640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc22c0d1300 0x7fc22c0b1c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:00.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:00.724+0000 7fc232ff4640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc22c0d4c40 0x7fc22c0b8920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:00.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:00.724+0000 7fc232ff4640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc22c15e5c0 0x7fc22c0b8e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:00.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:00.724+0000 7fc232ff4640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fc22c0d5090 con 0x7fc22c0d1300 2024-05-09T06:36:00.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:00.724+0000 7fc232ff4640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fc22c0b42c0 con 0x7fc22c15e5c0 2024-05-09T06:36:00.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:00.724+0000 7fc232ff4640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fc22c0b4ef0 con 0x7fc22c0d4c40 2024-05-09T06:36:00.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:00.724+0000 7fc230d90640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc22c0d1300 0x7fc22c0b1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:00.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:00.724+0000 7fc230d90640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc22c0d1300 0x7fc22c0b1c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45566/0 (socket says 172.21.15.3:45566) 2024-05-09T06:36:00.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:00.724+0000 7fc230d90640 1 -- 172.21.15.3:0/1462990640 learned_addr learned my addr 172.21.15.3:0/1462990640 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:36:00.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:00.724+0000 7fc22bfff640 1 --2- 172.21.15.3:0/1462990640 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc22c0d4c40 0x7fc22c0b8920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:00.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:00.724+0000 7fc231591640 1 --2- 172.21.15.3:0/1462990640 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc22c15e5c0 0x7fc22c0b8e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:00.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:00.724+0000 7fc230d90640 1 -- 172.21.15.3:0/1462990640 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc22c0d4c40 msgr2=0x7fc22c0b8920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:00.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:00.724+0000 7fc230d90640 1 --2- 172.21.15.3:0/1462990640 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc22c0d4c40 0x7fc22c0b8920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:00.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:00.724+0000 7fc230d90640 1 -- 172.21.15.3:0/1462990640 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc22c15e5c0 msgr2=0x7fc22c0b8e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:00.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:00.724+0000 7fc230d90640 1 --2- 172.21.15.3:0/1462990640 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc22c15e5c0 0x7fc22c0b8e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:00.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:00.724+0000 7fc230d90640 1 -- 172.21.15.3:0/1462990640 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc22c0b6ad0 con 0x7fc22c0d1300 2024-05-09T06:36:00.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:00.724+0000 7fc230d90640 1 --2- 172.21.15.3:0/1462990640 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc22c0d1300 0x7fc22c0b1c10 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fc21c00c550 tx=0x7fc21c00ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:00.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:00.725+0000 7fc2297fa640 1 -- 172.21.15.3:0/1462990640 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc21c017070 con 0x7fc22c0d1300 2024-05-09T06:36:00.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:00.725+0000 7fc2297fa640 1 -- 172.21.15.3:0/1462990640 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc21c0035f0 con 0x7fc22c0d1300 2024-05-09T06:36:00.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:00.725+0000 7fc232ff4640 1 -- 172.21.15.3:0/1462990640 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc22c0b6db0 con 0x7fc22c0d1300 2024-05-09T06:36:00.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:00.725+0000 7fc2297fa640 1 -- 172.21.15.3:0/1462990640 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc21c0039e0 con 0x7fc22c0d1300 2024-05-09T06:36:00.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:00.725+0000 7fc232ff4640 1 -- 172.21.15.3:0/1462990640 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc22c0b7370 con 0x7fc22c0d1300 2024-05-09T06:36:00.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:00.726+0000 7fc232ff4640 1 -- 172.21.15.3:0/1462990640 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc22c069550 con 0x7fc22c0d1300 2024-05-09T06:36:00.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:00.726+0000 7fc2297fa640 1 -- 172.21.15.3:0/1462990640 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc21c01a070 con 0x7fc22c0d1300 2024-05-09T06:36:00.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:00.727+0000 7fc2297fa640 1 --2- 172.21.15.3:0/1462990640 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fc21806be30 0x7fc21806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:00.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:00.727+0000 7fc2297fa640 1 -- 172.21.15.3:0/1462990640 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7fc21c089ef0 con 0x7fc22c0d1300 2024-05-09T06:36:00.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:00.727+0000 7fc22bfff640 1 --2- 172.21.15.3:0/1462990640 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fc21806be30 0x7fc21806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:00.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:00.728+0000 7fc22bfff640 1 --2- 172.21.15.3:0/1462990640 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fc21806be30 0x7fc21806e2f0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fc220034000 tx=0x7fc22002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:00.712 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:00.732+0000 7fc2297fa640 1 -- 172.21.15.3:0/1462990640 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fc21c0585a0 con 0x7fc22c0d1300 2024-05-09T06:36:00.895 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:00.914+0000 7fc232ff4640 1 -- 172.21.15.3:0/1462990640 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fc22c15d9c0 con 0x7fc22c0d1300 2024-05-09T06:36:00.895 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:00.914+0000 7fc2297fa640 1 -- 172.21.15.3:0/1462990640 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) ==== 93+0+10 (secure 0 0 0) 0x7fc21c05c8a0 con 0x7fc22c0d1300 2024-05-09T06:36:00.895 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 1 2024-05-09T06:36:00.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:00.916+0000 7fc232ff4640 1 -- 172.21.15.3:0/1462990640 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fc21806be30 msgr2=0x7fc21806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:00.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:00.916+0000 7fc232ff4640 1 --2- 172.21.15.3:0/1462990640 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fc21806be30 0x7fc21806e2f0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fc220034000 tx=0x7fc22002e000 comp rx=0 tx=0).stop 2024-05-09T06:36:00.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:00.917+0000 7fc232ff4640 1 -- 172.21.15.3:0/1462990640 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc22c0d1300 msgr2=0x7fc22c0b1c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:00.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:00.917+0000 7fc232ff4640 1 --2- 172.21.15.3:0/1462990640 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc22c0d1300 0x7fc22c0b1c10 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fc21c00c550 tx=0x7fc21c00ca10 comp rx=0 tx=0).stop 2024-05-09T06:36:00.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:00.917+0000 7fc232ff4640 1 -- 172.21.15.3:0/1462990640 shutdown_connections 2024-05-09T06:36:00.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:00.917+0000 7fc232ff4640 1 --2- 172.21.15.3:0/1462990640 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fc21806be30 0x7fc21806e2f0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:00.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:00.917+0000 7fc232ff4640 1 --2- 172.21.15.3:0/1462990640 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc22c15e5c0 0x7fc22c0b8e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:00.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:00.917+0000 7fc232ff4640 1 --2- 172.21.15.3:0/1462990640 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc22c0d4c40 0x7fc22c0b8920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:00.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:00.917+0000 7fc232ff4640 1 --2- 172.21.15.3:0/1462990640 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc22c0d1300 0x7fc22c0b1c10 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:00.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:00.917+0000 7fc232ff4640 1 -- 172.21.15.3:0/1462990640 >> 172.21.15.3:0/1462990640 conn(0x7fc22c09d500 msgr2=0x7fc22c0d0310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:00.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:00.917+0000 7fc232ff4640 1 -- 172.21.15.3:0/1462990640 shutdown_connections 2024-05-09T06:36:00.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:00.917+0000 7fc232ff4640 1 -- 172.21.15.3:0/1462990640 wait complete. 2024-05-09T06:36:00.909 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T06:36:01.013 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:01.032+0000 7f8c094fc640 1 Processor -- start 2024-05-09T06:36:01.013 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:01.032+0000 7f8c094fc640 1 -- start start 2024-05-09T06:36:01.014 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:01.033+0000 7f8c094fc640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8c0415e4e0 0x7f8c0406ec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:01.014 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:01.033+0000 7f8c094fc640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f8c040af720 0x7f8c040cc7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:01.014 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:01.033+0000 7f8c094fc640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f8c040cccf0 0x7f8c040d0120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:01.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:01.033+0000 7f8c094fc640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f8c0406f570 con 0x7f8c040cccf0 2024-05-09T06:36:01.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:01.033+0000 7f8c094fc640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f8c040cea20 con 0x7f8c040af720 2024-05-09T06:36:01.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:01.033+0000 7f8c094fc640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f8c040736b0 con 0x7f8c0415e4e0 2024-05-09T06:36:01.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:01.033+0000 7f8c037fe640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f8c040cccf0 0x7f8c040d0120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:01.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:01.033+0000 7f8c037fe640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f8c040cccf0 0x7f8c040d0120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45574/0 (socket says 172.21.15.3:45574) 2024-05-09T06:36:01.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:01.033+0000 7f8c037fe640 1 -- 172.21.15.3:0/543529254 learned_addr learned my addr 172.21.15.3:0/543529254 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:36:01.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:01.033+0000 7f8c02ffd640 1 --2- 172.21.15.3:0/543529254 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8c0415e4e0 0x7f8c0406ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:01.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:01.033+0000 7f8bfbfff640 1 --2- 172.21.15.3:0/543529254 >> v2:172.21.15.190:3300/0 conn(0x7f8c040af720 0x7f8c040cc7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:01.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:01.033+0000 7f8c037fe640 1 -- 172.21.15.3:0/543529254 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8c0415e4e0 msgr2=0x7f8c0406ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:01.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:01.033+0000 7f8c037fe640 1 --2- 172.21.15.3:0/543529254 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8c0415e4e0 0x7f8c0406ec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:01.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:01.033+0000 7f8c037fe640 1 -- 172.21.15.3:0/543529254 >> v2:172.21.15.190:3300/0 conn(0x7f8c040af720 msgr2=0x7f8c040cc7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:01.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:01.033+0000 7f8c037fe640 1 --2- 172.21.15.3:0/543529254 >> v2:172.21.15.190:3300/0 conn(0x7f8c040af720 0x7f8c040cc7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:01.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:01.033+0000 7f8c037fe640 1 -- 172.21.15.3:0/543529254 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c0406f290 con 0x7f8c040cccf0 2024-05-09T06:36:01.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:01.033+0000 7f8bfbfff640 1 --2- 172.21.15.3:0/543529254 >> v2:172.21.15.190:3300/0 conn(0x7f8c040af720 0x7f8c040cc7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:36:01.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:01.034+0000 7f8c037fe640 1 --2- 172.21.15.3:0/543529254 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8c040cccf0 0x7f8c040d0120 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f8c0406ccb0 tx=0x7f8bfc02a570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4984a8ac22aa17ba server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:01.017 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:01.034+0000 7f8c027fc640 1 -- 172.21.15.3:0/543529254 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bfc036070 con 0x7f8c040cccf0 2024-05-09T06:36:01.017 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:01.034+0000 7f8c027fc640 1 -- 172.21.15.3:0/543529254 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8bfc031600 con 0x7f8c040cccf0 2024-05-09T06:36:01.017 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:01.034+0000 7f8c027fc640 1 -- 172.21.15.3:0/543529254 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bfc0319d0 con 0x7f8c040cccf0 2024-05-09T06:36:01.017 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:01.035+0000 7f8c094fc640 1 -- 172.21.15.3:0/543529254 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8c040cccf0 msgr2=0x7f8c040d0120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:01.017 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:01.035+0000 7f8c094fc640 1 --2- 172.21.15.3:0/543529254 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8c040cccf0 0x7f8c040d0120 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f8c0406ccb0 tx=0x7f8bfc02a570 comp rx=0 tx=0).stop 2024-05-09T06:36:01.017 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:01.035+0000 7f8c094fc640 1 -- 172.21.15.3:0/543529254 shutdown_connections 2024-05-09T06:36:01.017 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:01.035+0000 7f8c094fc640 1 --2- 172.21.15.3:0/543529254 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8c040cccf0 0x7f8c040d0120 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:01.017 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:01.035+0000 7f8c094fc640 1 --2- 172.21.15.3:0/543529254 >> v2:172.21.15.190:3300/0 conn(0x7f8c040af720 0x7f8c040cc7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:01.017 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:01.035+0000 7f8c094fc640 1 --2- 172.21.15.3:0/543529254 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8c0415e4e0 0x7f8c0406ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:01.017 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:01.035+0000 7f8c094fc640 1 -- 172.21.15.3:0/543529254 >> 172.21.15.3:0/543529254 conn(0x7f8c04072870 msgr2=0x7f8c04072c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:01.017 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:01.035+0000 7f8c094fc640 1 -- 172.21.15.3:0/543529254 shutdown_connections 2024-05-09T06:36:01.017 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:01.035+0000 7f8c094fc640 1 -- 172.21.15.3:0/543529254 wait complete. 2024-05-09T06:36:01.018 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:01.036+0000 7f8c094fc640 1 Processor -- start 2024-05-09T06:36:01.018 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:01.036+0000 7f8c094fc640 1 -- start start 2024-05-09T06:36:01.020 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:01.037+0000 7f8c094fc640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8c040af720 0x7f8c0415f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:01.020 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:01.037+0000 7f8c094fc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8c040cccf0 0x7f8c04149650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:01.020 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:01.037+0000 7f8c094fc640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8c0415e4e0 0x7f8c0415c600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:01.020 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:01.037+0000 7f8c094fc640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f8c0415e8c0 con 0x7f8c040cccf0 2024-05-09T06:36:01.020 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:01.037+0000 7f8c094fc640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f8c040ce6b0 con 0x7f8c040af720 2024-05-09T06:36:01.020 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:01.037+0000 7f8c094fc640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f8c0415ea40 con 0x7f8c0415e4e0 2024-05-09T06:36:01.020 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:01.037+0000 7f8bfbfff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8c040cccf0 0x7f8c04149650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:01.020 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:01.037+0000 7f8bfbfff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8c040cccf0 0x7f8c04149650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45576/0 (socket says 172.21.15.3:45576) 2024-05-09T06:36:01.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:01.037+0000 7f8bfbfff640 1 -- 172.21.15.3:0/3392778081 learned_addr learned my addr 172.21.15.3:0/3392778081 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:36:01.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:01.037+0000 7f8c037fe640 1 --2- 172.21.15.3:0/3392778081 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8c0415e4e0 0x7f8c0415c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:01.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:01.037+0000 7f8c02ffd640 1 --2- 172.21.15.3:0/3392778081 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8c040af720 0x7f8c0415f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:01.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:01.037+0000 7f8bfbfff640 1 -- 172.21.15.3:0/3392778081 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8c0415e4e0 msgr2=0x7f8c0415c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:01.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:01.037+0000 7f8bfbfff640 1 --2- 172.21.15.3:0/3392778081 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8c0415e4e0 0x7f8c0415c600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:01.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:01.037+0000 7f8bfbfff640 1 -- 172.21.15.3:0/3392778081 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8c040af720 msgr2=0x7f8c0415f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:01.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:01.037+0000 7f8bfbfff640 1 --2- 172.21.15.3:0/3392778081 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8c040af720 0x7f8c0415f7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:01.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:01.037+0000 7f8bfbfff640 1 -- 172.21.15.3:0/3392778081 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c041501b0 con 0x7f8c040cccf0 2024-05-09T06:36:01.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:01.037+0000 7f8bfbfff640 1 --2- 172.21.15.3:0/3392778081 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8c040cccf0 0x7f8c04149650 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f8bec00e810 tx=0x7f8bec00ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:01.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:01.037+0000 7f8bfb7fe640 1 -- 172.21.15.3:0/3392778081 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bec00f040 con 0x7f8c040cccf0 2024-05-09T06:36:01.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:01.037+0000 7f8bfb7fe640 1 -- 172.21.15.3:0/3392778081 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8bec004030 con 0x7f8c040cccf0 2024-05-09T06:36:01.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:01.037+0000 7f8c094fc640 1 -- 172.21.15.3:0/3392778081 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c041f8070 con 0x7f8c040cccf0 2024-05-09T06:36:01.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:01.037+0000 7f8bfb7fe640 1 -- 172.21.15.3:0/3392778081 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bec004400 con 0x7f8c040cccf0 2024-05-09T06:36:01.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:01.037+0000 7f8c094fc640 1 -- 172.21.15.3:0/3392778081 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c0415ecf0 con 0x7f8c040cccf0 2024-05-09T06:36:01.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:01.038+0000 7f8c094fc640 1 -- 172.21.15.3:0/3392778081 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c0404f660 con 0x7f8c040cccf0 2024-05-09T06:36:01.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:01.039+0000 7f8bfb7fe640 1 -- 172.21.15.3:0/3392778081 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8bec00c6d0 con 0x7f8c040cccf0 2024-05-09T06:36:01.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:01.039+0000 7f8bfb7fe640 1 --2- 172.21.15.3:0/3392778081 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f8be406be60 0x7f8be406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:01.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:01.039+0000 7f8bfb7fe640 1 -- 172.21.15.3:0/3392778081 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f8bec08a750 con 0x7f8c040cccf0 2024-05-09T06:36:01.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:01.039+0000 7f8c02ffd640 1 --2- 172.21.15.3:0/3392778081 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f8be406be60 0x7f8be406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:01.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:01.040+0000 7f8c02ffd640 1 --2- 172.21.15.3:0/3392778081 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f8be406be60 0x7f8be406e320 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f8bf4009970 tx=0x7f8bf4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:01.024 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:01.043+0000 7f8bfb7fe640 1 -- 172.21.15.3:0/3392778081 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f8c041501b0 con 0x7f8c040cccf0 2024-05-09T06:36:01.247 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:01.266+0000 7f8c094fc640 1 -- 172.21.15.3:0/3392778081 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8c0406cfd0 con 0x7f8c040cccf0 2024-05-09T06:36:01.248 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:01.267+0000 7f8bfb7fe640 1 -- 172.21.15.3:0/3392778081 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1386 (secure 0 0 0) 0x7f8bec017020 con 0x7f8c040cccf0 2024-05-09T06:36:01.248 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-09T06:36:01.248 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4272,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6834/126273055","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":126273055},{"type":"v1","addr":"172.21.15.3:6834","nonce":126273055}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-05-09T06:36:01.248 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 2 2024-05-09T06:36:01.250 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:01.269+0000 7f8c094fc640 1 -- 172.21.15.3:0/3392778081 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f8be406be60 msgr2=0x7f8be406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:01.250 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:01.269+0000 7f8c094fc640 1 --2- 172.21.15.3:0/3392778081 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f8be406be60 0x7f8be406e320 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f8bf4009970 tx=0x7f8bf4008040 comp rx=0 tx=0).stop 2024-05-09T06:36:01.250 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:01.269+0000 7f8c094fc640 1 -- 172.21.15.3:0/3392778081 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8c040cccf0 msgr2=0x7f8c04149650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:01.250 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:01.269+0000 7f8c094fc640 1 --2- 172.21.15.3:0/3392778081 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8c040cccf0 0x7f8c04149650 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f8bec00e810 tx=0x7f8bec00ecd0 comp rx=0 tx=0).stop 2024-05-09T06:36:01.250 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:01.269+0000 7f8c094fc640 1 -- 172.21.15.3:0/3392778081 shutdown_connections 2024-05-09T06:36:01.250 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:01.269+0000 7f8c094fc640 1 --2- 172.21.15.3:0/3392778081 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f8be406be60 0x7f8be406e320 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:01.250 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:01.269+0000 7f8c094fc640 1 --2- 172.21.15.3:0/3392778081 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8c0415e4e0 0x7f8c0415c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:01.250 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:01.269+0000 7f8c094fc640 1 --2- 172.21.15.3:0/3392778081 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8c040cccf0 0x7f8c04149650 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:01.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:01.269+0000 7f8c094fc640 1 --2- 172.21.15.3:0/3392778081 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8c040af720 0x7f8c0415f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:01.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:01.269+0000 7f8c094fc640 1 -- 172.21.15.3:0/3392778081 >> 172.21.15.3:0/3392778081 conn(0x7f8c04072870 msgr2=0x7f8c040b4510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:01.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:01.269+0000 7f8c094fc640 1 -- 172.21.15.3:0/3392778081 shutdown_connections 2024-05-09T06:36:01.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:01.269+0000 7f8c094fc640 1 -- 172.21.15.3:0/3392778081 wait complete. 2024-05-09T06:36:03.262 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T06:36:03.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:03.398+0000 7fe287b1d640 1 Processor -- start 2024-05-09T06:36:03.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:03.398+0000 7fe287b1d640 1 -- start start 2024-05-09T06:36:03.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:03.399+0000 7fe287b1d640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe28015e4c0 0x7fe2800d3c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:03.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:03.399+0000 7fe287b1d640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe2800d2950 con 0x7fe2800cf5b0 2024-05-09T06:36:03.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:03.399+0000 7fe287b1d640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe2800d2bd0 con 0x7fe2800d2510 2024-05-09T06:36:03.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:03.399+0000 7fe287b1d640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fe2800d2eb0 con 0x7fe28015e4c0 2024-05-09T06:36:03.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:03.399+0000 7fe2860ba640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fe2800cf5b0 0x7fe2800b7260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:33016/0 (socket says 172.21.15.3:33016) 2024-05-09T06:36:03.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:03.399+0000 7fe2860ba640 1 -- 172.21.15.3:0/389121363 learned_addr learned my addr 172.21.15.3:0/389121363 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-05-09T06:36:03.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:03.400+0000 7fe2858b9640 1 --2- 172.21.15.3:0/389121363 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe28015e4c0 0x7fe2800d3c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:03.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:03.400+0000 7fe2848b7640 1 -- 172.21.15.3:0/389121363 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3428277851 0 0) 0x7fe278001110 con 0x7fe2800cf5b0 2024-05-09T06:36:03.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:03.400+0000 7fe2848b7640 1 -- 172.21.15.3:0/389121363 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe26c003520 con 0x7fe2800cf5b0 2024-05-09T06:36:03.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:03.400+0000 7fe2848b7640 1 -- 172.21.15.3:0/389121363 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 54087777 0 0) 0x7fe2800d2bd0 con 0x7fe2800d2510 2024-05-09T06:36:03.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:03.400+0000 7fe2848b7640 1 -- 172.21.15.3:0/389121363 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe278001110 con 0x7fe2800d2510 2024-05-09T06:36:03.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:03.400+0000 7fe2848b7640 1 -- 172.21.15.3:0/389121363 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1720202396 0 0) 0x7fe26c003520 con 0x7fe2800cf5b0 2024-05-09T06:36:03.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:03.400+0000 7fe2858b9640 1 -- 172.21.15.3:0/389121363 >> v1:172.21.15.190:6789/0 conn(0x7fe2800d2510 legacy=0x7fe2800ceea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:03.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:03.400+0000 7fe2858b9640 1 -- 172.21.15.3:0/389121363 >> v1:172.21.15.3:6789/0 conn(0x7fe2800cf5b0 legacy=0x7fe2800b7260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:03.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:03.400+0000 7fe2858b9640 1 -- 172.21.15.3:0/389121363 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2800d3190 con 0x7fe28015e4c0 2024-05-09T06:36:03.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:03.401+0000 7fe2858b9640 1 --2- 172.21.15.3:0/389121363 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe28015e4c0 0x7fe2800d3c70 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fe27c009dc0 tx=0x7fe27c02a540 comp rx=0 tx=0).ready entity=mon.2 client_cookie=481e3d6ed22e9e03 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:03.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:03.401+0000 7fe2848b7640 1 -- 172.21.15.3:0/389121363 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe278001840 con 0x7fe28015e4c0 2024-05-09T06:36:03.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:03.401+0000 7fe2848b7640 1 -- 172.21.15.3:0/389121363 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe27c007540 con 0x7fe28015e4c0 2024-05-09T06:36:03.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:03.401+0000 7fe2848b7640 1 -- 172.21.15.3:0/389121363 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe27c0078f0 con 0x7fe28015e4c0 2024-05-09T06:36:03.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:03.401+0000 7fe287b1d640 1 -- 172.21.15.3:0/389121363 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe28015e4c0 msgr2=0x7fe2800d3c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:03.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:03.401+0000 7fe287b1d640 1 --2- 172.21.15.3:0/389121363 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe28015e4c0 0x7fe2800d3c70 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fe27c009dc0 tx=0x7fe27c02a540 comp rx=0 tx=0).stop 2024-05-09T06:36:03.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:03.402+0000 7fe287b1d640 1 -- 172.21.15.3:0/389121363 shutdown_connections 2024-05-09T06:36:03.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:03.402+0000 7fe287b1d640 1 --2- 172.21.15.3:0/389121363 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe28015e4c0 0x7fe2800d3c70 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:03.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:03.402+0000 7fe287b1d640 1 -- 172.21.15.3:0/389121363 >> 172.21.15.3:0/389121363 conn(0x7fe2800673b0 msgr2=0x7fe2800677c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:03.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:03.402+0000 7fe287b1d640 1 -- 172.21.15.3:0/389121363 shutdown_connections 2024-05-09T06:36:03.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:03.402+0000 7fe287b1d640 1 -- 172.21.15.3:0/389121363 wait complete. 2024-05-09T06:36:03.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:03.403+0000 7fe287b1d640 1 Processor -- start 2024-05-09T06:36:03.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:03.403+0000 7fe287b1d640 1 -- start start 2024-05-09T06:36:03.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:03.404+0000 7fe287b1d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe2800cf5b0 0x7fe28015a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:03.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:03.404+0000 7fe287b1d640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe2800d2510 0x7fe2801f3c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:03.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:03.404+0000 7fe287b1d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe28015e4c0 0x7fe280160b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:03.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:03.404+0000 7fe287b1d640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fe2800b9cc0 con 0x7fe2800cf5b0 2024-05-09T06:36:03.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:03.404+0000 7fe287b1d640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fe2800d3750 con 0x7fe28015e4c0 2024-05-09T06:36:03.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:03.404+0000 7fe287b1d640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fe2800b9e40 con 0x7fe2800d2510 2024-05-09T06:36:03.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:03.404+0000 7fe2858b9640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe2800cf5b0 0x7fe28015a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:03.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:03.404+0000 7fe2858b9640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe2800cf5b0 0x7fe28015a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45582/0 (socket says 172.21.15.3:45582) 2024-05-09T06:36:03.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:03.404+0000 7fe2858b9640 1 -- 172.21.15.3:0/754571725 learned_addr learned my addr 172.21.15.3:0/754571725 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:36:03.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:03.404+0000 7fe2850b8640 1 --2- 172.21.15.3:0/754571725 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe2800d2510 0x7fe2801f3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:03.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:03.404+0000 7fe2860ba640 1 --2- 172.21.15.3:0/754571725 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe28015e4c0 0x7fe280160b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:03.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:03.404+0000 7fe2858b9640 1 -- 172.21.15.3:0/754571725 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe2800d2510 msgr2=0x7fe2801f3c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:03.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:03.404+0000 7fe2858b9640 1 --2- 172.21.15.3:0/754571725 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe2800d2510 0x7fe2801f3c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:03.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:03.404+0000 7fe2858b9640 1 -- 172.21.15.3:0/754571725 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe28015e4c0 msgr2=0x7fe280160b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:03.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:03.404+0000 7fe2858b9640 1 --2- 172.21.15.3:0/754571725 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe28015e4c0 0x7fe280160b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:03.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:03.404+0000 7fe2858b9640 1 -- 172.21.15.3:0/754571725 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2800c8db0 con 0x7fe2800cf5b0 2024-05-09T06:36:03.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:03.404+0000 7fe2858b9640 1 --2- 172.21.15.3:0/754571725 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe2800cf5b0 0x7fe28015a2f0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fe27c009ee0 tx=0x7fe27c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:03.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:03.405+0000 7fe2767fc640 1 -- 172.21.15.3:0/754571725 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe27c040070 con 0x7fe2800cf5b0 2024-05-09T06:36:03.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:03.405+0000 7fe287b1d640 1 -- 172.21.15.3:0/754571725 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2800c95f0 con 0x7fe2800cf5b0 2024-05-09T06:36:03.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:03.405+0000 7fe2767fc640 1 -- 172.21.15.3:0/754571725 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe27c031c40 con 0x7fe2800cf5b0 2024-05-09T06:36:03.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:03.405+0000 7fe2767fc640 1 -- 172.21.15.3:0/754571725 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe27c009450 con 0x7fe2800cf5b0 2024-05-09T06:36:03.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:03.405+0000 7fe287b1d640 1 -- 172.21.15.3:0/754571725 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2800c9bb0 con 0x7fe2800cf5b0 2024-05-09T06:36:03.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:03.406+0000 7fe287b1d640 1 -- 172.21.15.3:0/754571725 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2800c9400 con 0x7fe2800cf5b0 2024-05-09T06:36:03.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:03.406+0000 7fe2767fc640 1 -- 172.21.15.3:0/754571725 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe27c039040 con 0x7fe2800cf5b0 2024-05-09T06:36:03.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:03.407+0000 7fe2767fc640 1 --2- 172.21.15.3:0/754571725 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fe26406be30 0x7fe26406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:03.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:03.407+0000 7fe2767fc640 1 -- 172.21.15.3:0/754571725 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7fe27c0ac2d0 con 0x7fe2800cf5b0 2024-05-09T06:36:03.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:03.407+0000 7fe2850b8640 1 --2- 172.21.15.3:0/754571725 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fe26406be30 0x7fe26406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:03.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:03.407+0000 7fe2850b8640 1 --2- 172.21.15.3:0/754571725 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fe26406be30 0x7fe26406e2f0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fe280161650 tx=0x7fe270009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:03.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:03.411+0000 7fe2767fc640 1 -- 172.21.15.3:0/754571725 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fe27c07af50 con 0x7fe2800cf5b0 2024-05-09T06:36:03.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:03.631+0000 7fe287b1d640 1 -- 172.21.15.3:0/754571725 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe28015d1e0 con 0x7fe2800cf5b0 2024-05-09T06:36:03.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:03.631+0000 7fe2767fc640 1 -- 172.21.15.3:0/754571725 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+3861 (secure 0 0 0) 0x7fe27c07a850 con 0x7fe2800cf5b0 2024-05-09T06:36:03.612 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-09T06:36:03.613 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4272,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6834/126273055","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":126273055},{"type":"v1","addr":"172.21.15.3:6834","nonce":126273055}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4273,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6836/3885692063","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":3885692063},{"type":"v1","addr":"172.21.15.190:6836","nonce":3885692063}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4275,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/3461228756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6833","nonce":3461228756},{"type":"v1","addr":"172.21.15.3:6835","nonce":3461228756}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4276,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6837/4111472052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6835","nonce":4111472052},{"type":"v1","addr":"172.21.15.190:6837","nonce":4111472052}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-05-09T06:36:03.613 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 3 2024-05-09T06:36:03.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:03.633+0000 7fe287b1d640 1 -- 172.21.15.3:0/754571725 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fe26406be30 msgr2=0x7fe26406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:03.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:03.633+0000 7fe287b1d640 1 --2- 172.21.15.3:0/754571725 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fe26406be30 0x7fe26406e2f0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fe280161650 tx=0x7fe270009040 comp rx=0 tx=0).stop 2024-05-09T06:36:03.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:03.633+0000 7fe287b1d640 1 -- 172.21.15.3:0/754571725 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe2800cf5b0 msgr2=0x7fe28015a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:03.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:03.633+0000 7fe287b1d640 1 --2- 172.21.15.3:0/754571725 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe2800cf5b0 0x7fe28015a2f0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fe27c009ee0 tx=0x7fe27c004a40 comp rx=0 tx=0).stop 2024-05-09T06:36:03.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:03.633+0000 7fe287b1d640 1 -- 172.21.15.3:0/754571725 shutdown_connections 2024-05-09T06:36:03.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:03.633+0000 7fe287b1d640 1 --2- 172.21.15.3:0/754571725 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fe26406be30 0x7fe26406e2f0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:03.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:03.633+0000 7fe287b1d640 1 --2- 172.21.15.3:0/754571725 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe28015e4c0 0x7fe280160b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:03.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:03.633+0000 7fe287b1d640 1 --2- 172.21.15.3:0/754571725 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe2800d2510 0x7fe2801f3c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:03.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:03.633+0000 7fe287b1d640 1 --2- 172.21.15.3:0/754571725 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe2800cf5b0 0x7fe28015a2f0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:03.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:03.633+0000 7fe287b1d640 1 -- 172.21.15.3:0/754571725 >> 172.21.15.3:0/754571725 conn(0x7fe2800673b0 msgr2=0x7fe2800b5c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:03.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:03.633+0000 7fe287b1d640 1 -- 172.21.15.3:0/754571725 shutdown_connections 2024-05-09T06:36:03.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:03.633+0000 7fe287b1d640 1 -- 172.21.15.3:0/754571725 wait complete. 2024-05-09T06:36:03.625 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T06:36:03.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:03.748+0000 7f160c253640 1 Processor -- start 2024-05-09T06:36:03.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:03.748+0000 7f160c253640 1 -- start start 2024-05-09T06:36:03.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:03.749+0000 7f160c253640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f16040cf5b0 0x7f16040d3c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:03.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:03.749+0000 7f160c253640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f16040d2510 0x7f16040b9cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:03.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:03.749+0000 7f160c253640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f16040d2bd0 con 0x7f160415e4c0 2024-05-09T06:36:03.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:03.749+0000 7f160c253640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f16040d2970 con 0x7f16040d2510 2024-05-09T06:36:03.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:03.749+0000 7f160c253640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f16040d2eb0 con 0x7f16040cf5b0 2024-05-09T06:36:03.731 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:03.749+0000 7f160a7f0640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f16040d2510 0x7f16040b9cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:03.731 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:03.749+0000 7f160a7f0640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f16040d2510 0x7f16040b9cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45598/0 (socket says 172.21.15.3:45598) 2024-05-09T06:36:03.731 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:03.749+0000 7f160a7f0640 1 -- 172.21.15.3:0/3468021526 learned_addr learned my addr 172.21.15.3:0/3468021526 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:36:03.731 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:03.749+0000 7f16097ee640 1 --2- 172.21.15.3:0/3468021526 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f16040cf5b0 0x7f16040d3c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:03.731 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:03.749+0000 7f160a7f0640 1 -- 172.21.15.3:0/3468021526 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f16040cf5b0 msgr2=0x7f16040d3c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:03.731 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:03.749+0000 7f160a7f0640 1 --2- 172.21.15.3:0/3468021526 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f16040cf5b0 0x7f16040d3c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:03.731 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:03.749+0000 7f160a7f0640 1 -- 172.21.15.3:0/3468021526 >> v1:172.21.15.190:6789/0 conn(0x7f160415e4c0 legacy=0x7f16040ceea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:03.731 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:03.749+0000 7f160a7f0640 1 -- 172.21.15.3:0/3468021526 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16040d3190 con 0x7f16040d2510 2024-05-09T06:36:03.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:03.750+0000 7f160a7f0640 1 --2- 172.21.15.3:0/3468021526 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f16040d2510 0x7f16040b9cc0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f15fc009e50 tx=0x7f15fc02a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f3c3db0dafcd073 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:03.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:03.750+0000 7f1608fed640 1 -- 172.21.15.3:0/3468021526 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15fc036070 con 0x7f16040d2510 2024-05-09T06:36:03.733 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:03.750+0000 7f1608fed640 1 -- 172.21.15.3:0/3468021526 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f15fc031600 con 0x7f16040d2510 2024-05-09T06:36:03.733 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:03.750+0000 7f1608fed640 1 -- 172.21.15.3:0/3468021526 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15fc0319d0 con 0x7f16040d2510 2024-05-09T06:36:03.733 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:03.751+0000 7f160c253640 1 -- 172.21.15.3:0/3468021526 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f16040d2510 msgr2=0x7f16040b9cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:03.733 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:03.751+0000 7f160c253640 1 --2- 172.21.15.3:0/3468021526 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f16040d2510 0x7f16040b9cc0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f15fc009e50 tx=0x7f15fc02a5f0 comp rx=0 tx=0).stop 2024-05-09T06:36:03.733 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:03.751+0000 7f160c253640 1 -- 172.21.15.3:0/3468021526 shutdown_connections 2024-05-09T06:36:03.733 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:03.751+0000 7f160c253640 1 --2- 172.21.15.3:0/3468021526 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f16040d2510 0x7f16040b9cc0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:03.733 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:03.751+0000 7f160c253640 1 --2- 172.21.15.3:0/3468021526 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f16040cf5b0 0x7f16040d3c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:03.733 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:03.751+0000 7f160c253640 1 -- 172.21.15.3:0/3468021526 >> 172.21.15.3:0/3468021526 conn(0x7f16040673b0 msgr2=0x7f16040677c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:03.733 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:03.751+0000 7f160c253640 1 -- 172.21.15.3:0/3468021526 shutdown_connections 2024-05-09T06:36:03.733 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:03.751+0000 7f160c253640 1 -- 172.21.15.3:0/3468021526 wait complete. 2024-05-09T06:36:03.733 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:03.752+0000 7f160c253640 1 Processor -- start 2024-05-09T06:36:03.733 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:03.752+0000 7f160c253640 1 -- start start 2024-05-09T06:36:03.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:03.753+0000 7f160c253640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f16040cf5b0 0x7f160415bf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:03.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:03.753+0000 7f160c253640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f16040d2510 0x7f16040ca140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:03.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:03.753+0000 7f160c253640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f160415e4c0 0x7f16041546b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:03.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:03.753+0000 7f160c253640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f16040b7260 con 0x7f160415e4c0 2024-05-09T06:36:03.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:03.753+0000 7f160c253640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f16040d3750 con 0x7f16040d2510 2024-05-09T06:36:03.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:03.753+0000 7f160c253640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f16040b73e0 con 0x7f16040cf5b0 2024-05-09T06:36:03.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:03.753+0000 7f160a7f0640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f160415e4c0 0x7f16041546b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:03.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:03.753+0000 7f160a7f0640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f160415e4c0 0x7f16041546b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45612/0 (socket says 172.21.15.3:45612) 2024-05-09T06:36:03.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:03.753+0000 7f160a7f0640 1 -- 172.21.15.3:0/45806174 learned_addr learned my addr 172.21.15.3:0/45806174 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:36:03.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:03.753+0000 7f1609fef640 1 --2- 172.21.15.3:0/45806174 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f16040cf5b0 0x7f160415bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:03.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:03.753+0000 7f16097ee640 1 --2- 172.21.15.3:0/45806174 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f16040d2510 0x7f16040ca140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:03.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:03.753+0000 7f160a7f0640 1 -- 172.21.15.3:0/45806174 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f16040cf5b0 msgr2=0x7f160415bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:03.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:03.753+0000 7f160a7f0640 1 --2- 172.21.15.3:0/45806174 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f16040cf5b0 0x7f160415bf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:03.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:03.753+0000 7f160a7f0640 1 -- 172.21.15.3:0/45806174 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f16040d2510 msgr2=0x7f16040ca140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:03.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:03.753+0000 7f160a7f0640 1 --2- 172.21.15.3:0/45806174 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f16040d2510 0x7f16040ca140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:03.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:03.753+0000 7f160a7f0640 1 -- 172.21.15.3:0/45806174 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16040c8c80 con 0x7f160415e4c0 2024-05-09T06:36:03.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:03.753+0000 7f160a7f0640 1 --2- 172.21.15.3:0/45806174 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f160415e4c0 0x7f16041546b0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f15fc009e50 tx=0x7f15fc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:03.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:03.754+0000 7f15fa7fc640 1 -- 172.21.15.3:0/45806174 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15fc036050 con 0x7f160415e4c0 2024-05-09T06:36:03.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:03.754+0000 7f15fa7fc640 1 -- 172.21.15.3:0/45806174 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f15fc042070 con 0x7f160415e4c0 2024-05-09T06:36:03.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:03.754+0000 7f160c253640 1 -- 172.21.15.3:0/45806174 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16040c8f60 con 0x7f160415e4c0 2024-05-09T06:36:03.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:03.754+0000 7f15fa7fc640 1 -- 172.21.15.3:0/45806174 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15fc007bd0 con 0x7f160415e4c0 2024-05-09T06:36:03.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:03.754+0000 7f160c253640 1 -- 172.21.15.3:0/45806174 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16040c9240 con 0x7f160415e4c0 2024-05-09T06:36:03.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:03.755+0000 7f160c253640 1 -- 172.21.15.3:0/45806174 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f160404f660 con 0x7f160415e4c0 2024-05-09T06:36:03.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:03.755+0000 7f15fa7fc640 1 -- 172.21.15.3:0/45806174 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f15fc047460 con 0x7f160415e4c0 2024-05-09T06:36:03.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:03.756+0000 7f15fa7fc640 1 --2- 172.21.15.3:0/45806174 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f15e806be30 0x7f15e806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:03.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:03.756+0000 7f15fa7fc640 1 -- 172.21.15.3:0/45806174 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f15fc0ab860 con 0x7f160415e4c0 2024-05-09T06:36:03.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:03.756+0000 7f1609fef640 1 --2- 172.21.15.3:0/45806174 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f15e806be30 0x7f15e806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:03.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:03.756+0000 7f1609fef640 1 --2- 172.21.15.3:0/45806174 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f15e806be30 0x7f15e806e2f0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f16000016e0 tx=0x7f1600009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:03.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:03.760+0000 7f15fa7fc640 1 -- 172.21.15.3:0/45806174 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f15fc079fa0 con 0x7f160415e4c0 2024-05-09T06:36:03.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:03.944+0000 7f160c253640 1 -- 172.21.15.3:0/45806174 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1604155c10 con 0x7f160415e4c0 2024-05-09T06:36:03.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:03.945+0000 7f15fa7fc640 1 -- 172.21.15.3:0/45806174 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) ==== 74+0+12800 (secure 0 0 0) 0x7f15fc07e2a0 con 0x7f160415e4c0 2024-05-09T06:36:03.926 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-09T06:36:03.926 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":16,"fsid":"16ed737d-2665-431a-8ac6-ef242be3975a","created":"2024-05-09T06:35:42.541368+0000","modified":"2024-05-09T06:35:56.885471+0000","last_up_change":"2024-05-09T06:35:53.778274+0000","last_in_change":"2024-05-09T06:35:44.780586+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T06:35:54.819999+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fae3a8db-4826-4f3c-b33a-12c09bb5df6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":1897849827},{"type":"v1","addr":"172.21.15.3:6817","nonce":1897849827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":1897849827},{"type":"v1","addr":"172.21.15.3:6819","nonce":1897849827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":1897849827},{"type":"v1","addr":"172.21.15.3:6823","nonce":1897849827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":1897849827},{"type":"v1","addr":"172.21.15.3:6821","nonce":1897849827}]},"public_addr":"172.21.15.3:6817/1897849827","cluster_addr":"172.21.15.3:6819/1897849827","heartbeat_back_addr":"172.21.15.3:6823/1897849827","heartbeat_front_addr":"172.21.15.3:6821/1897849827","state":["exists","up"]},{"osd":1,"uuid":"646e882d-4ce4-4720-8c84-f71df563a35e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":926805498},{"type":"v1","addr":"172.21.15.3:6801","nonce":926805498}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":926805498},{"type":"v1","addr":"172.21.15.3:6803","nonce":926805498}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":926805498},{"type":"v1","addr":"172.21.15.3:6807","nonce":926805498}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":926805498},{"type":"v1","addr":"172.21.15.3:6805","nonce":926805498}]},"public_addr":"172.21.15.3:6801/926805498","cluster_addr":"172.21.15.3:6803/926805498","heartbeat_back_addr":"172.21.15.3:6807/926805498","heartbeat_front_addr":"172.21.15.3:6805/926805498","state":["exists","up"]},{"osd":2,"uuid":"1abbadb5-a570-4359-9643-acff3ede35a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":801295451},{"type":"v1","addr":"172.21.15.3:6809","nonce":801295451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":801295451},{"type":"v1","addr":"172.21.15.3:6811","nonce":801295451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":801295451},{"type":"v1","addr":"172.21.15.3:6815","nonce":801295451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":801295451},{"type":"v1","addr":"172.21.15.3:6813","nonce":801295451}]},"public_addr":"172.21.15.3:6809/801295451","cluster_addr":"172.21.15.3:6811/801295451","heartbeat_back_addr":"172.21.15.3:6815/801295451","heartbeat_front_addr":"172.21.15.3:6813/801295451","state":["exists","up"]},{"osd":3,"uuid":"62ac4dc1-2371-4dd2-acd3-e2dbb11aedf5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":1903389857},{"type":"v1","addr":"172.21.15.3:6825","nonce":1903389857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":1903389857},{"type":"v1","addr":"172.21.15.3:6827","nonce":1903389857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":1903389857},{"type":"v1","addr":"172.21.15.3:6831","nonce":1903389857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":1903389857},{"type":"v1","addr":"172.21.15.3:6829","nonce":1903389857}]},"public_addr":"172.21.15.3:6825/1903389857","cluster_addr":"172.21.15.3:6827/1903389857","heartbeat_back_addr":"172.21.15.3:6831/1903389857","heartbeat_front_addr":"172.21.15.3:6829/1903389857","state":["exists","up"]},{"osd":4,"uuid":"ae808398-8326-4d7c-9319-a2a0eafffb59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":2143624576},{"type":"v1","addr":"172.21.15.190:6801","nonce":2143624576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":2143624576},{"type":"v1","addr":"172.21.15.190:6803","nonce":2143624576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":2143624576},{"type":"v1","addr":"172.21.15.190:6807","nonce":2143624576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":2143624576},{"type":"v1","addr":"172.21.15.190:6805","nonce":2143624576}]},"public_addr":"172.21.15.190:6801/2143624576","cluster_addr":"172.21.15.190:6803/2143624576","heartbeat_back_addr":"172.21.15.190:6807/2143624576","heartbeat_front_addr":"172.21.15.190:6805/2143624576","state":["exists","up"]},{"osd":5,"uuid":"918adf50-c440-48ac-9a8d-4837562d7275","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":1766923126},{"type":"v1","addr":"172.21.15.190:6814","nonce":1766923126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":1766923126},{"type":"v1","addr":"172.21.15.190:6818","nonce":1766923126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":1766923126},{"type":"v1","addr":"172.21.15.190:6823","nonce":1766923126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":1766923126},{"type":"v1","addr":"172.21.15.190:6821","nonce":1766923126}]},"public_addr":"172.21.15.190:6814/1766923126","cluster_addr":"172.21.15.190:6818/1766923126","heartbeat_back_addr":"172.21.15.190:6823/1766923126","heartbeat_front_addr":"172.21.15.190:6821/1766923126","state":["exists","up"]},{"osd":6,"uuid":"ac2882bc-2a7c-463c-8f6c-76ff49828adc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":342414248},{"type":"v1","addr":"172.21.15.190:6825","nonce":342414248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":342414248},{"type":"v1","addr":"172.21.15.190:6827","nonce":342414248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":342414248},{"type":"v1","addr":"172.21.15.190:6831","nonce":342414248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":342414248},{"type":"v1","addr":"172.21.15.190:6829","nonce":342414248}]},"public_addr":"172.21.15.190:6825/342414248","cluster_addr":"172.21.15.190:6827/342414248","heartbeat_back_addr":"172.21.15.190:6831/342414248","heartbeat_front_addr":"172.21.15.190:6829/342414248","state":["exists","up"]},{"osd":7,"uuid":"dd4e767a-4730-4c70-8bd5-3f088140a5a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":537849891},{"type":"v1","addr":"172.21.15.190:6809","nonce":537849891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":537849891},{"type":"v1","addr":"172.21.15.190:6811","nonce":537849891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6817","nonce":537849891},{"type":"v1","addr":"172.21.15.190:6819","nonce":537849891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6813","nonce":537849891},{"type":"v1","addr":"172.21.15.190:6815","nonce":537849891}]},"public_addr":"172.21.15.190:6809/537849891","cluster_addr":"172.21.15.190:6811/537849891","heartbeat_back_addr":"172.21.15.190:6819/537849891","heartbeat_front_addr":"172.21.15.190:6815/537849891","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.889784+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.974002+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.973549+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.955396+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.928471+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.902418+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.958696+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.926474+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T06:36:03.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:03.947+0000 7f160c253640 1 -- 172.21.15.3:0/45806174 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f15e806be30 msgr2=0x7f15e806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:03.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:03.947+0000 7f160c253640 1 --2- 172.21.15.3:0/45806174 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f15e806be30 0x7f15e806e2f0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f16000016e0 tx=0x7f1600009040 comp rx=0 tx=0).stop 2024-05-09T06:36:03.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:03.947+0000 7f160c253640 1 -- 172.21.15.3:0/45806174 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f160415e4c0 msgr2=0x7f16041546b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:03.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:03.947+0000 7f160c253640 1 --2- 172.21.15.3:0/45806174 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f160415e4c0 0x7f16041546b0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f15fc009e50 tx=0x7f15fc004a40 comp rx=0 tx=0).stop 2024-05-09T06:36:03.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:03.947+0000 7f160c253640 1 -- 172.21.15.3:0/45806174 shutdown_connections 2024-05-09T06:36:03.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:03.947+0000 7f160c253640 1 --2- 172.21.15.3:0/45806174 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f15e806be30 0x7f15e806e2f0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:03.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:03.947+0000 7f160c253640 1 --2- 172.21.15.3:0/45806174 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f160415e4c0 0x7f16041546b0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:03.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:03.947+0000 7f160c253640 1 --2- 172.21.15.3:0/45806174 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f16040d2510 0x7f16040ca140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:03.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:03.947+0000 7f160c253640 1 --2- 172.21.15.3:0/45806174 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f16040cf5b0 0x7f160415bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:03.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:03.947+0000 7f160c253640 1 -- 172.21.15.3:0/45806174 >> 172.21.15.3:0/45806174 conn(0x7f16040673b0 msgr2=0x7f16040b8890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:03.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:03.947+0000 7f160c253640 1 -- 172.21.15.3:0/45806174 shutdown_connections 2024-05-09T06:36:03.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:03.947+0000 7f160c253640 1 -- 172.21.15.3:0/45806174 wait complete. 2024-05-09T06:36:03.939 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-09T06:35:54.819999+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-09T06:36:03.939 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-09T06:36:04.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:04.066+0000 7f6235831640 1 Processor -- start 2024-05-09T06:36:04.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:04.066+0000 7f6235831640 1 -- start start 2024-05-09T06:36:04.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:04.067+0000 7f6235831640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f62300d11f0 0x7f623015ff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:04.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:04.067+0000 7f6235831640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f62300d1fd0 0x7f62300d27d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:04.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:04.067+0000 7f6235831640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f62300d15d0 con 0x7f62300d2d10 2024-05-09T06:36:04.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:04.067+0000 7f6235831640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f62300cf0a0 con 0x7f62300d1fd0 2024-05-09T06:36:04.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:04.067+0000 7f6235831640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f62300ced70 con 0x7f62300d11f0 2024-05-09T06:36:04.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:04.067+0000 7f62267fc640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f62300d1fd0 0x7f62300d27d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:04.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:04.067+0000 7f622f7fe640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f62300d2d10 0x7f62300cf530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:33028/0 (socket says 172.21.15.3:33028) 2024-05-09T06:36:04.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:04.067+0000 7f622f7fe640 1 -- 172.21.15.3:0/1483900705 learned_addr learned my addr 172.21.15.3:0/1483900705 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-05-09T06:36:04.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:04.067+0000 7f622effd640 1 --2- 172.21.15.3:0/1483900705 >> v2:172.21.15.190:3300/0 conn(0x7f62300d11f0 0x7f623015ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:04.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:04.067+0000 7f62267fc640 1 -- 172.21.15.3:0/1483900705 >> v2:172.21.15.190:3300/0 conn(0x7f62300d11f0 msgr2=0x7f623015ff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:04.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:04.067+0000 7f62267fc640 1 --2- 172.21.15.3:0/1483900705 >> v2:172.21.15.190:3300/0 conn(0x7f62300d11f0 0x7f623015ff40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:04.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:04.067+0000 7f62267fc640 1 -- 172.21.15.3:0/1483900705 >> v1:172.21.15.3:6789/0 conn(0x7f62300d2d10 legacy=0x7f62300cf530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:04.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:04.067+0000 7f62267fc640 1 -- 172.21.15.3:0/1483900705 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6230160480 con 0x7f62300d1fd0 2024-05-09T06:36:04.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:04.067+0000 7f622e7fc640 1 -- 172.21.15.3:0/1483900705 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 249131636 0 0) 0x7f62300d15d0 con 0x7f62300d2d10 2024-05-09T06:36:04.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:04.067+0000 7f622effd640 1 --2- 172.21.15.3:0/1483900705 >> v2:172.21.15.190:3300/0 conn(0x7f62300d11f0 0x7f623015ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:36:04.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:04.068+0000 7f62267fc640 1 --2- 172.21.15.3:0/1483900705 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f62300d1fd0 0x7f62300d27d0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f623006b9e0 tx=0x7f621802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=841e42ebad002da3 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:04.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:04.068+0000 7f622e7fc640 1 -- 172.21.15.3:0/1483900705 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6218037070 con 0x7f62300d1fd0 2024-05-09T06:36:04.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:04.068+0000 7f622e7fc640 1 -- 172.21.15.3:0/1483900705 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6218032440 con 0x7f62300d1fd0 2024-05-09T06:36:04.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:04.068+0000 7f622e7fc640 1 -- 172.21.15.3:0/1483900705 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6218032810 con 0x7f62300d1fd0 2024-05-09T06:36:04.053 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:04.069+0000 7f6235831640 1 -- 172.21.15.3:0/1483900705 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f62300d1fd0 msgr2=0x7f62300d27d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:04.053 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:04.069+0000 7f6235831640 1 --2- 172.21.15.3:0/1483900705 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f62300d1fd0 0x7f62300d27d0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f623006b9e0 tx=0x7f621802a1a0 comp rx=0 tx=0).stop 2024-05-09T06:36:04.053 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:04.069+0000 7f6235831640 1 -- 172.21.15.3:0/1483900705 shutdown_connections 2024-05-09T06:36:04.053 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:04.069+0000 7f6235831640 1 --2- 172.21.15.3:0/1483900705 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f62300d1fd0 0x7f62300d27d0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:04.053 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:04.069+0000 7f6235831640 1 --2- 172.21.15.3:0/1483900705 >> v2:172.21.15.190:3300/0 conn(0x7f62300d11f0 0x7f623015ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:04.053 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:04.069+0000 7f6235831640 1 -- 172.21.15.3:0/1483900705 >> 172.21.15.3:0/1483900705 conn(0x7f623009d4d0 msgr2=0x7f62300d41f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:04.053 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:04.069+0000 7f6235831640 1 -- 172.21.15.3:0/1483900705 shutdown_connections 2024-05-09T06:36:04.053 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:04.069+0000 7f6235831640 1 -- 172.21.15.3:0/1483900705 wait complete. 2024-05-09T06:36:04.053 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:04.070+0000 7f6235831640 1 Processor -- start 2024-05-09T06:36:04.053 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:04.070+0000 7f6235831640 1 -- start start 2024-05-09T06:36:04.053 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:04.071+0000 7f6235831640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f62300d11f0 0x7f62300b7da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:04.053 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:04.071+0000 7f6235831640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f62300d1fd0 0x7f623014ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:04.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:04.071+0000 7f6235831640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f62300d2d10 0x7f62300b63d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:04.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:04.071+0000 7f6235831640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f623015f900 con 0x7f62300d1fd0 2024-05-09T06:36:04.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:04.071+0000 7f6235831640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f623015f620 con 0x7f62300d2d10 2024-05-09T06:36:04.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:04.071+0000 7f6235831640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f62300cfc40 con 0x7f62300d11f0 2024-05-09T06:36:04.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:04.071+0000 7f62267fc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f62300d1fd0 0x7f623014ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:04.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:04.071+0000 7f62267fc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f62300d1fd0 0x7f623014ce80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45624/0 (socket says 172.21.15.3:45624) 2024-05-09T06:36:04.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:04.071+0000 7f62267fc640 1 -- 172.21.15.3:0/3856346586 learned_addr learned my addr 172.21.15.3:0/3856346586 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:36:04.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:04.071+0000 7f622f7fe640 1 --2- 172.21.15.3:0/3856346586 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f62300d2d10 0x7f62300b63d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:04.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:04.071+0000 7f622effd640 1 --2- 172.21.15.3:0/3856346586 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f62300d11f0 0x7f62300b7da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:04.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:04.071+0000 7f62267fc640 1 -- 172.21.15.3:0/3856346586 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f62300d11f0 msgr2=0x7f62300b7da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:04.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:04.071+0000 7f62267fc640 1 --2- 172.21.15.3:0/3856346586 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f62300d11f0 0x7f62300b7da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:04.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:04.071+0000 7f62267fc640 1 -- 172.21.15.3:0/3856346586 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f62300d2d10 msgr2=0x7f62300b63d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:04.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:04.071+0000 7f62267fc640 1 --2- 172.21.15.3:0/3856346586 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f62300d2d10 0x7f62300b63d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:04.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:04.071+0000 7f62267fc640 1 -- 172.21.15.3:0/3856346586 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f623014d3c0 con 0x7f62300d1fd0 2024-05-09T06:36:04.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:04.071+0000 7f622effd640 1 --2- 172.21.15.3:0/3856346586 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f62300d11f0 0x7f62300b7da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:36:04.055 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:04.071+0000 7f62267fc640 1 --2- 172.21.15.3:0/3856346586 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f62300d1fd0 0x7f623014ce80 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f6218000c00 tx=0x7f62180046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:04.055 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:04.071+0000 7f622cff9640 1 -- 172.21.15.3:0/3856346586 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6218041070 con 0x7f62300d1fd0 2024-05-09T06:36:04.055 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:04.071+0000 7f622cff9640 1 -- 172.21.15.3:0/3856346586 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f621803c890 con 0x7f62300d1fd0 2024-05-09T06:36:04.055 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:04.072+0000 7f622cff9640 1 -- 172.21.15.3:0/3856346586 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6218037040 con 0x7f62300d1fd0 2024-05-09T06:36:04.055 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:04.072+0000 7f6235831640 1 -- 172.21.15.3:0/3856346586 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62300b8380 con 0x7f62300d1fd0 2024-05-09T06:36:04.055 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:04.072+0000 7f6235831640 1 -- 172.21.15.3:0/3856346586 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62300b8940 con 0x7f62300d1fd0 2024-05-09T06:36:04.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:04.073+0000 7f6235831640 1 -- 172.21.15.3:0/3856346586 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62300b9e80 con 0x7f62300d1fd0 2024-05-09T06:36:04.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:04.073+0000 7f622cff9640 1 -- 172.21.15.3:0/3856346586 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6218044070 con 0x7f62300d1fd0 2024-05-09T06:36:04.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:04.074+0000 7f622cff9640 1 --2- 172.21.15.3:0/3856346586 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f621006be30 0x7f621006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:04.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:04.074+0000 7f622cff9640 1 -- 172.21.15.3:0/3856346586 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f62180ab900 con 0x7f62300d1fd0 2024-05-09T06:36:04.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:04.074+0000 7f622effd640 1 --2- 172.21.15.3:0/3856346586 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f621006be30 0x7f621006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:04.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:04.074+0000 7f622effd640 1 --2- 172.21.15.3:0/3856346586 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f621006be30 0x7f621006e2f0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f622000aa30 tx=0x7f6220008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:04.058 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:04.078+0000 7f622cff9640 1 -- 172.21.15.3:0/3856346586 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f62300b9e80 con 0x7f62300d1fd0 2024-05-09T06:36:04.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:04.259+0000 7f6235831640 1 -- 172.21.15.3:0/3856346586 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f6230069550 con 0x7f62300d1fd0 2024-05-09T06:36:04.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:04.260+0000 7f622cff9640 1 -- 172.21.15.3:0/3856346586 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) ==== 93+0+10 (secure 0 0 0) 0x7f621807a040 con 0x7f62300d1fd0 2024-05-09T06:36:04.241 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 1 2024-05-09T06:36:04.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:04.261+0000 7f6235831640 1 -- 172.21.15.3:0/3856346586 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f621006be30 msgr2=0x7f621006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:04.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:04.261+0000 7f6235831640 1 --2- 172.21.15.3:0/3856346586 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f621006be30 0x7f621006e2f0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f622000aa30 tx=0x7f6220008040 comp rx=0 tx=0).stop 2024-05-09T06:36:04.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:04.261+0000 7f6235831640 1 -- 172.21.15.3:0/3856346586 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f62300d1fd0 msgr2=0x7f623014ce80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:04.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:04.261+0000 7f6235831640 1 --2- 172.21.15.3:0/3856346586 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f62300d1fd0 0x7f623014ce80 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f6218000c00 tx=0x7f62180046c0 comp rx=0 tx=0).stop 2024-05-09T06:36:04.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:04.262+0000 7f6235831640 1 -- 172.21.15.3:0/3856346586 shutdown_connections 2024-05-09T06:36:04.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:04.262+0000 7f6235831640 1 --2- 172.21.15.3:0/3856346586 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f621006be30 0x7f621006e2f0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:04.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:04.262+0000 7f6235831640 1 --2- 172.21.15.3:0/3856346586 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f62300d2d10 0x7f62300b63d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:04.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:04.262+0000 7f6235831640 1 --2- 172.21.15.3:0/3856346586 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f62300d1fd0 0x7f623014ce80 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:04.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:04.262+0000 7f6235831640 1 --2- 172.21.15.3:0/3856346586 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f62300d11f0 0x7f62300b7da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:04.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:04.262+0000 7f6235831640 1 -- 172.21.15.3:0/3856346586 >> 172.21.15.3:0/3856346586 conn(0x7f623009d4d0 msgr2=0x7f62300cd2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:04.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:04.262+0000 7f6235831640 1 -- 172.21.15.3:0/3856346586 shutdown_connections 2024-05-09T06:36:04.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:04.262+0000 7f6235831640 1 -- 172.21.15.3:0/3856346586 wait complete. 2024-05-09T06:36:04.253 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-05-09T06:36:04.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:04.377+0000 7f58e9ba9640 1 Processor -- start 2024-05-09T06:36:04.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:04.377+0000 7f58e9ba9640 1 -- start start 2024-05-09T06:36:04.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:04.378+0000 7f58e9ba9640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f58e415e350 0x7f58e40d3b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:04.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:04.378+0000 7f58e9ba9640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f58e40cee10 0x7f58e40cf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:04.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:04.378+0000 7f58e9ba9640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f58e40cf7c0 con 0x7f58e40b9e40 2024-05-09T06:36:04.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:04.378+0000 7f58e9ba9640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f58e40d2890 con 0x7f58e415e350 2024-05-09T06:36:04.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:04.378+0000 7f58e9ba9640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f58e40d2af0 con 0x7f58e40cee10 2024-05-09T06:36:04.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:04.378+0000 7f58e37fe640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f58e415e350 0x7f58e40d3b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:04.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:04.378+0000 7f58e2ffd640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f58e40b9e40 0x7f58e40d2180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:33040/0 (socket says 172.21.15.3:33040) 2024-05-09T06:36:04.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:04.378+0000 7f58e2ffd640 1 -- 172.21.15.3:0/3876798094 learned_addr learned my addr 172.21.15.3:0/3876798094 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-05-09T06:36:04.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:04.378+0000 7f58e3fff640 1 --2- 172.21.15.3:0/3876798094 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f58e40cee10 0x7f58e40cf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:04.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:04.378+0000 7f58e37fe640 1 -- 172.21.15.3:0/3876798094 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f58e40cee10 msgr2=0x7f58e40cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:04.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:04.378+0000 7f58e37fe640 1 --2- 172.21.15.3:0/3876798094 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f58e40cee10 0x7f58e40cf280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:04.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:04.378+0000 7f58e37fe640 1 -- 172.21.15.3:0/3876798094 >> v1:172.21.15.3:6789/0 conn(0x7f58e40b9e40 legacy=0x7f58e40d2180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:04.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:04.378+0000 7f58e27fc640 1 -- 172.21.15.3:0/3876798094 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1416659449 0 0) 0x7f58e40cf7c0 con 0x7f58e40b9e40 2024-05-09T06:36:04.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:04.378+0000 7f58e37fe640 1 -- 172.21.15.3:0/3876798094 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58e40d2dd0 con 0x7f58e415e350 2024-05-09T06:36:04.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:04.379+0000 7f58e37fe640 1 --2- 172.21.15.3:0/3876798094 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f58e415e350 0x7f58e40d3b90 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f58d40097a0 tx=0x7f58d402a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5023e1ff8731c1c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:04.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:04.379+0000 7f58e27fc640 1 -- 172.21.15.3:0/3876798094 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58d4037070 con 0x7f58e415e350 2024-05-09T06:36:04.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:04.379+0000 7f58e27fc640 1 -- 172.21.15.3:0/3876798094 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58d4032440 con 0x7f58e415e350 2024-05-09T06:36:04.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:04.379+0000 7f58e27fc640 1 -- 172.21.15.3:0/3876798094 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58d4032810 con 0x7f58e415e350 2024-05-09T06:36:04.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:04.380+0000 7f58e9ba9640 1 -- 172.21.15.3:0/3876798094 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f58e415e350 msgr2=0x7f58e40d3b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:04.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:04.380+0000 7f58e9ba9640 1 --2- 172.21.15.3:0/3876798094 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f58e415e350 0x7f58e40d3b90 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f58d40097a0 tx=0x7f58d402a210 comp rx=0 tx=0).stop 2024-05-09T06:36:04.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:04.380+0000 7f58e9ba9640 1 -- 172.21.15.3:0/3876798094 shutdown_connections 2024-05-09T06:36:04.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:04.380+0000 7f58e9ba9640 1 --2- 172.21.15.3:0/3876798094 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f58e40cee10 0x7f58e40cf280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:04.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:04.380+0000 7f58e9ba9640 1 --2- 172.21.15.3:0/3876798094 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f58e415e350 0x7f58e40d3b90 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:04.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:04.380+0000 7f58e9ba9640 1 -- 172.21.15.3:0/3876798094 >> 172.21.15.3:0/3876798094 conn(0x7f58e40675c0 msgr2=0x7f58e40d0bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:04.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:04.380+0000 7f58e9ba9640 1 -- 172.21.15.3:0/3876798094 shutdown_connections 2024-05-09T06:36:04.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:04.380+0000 7f58e9ba9640 1 -- 172.21.15.3:0/3876798094 wait complete. 2024-05-09T06:36:04.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:04.381+0000 7f58e9ba9640 1 Processor -- start 2024-05-09T06:36:04.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:04.381+0000 7f58e9ba9640 1 -- start start 2024-05-09T06:36:04.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:04.382+0000 7f58e9ba9640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f58e40b9e40 0x7f58e415e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:04.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:04.382+0000 7f58e9ba9640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f58e40cee10 0x7f58e415ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:04.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:04.382+0000 7f58e9ba9640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f58e415e350 0x7f58e414cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:04.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:04.382+0000 7f58e9ba9640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f58e40d3670 con 0x7f58e40cee10 2024-05-09T06:36:04.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:04.382+0000 7f58e9ba9640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f58e40d3390 con 0x7f58e40b9e40 2024-05-09T06:36:04.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:04.382+0000 7f58e9ba9640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f58e40b71b0 con 0x7f58e415e350 2024-05-09T06:36:04.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:04.382+0000 7f58e2ffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f58e40cee10 0x7f58e415ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:04.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:04.382+0000 7f58e2ffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f58e40cee10 0x7f58e415ed20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45644/0 (socket says 172.21.15.3:45644) 2024-05-09T06:36:04.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:04.382+0000 7f58e2ffd640 1 -- 172.21.15.3:0/3415147913 learned_addr learned my addr 172.21.15.3:0/3415147913 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:36:04.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:04.382+0000 7f58e37fe640 1 --2- 172.21.15.3:0/3415147913 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f58e40b9e40 0x7f58e415e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:04.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:04.382+0000 7f58e3fff640 1 --2- 172.21.15.3:0/3415147913 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f58e415e350 0x7f58e414cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:04.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:04.382+0000 7f58e2ffd640 1 -- 172.21.15.3:0/3415147913 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f58e415e350 msgr2=0x7f58e414cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:04.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:04.382+0000 7f58e2ffd640 1 --2- 172.21.15.3:0/3415147913 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f58e415e350 0x7f58e414cdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:04.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:04.382+0000 7f58e2ffd640 1 -- 172.21.15.3:0/3415147913 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f58e40b9e40 msgr2=0x7f58e415e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:04.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:04.382+0000 7f58e2ffd640 1 --2- 172.21.15.3:0/3415147913 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f58e40b9e40 0x7f58e415e7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:04.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:04.382+0000 7f58e2ffd640 1 -- 172.21.15.3:0/3415147913 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58e415f580 con 0x7f58e40cee10 2024-05-09T06:36:04.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:04.382+0000 7f58e3fff640 1 --2- 172.21.15.3:0/3415147913 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f58e415e350 0x7f58e414cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:36:04.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:04.383+0000 7f58e2ffd640 1 --2- 172.21.15.3:0/3415147913 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f58e40cee10 0x7f58e415ed20 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f58d800c4b0 tx=0x7f58d800c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:04.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:04.383+0000 7f58cbfff640 1 -- 172.21.15.3:0/3415147913 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58d8018070 con 0x7f58e40cee10 2024-05-09T06:36:04.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:04.383+0000 7f58cbfff640 1 -- 172.21.15.3:0/3415147913 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58d8007500 con 0x7f58e40cee10 2024-05-09T06:36:04.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:04.383+0000 7f58cbfff640 1 -- 172.21.15.3:0/3415147913 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58d8004a20 con 0x7f58e40cee10 2024-05-09T06:36:04.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:04.383+0000 7f58e9ba9640 1 -- 172.21.15.3:0/3415147913 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58e415f860 con 0x7f58e40cee10 2024-05-09T06:36:04.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:04.383+0000 7f58e9ba9640 1 -- 172.21.15.3:0/3415147913 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58e415fe20 con 0x7f58e40cee10 2024-05-09T06:36:04.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:04.384+0000 7f58e9ba9640 1 -- 172.21.15.3:0/3415147913 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58e404f660 con 0x7f58e40cee10 2024-05-09T06:36:04.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:04.384+0000 7f58cbfff640 1 -- 172.21.15.3:0/3415147913 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f58d801b070 con 0x7f58e40cee10 2024-05-09T06:36:04.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:04.385+0000 7f58cbfff640 1 --2- 172.21.15.3:0/3415147913 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f58d006be30 0x7f58d006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:04.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:04.385+0000 7f58cbfff640 1 -- 172.21.15.3:0/3415147913 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f58d8089d50 con 0x7f58e40cee10 2024-05-09T06:36:04.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:04.385+0000 7f58e37fe640 1 --2- 172.21.15.3:0/3415147913 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f58d006be30 0x7f58d006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:04.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:04.386+0000 7f58e37fe640 1 --2- 172.21.15.3:0/3415147913 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f58d006be30 0x7f58d006e2f0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f58d4034000 tx=0x7f58d402e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:04.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:04.389+0000 7f58cbfff640 1 -- 172.21.15.3:0/3415147913 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f58d808d030 con 0x7f58e40cee10 2024-05-09T06:36:04.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:04.577+0000 7f58e9ba9640 1 -- 172.21.15.3:0/3415147913 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f58e40cf280 con 0x7f58e40cee10 2024-05-09T06:36:04.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:04.578+0000 7f58cbfff640 1 -- 172.21.15.3:0/3415147913 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v16) ==== 84+0+65 (secure 0 0 0) 0x7f58d8058550 con 0x7f58e40cee10 2024-05-09T06:36:04.559 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-09T06:36:04.559 INFO:teuthology.orchestra.run.smithi003.stdout:[ 2024-05-09T06:36:04.559 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-09T06:36:04.559 INFO:teuthology.orchestra.run.smithi003.stdout: "poolnum": 1, 2024-05-09T06:36:04.559 INFO:teuthology.orchestra.run.smithi003.stdout: "poolname": ".mgr" 2024-05-09T06:36:04.559 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-09T06:36:04.559 INFO:teuthology.orchestra.run.smithi003.stdout:] 2024-05-09T06:36:04.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:04.579+0000 7f58e9ba9640 1 -- 172.21.15.3:0/3415147913 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f58d006be30 msgr2=0x7f58d006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:04.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:04.579+0000 7f58e9ba9640 1 --2- 172.21.15.3:0/3415147913 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f58d006be30 0x7f58d006e2f0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f58d4034000 tx=0x7f58d402e000 comp rx=0 tx=0).stop 2024-05-09T06:36:04.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:04.579+0000 7f58e9ba9640 1 -- 172.21.15.3:0/3415147913 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f58e40cee10 msgr2=0x7f58e415ed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:04.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:04.579+0000 7f58e9ba9640 1 --2- 172.21.15.3:0/3415147913 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f58e40cee10 0x7f58e415ed20 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f58d800c4b0 tx=0x7f58d800c970 comp rx=0 tx=0).stop 2024-05-09T06:36:04.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:04.580+0000 7f58e9ba9640 1 -- 172.21.15.3:0/3415147913 shutdown_connections 2024-05-09T06:36:04.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:04.580+0000 7f58e9ba9640 1 --2- 172.21.15.3:0/3415147913 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f58d006be30 0x7f58d006e2f0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:04.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:04.580+0000 7f58e9ba9640 1 --2- 172.21.15.3:0/3415147913 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f58e415e350 0x7f58e414cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:04.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:04.580+0000 7f58e9ba9640 1 --2- 172.21.15.3:0/3415147913 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f58e40cee10 0x7f58e415ed20 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:04.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:04.580+0000 7f58e9ba9640 1 --2- 172.21.15.3:0/3415147913 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f58e40b9e40 0x7f58e415e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:04.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:04.580+0000 7f58e9ba9640 1 -- 172.21.15.3:0/3415147913 >> 172.21.15.3:0/3415147913 conn(0x7f58e40675c0 msgr2=0x7f58e40b3ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:04.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:04.580+0000 7f58e9ba9640 1 -- 172.21.15.3:0/3415147913 shutdown_connections 2024-05-09T06:36:04.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:04.580+0000 7f58e9ba9640 1 -- 172.21.15.3:0/3415147913 wait complete. 2024-05-09T06:36:04.571 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-05-09T06:36:04.571 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-05-09T06:36:04.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:04.698+0000 7fd82854f640 1 Processor -- start 2024-05-09T06:36:04.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:04.698+0000 7fd82854f640 1 -- start start 2024-05-09T06:36:04.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:04.699+0000 7fd82854f640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fd82015e400 0x7fd8200b99c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:04.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:04.699+0000 7fd82854f640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd8200cee30 0x7fd8200cf2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:04.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:04.699+0000 7fd82854f640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd8200b9f00 con 0x7fd8200b32b0 2024-05-09T06:36:04.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:04.699+0000 7fd82854f640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fd8200d2f50 con 0x7fd82015e400 2024-05-09T06:36:04.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:04.699+0000 7fd82854f640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fd8200cf800 con 0x7fd8200cee30 2024-05-09T06:36:04.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:04.699+0000 7fd8262eb640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fd82015e400 0x7fd8200b99c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:04.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:04.699+0000 7fd8262eb640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fd82015e400 0x7fd8200b99c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45650/0 (socket says 172.21.15.3:45650) 2024-05-09T06:36:04.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:04.699+0000 7fd8262eb640 1 -- 172.21.15.3:0/2905486467 learned_addr learned my addr 172.21.15.3:0/2905486467 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:36:04.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:04.700+0000 7fd8262eb640 1 -- 172.21.15.3:0/2905486467 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd8200cee30 msgr2=0x7fd8200cf2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:04.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:04.700+0000 7fd826aec640 1 --2- 172.21.15.3:0/2905486467 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd8200cee30 0x7fd8200cf2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:04.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:04.700+0000 7fd8262eb640 1 --2- 172.21.15.3:0/2905486467 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd8200cee30 0x7fd8200cf2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:04.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:04.700+0000 7fd8262eb640 1 -- 172.21.15.3:0/2905486467 >> v1:172.21.15.190:6789/0 conn(0x7fd8200b32b0 legacy=0x7fd8200b71d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:04.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:04.700+0000 7fd8262eb640 1 -- 172.21.15.3:0/2905486467 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8200d2620 con 0x7fd82015e400 2024-05-09T06:36:04.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:04.700+0000 7fd826aec640 1 --2- 172.21.15.3:0/2905486467 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd8200cee30 0x7fd8200cf2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:36:04.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:04.700+0000 7fd8262eb640 1 --2- 172.21.15.3:0/2905486467 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd82015e400 0x7fd8200b99c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fd818009870 tx=0x7fd81802a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3e4c7ab5251e3ea server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:04.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:04.701+0000 7fd8252e9640 1 -- 172.21.15.3:0/2905486467 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd818037070 con 0x7fd82015e400 2024-05-09T06:36:04.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:04.701+0000 7fd8252e9640 1 -- 172.21.15.3:0/2905486467 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd818032440 con 0x7fd82015e400 2024-05-09T06:36:04.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:04.701+0000 7fd8252e9640 1 -- 172.21.15.3:0/2905486467 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd818032810 con 0x7fd82015e400 2024-05-09T06:36:04.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:04.701+0000 7fd82854f640 1 -- 172.21.15.3:0/2905486467 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd82015e400 msgr2=0x7fd8200b99c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:04.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:04.701+0000 7fd82854f640 1 --2- 172.21.15.3:0/2905486467 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd82015e400 0x7fd8200b99c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fd818009870 tx=0x7fd81802a2a0 comp rx=0 tx=0).stop 2024-05-09T06:36:04.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:04.701+0000 7fd82854f640 1 -- 172.21.15.3:0/2905486467 shutdown_connections 2024-05-09T06:36:04.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:04.701+0000 7fd82854f640 1 --2- 172.21.15.3:0/2905486467 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd8200cee30 0x7fd8200cf2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:04.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:04.701+0000 7fd82854f640 1 --2- 172.21.15.3:0/2905486467 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd82015e400 0x7fd8200b99c0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:04.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:04.701+0000 7fd82854f640 1 -- 172.21.15.3:0/2905486467 >> 172.21.15.3:0/2905486467 conn(0x7fd8200675c0 msgr2=0x7fd8200d0be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:04.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:04.701+0000 7fd82854f640 1 -- 172.21.15.3:0/2905486467 shutdown_connections 2024-05-09T06:36:04.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:04.701+0000 7fd82854f640 1 -- 172.21.15.3:0/2905486467 wait complete. 2024-05-09T06:36:04.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:04.703+0000 7fd82854f640 1 Processor -- start 2024-05-09T06:36:04.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:04.703+0000 7fd82854f640 1 -- start start 2024-05-09T06:36:04.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:04.703+0000 7fd82854f640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd8200b32b0 0x7fd8201f4130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:04.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:04.703+0000 7fd82854f640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd8200cee30 0x7fd8200cddc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:04.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:04.703+0000 7fd82854f640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd82015e400 0x7fd8200cce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:04.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:04.703+0000 7fd82854f640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fd8200d33e0 con 0x7fd8200cee30 2024-05-09T06:36:04.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:04.703+0000 7fd82854f640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fd8200d2be0 con 0x7fd8200b32b0 2024-05-09T06:36:04.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:04.703+0000 7fd82854f640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fd8200d3560 con 0x7fd82015e400 2024-05-09T06:36:04.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:04.703+0000 7fd825aea640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd8200cee30 0x7fd8200cddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:04.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:04.703+0000 7fd825aea640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd8200cee30 0x7fd8200cddc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45666/0 (socket says 172.21.15.3:45666) 2024-05-09T06:36:04.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:04.704+0000 7fd825aea640 1 -- 172.21.15.3:0/1640737238 learned_addr learned my addr 172.21.15.3:0/1640737238 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:36:04.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:04.704+0000 7fd8262eb640 1 --2- 172.21.15.3:0/1640737238 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd8200b32b0 0x7fd8201f4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:04.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:04.704+0000 7fd826aec640 1 --2- 172.21.15.3:0/1640737238 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd82015e400 0x7fd8200cce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:04.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:04.704+0000 7fd825aea640 1 -- 172.21.15.3:0/1640737238 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd82015e400 msgr2=0x7fd8200cce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:04.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:04.704+0000 7fd825aea640 1 --2- 172.21.15.3:0/1640737238 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd82015e400 0x7fd8200cce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:04.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:04.704+0000 7fd825aea640 1 -- 172.21.15.3:0/1640737238 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd8200b32b0 msgr2=0x7fd8201f4130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:04.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:04.704+0000 7fd825aea640 1 --2- 172.21.15.3:0/1640737238 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd8200b32b0 0x7fd8201f4130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:04.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:04.704+0000 7fd825aea640 1 -- 172.21.15.3:0/1640737238 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8200c8230 con 0x7fd8200cee30 2024-05-09T06:36:04.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:04.704+0000 7fd826aec640 1 --2- 172.21.15.3:0/1640737238 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd82015e400 0x7fd8200cce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:36:04.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:04.704+0000 7fd825aea640 1 --2- 172.21.15.3:0/1640737238 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd8200cee30 0x7fd8200cddc0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fd81400c430 tx=0x7fd81400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:04.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:04.704+0000 7fd81effd640 1 -- 172.21.15.3:0/1640737238 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd814011790 con 0x7fd8200cee30 2024-05-09T06:36:04.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:04.704+0000 7fd81effd640 1 -- 172.21.15.3:0/1640737238 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd814017070 con 0x7fd8200cee30 2024-05-09T06:36:04.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:04.704+0000 7fd82854f640 1 -- 172.21.15.3:0/1640737238 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8200c8510 con 0x7fd8200cee30 2024-05-09T06:36:04.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:04.704+0000 7fd81effd640 1 -- 172.21.15.3:0/1640737238 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd814011bb0 con 0x7fd8200cee30 2024-05-09T06:36:04.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:04.704+0000 7fd82854f640 1 -- 172.21.15.3:0/1640737238 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8200c8db0 con 0x7fd8200cee30 2024-05-09T06:36:04.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:04.705+0000 7fd82854f640 1 -- 172.21.15.3:0/1640737238 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8200c86e0 con 0x7fd8200cee30 2024-05-09T06:36:04.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:04.706+0000 7fd81effd640 1 -- 172.21.15.3:0/1640737238 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd81401c460 con 0x7fd8200cee30 2024-05-09T06:36:04.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:04.706+0000 7fd81effd640 1 --2- 172.21.15.3:0/1640737238 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fd80c06be30 0x7fd80c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:04.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:04.706+0000 7fd81effd640 1 -- 172.21.15.3:0/1640737238 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7fd81408a930 con 0x7fd8200cee30 2024-05-09T06:36:04.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:04.707+0000 7fd8262eb640 1 --2- 172.21.15.3:0/1640737238 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fd80c06be30 0x7fd80c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:04.690 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:04.707+0000 7fd8262eb640 1 --2- 172.21.15.3:0/1640737238 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fd80c06be30 0x7fd80c06e2f0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fd818034000 tx=0x7fd81802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:04.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:04.711+0000 7fd81effd640 1 -- 172.21.15.3:0/1640737238 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fd814059630 con 0x7fd8200cee30 2024-05-09T06:36:04.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:04.899+0000 7fd82854f640 1 -- 172.21.15.3:0/1640737238 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fd8200cf2c0 con 0x7fd8200cee30 2024-05-09T06:36:05.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:05.836+0000 7fd81effd640 1 -- 172.21.15.3:0/1640737238 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v17) ==== 138+0+0 (secure 0 0 0) 0x7fd8200cf2c0 con 0x7fd8200cee30 2024-05-09T06:36:05.817 INFO:teuthology.orchestra.run.smithi003.stderr:pool 'cephfs_metadata' created 2024-05-09T06:36:05.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:05.840+0000 7fd82854f640 1 -- 172.21.15.3:0/1640737238 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fd80c06be30 msgr2=0x7fd80c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:05.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:05.840+0000 7fd82854f640 1 --2- 172.21.15.3:0/1640737238 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fd80c06be30 0x7fd80c06e2f0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fd818034000 tx=0x7fd81802e000 comp rx=0 tx=0).stop 2024-05-09T06:36:05.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:05.840+0000 7fd82854f640 1 -- 172.21.15.3:0/1640737238 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd8200cee30 msgr2=0x7fd8200cddc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:05.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:05.840+0000 7fd82854f640 1 --2- 172.21.15.3:0/1640737238 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd8200cee30 0x7fd8200cddc0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fd81400c430 tx=0x7fd81400c8f0 comp rx=0 tx=0).stop 2024-05-09T06:36:05.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:05.840+0000 7fd82854f640 1 -- 172.21.15.3:0/1640737238 shutdown_connections 2024-05-09T06:36:05.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:05.841+0000 7fd82854f640 1 --2- 172.21.15.3:0/1640737238 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fd80c06be30 0x7fd80c06e2f0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:05.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:05.841+0000 7fd82854f640 1 --2- 172.21.15.3:0/1640737238 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd82015e400 0x7fd8200cce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:05.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:05.841+0000 7fd82854f640 1 --2- 172.21.15.3:0/1640737238 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd8200cee30 0x7fd8200cddc0 secure :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fd81400c430 tx=0x7fd81400c8f0 comp rx=0 tx=0).stop 2024-05-09T06:36:05.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:05.841+0000 7fd82854f640 1 --2- 172.21.15.3:0/1640737238 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd8200b32b0 0x7fd8201f4130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:05.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:05.841+0000 7fd82854f640 1 -- 172.21.15.3:0/1640737238 >> 172.21.15.3:0/1640737238 conn(0x7fd8200675c0 msgr2=0x7fd82015aea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:05.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:05.841+0000 7fd82854f640 1 -- 172.21.15.3:0/1640737238 shutdown_connections 2024-05-09T06:36:05.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:05.842+0000 7fd82854f640 1 -- 172.21.15.3:0/1640737238 wait complete. 2024-05-09T06:36:05.839 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-05-09T06:36:05.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:05.975+0000 7f8c77ed4640 1 Processor -- start 2024-05-09T06:36:05.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:05.975+0000 7f8c77ed4640 1 -- start start 2024-05-09T06:36:05.958 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:05.976+0000 7f8c77ed4640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f8c7013de10 0x7f8c7015eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:05.958 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:05.976+0000 7f8c77ed4640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8c7013e1f0 0x7f8c7015f030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:05.958 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:05.976+0000 7f8c77ed4640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f8c700ce770 0x7f8c700cebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:05.958 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:05.976+0000 7f8c77ed4640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f8c7015f9e0 con 0x7f8c7013de10 2024-05-09T06:36:05.958 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:05.976+0000 7f8c77ed4640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f8c700cf4a0 con 0x7f8c700ce770 2024-05-09T06:36:05.958 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:05.976+0000 7f8c77ed4640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f8c700cf780 con 0x7f8c7013e1f0 2024-05-09T06:36:05.958 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:05.976+0000 7f8c75c70640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f8c7013de10 0x7f8c7015eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:05.958 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:05.976+0000 7f8c75c70640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f8c7013de10 0x7f8c7015eaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45670/0 (socket says 172.21.15.3:45670) 2024-05-09T06:36:05.958 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:05.976+0000 7f8c75c70640 1 -- 172.21.15.3:0/1682136657 learned_addr learned my addr 172.21.15.3:0/1682136657 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:36:05.958 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:05.976+0000 7f8c76471640 1 --2- 172.21.15.3:0/1682136657 >> v2:172.21.15.190:3300/0 conn(0x7f8c700ce770 0x7f8c700cebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:05.958 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:05.976+0000 7f8c7546f640 1 --2- 172.21.15.3:0/1682136657 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8c7013e1f0 0x7f8c7015f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:05.958 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:05.976+0000 7f8c75c70640 1 -- 172.21.15.3:0/1682136657 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8c7013e1f0 msgr2=0x7f8c7015f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:05.958 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:05.976+0000 7f8c75c70640 1 --2- 172.21.15.3:0/1682136657 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8c7013e1f0 0x7f8c7015f030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:05.959 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:05.976+0000 7f8c75c70640 1 -- 172.21.15.3:0/1682136657 >> v2:172.21.15.190:3300/0 conn(0x7f8c700ce770 msgr2=0x7f8c700cebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:05.959 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:05.976+0000 7f8c75c70640 1 --2- 172.21.15.3:0/1682136657 >> v2:172.21.15.190:3300/0 conn(0x7f8c700ce770 0x7f8c700cebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:05.959 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:05.976+0000 7f8c75c70640 1 -- 172.21.15.3:0/1682136657 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c700cfa60 con 0x7f8c7013de10 2024-05-09T06:36:05.959 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:05.976+0000 7f8c76471640 1 --2- 172.21.15.3:0/1682136657 >> v2:172.21.15.190:3300/0 conn(0x7f8c700ce770 0x7f8c700cebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:36:05.959 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:05.977+0000 7f8c75c70640 1 --2- 172.21.15.3:0/1682136657 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8c7013de10 0x7f8c7015eaf0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f8c700ce540 tx=0x7f8c6c02c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60f5d01e508f2e5f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:05.959 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:05.977+0000 7f8c74c6e640 1 -- 172.21.15.3:0/1682136657 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c6c038070 con 0x7f8c7013de10 2024-05-09T06:36:05.959 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:05.977+0000 7f8c74c6e640 1 -- 172.21.15.3:0/1682136657 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c6c031710 con 0x7f8c7013de10 2024-05-09T06:36:05.959 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:05.977+0000 7f8c74c6e640 1 -- 172.21.15.3:0/1682136657 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c6c031ae0 con 0x7f8c7013de10 2024-05-09T06:36:05.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:05.977+0000 7f8c77ed4640 1 -- 172.21.15.3:0/1682136657 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8c7013de10 msgr2=0x7f8c7015eaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:05.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:05.977+0000 7f8c77ed4640 1 --2- 172.21.15.3:0/1682136657 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8c7013de10 0x7f8c7015eaf0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f8c700ce540 tx=0x7f8c6c02c3f0 comp rx=0 tx=0).stop 2024-05-09T06:36:05.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:05.978+0000 7f8c77ed4640 1 -- 172.21.15.3:0/1682136657 shutdown_connections 2024-05-09T06:36:05.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:05.978+0000 7f8c77ed4640 1 --2- 172.21.15.3:0/1682136657 >> v2:172.21.15.190:3300/0 conn(0x7f8c700ce770 0x7f8c700cebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:05.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:05.978+0000 7f8c77ed4640 1 --2- 172.21.15.3:0/1682136657 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8c7013e1f0 0x7f8c7015f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:05.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:05.978+0000 7f8c77ed4640 1 --2- 172.21.15.3:0/1682136657 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8c7013de10 0x7f8c7015eaf0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:05.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:05.978+0000 7f8c77ed4640 1 -- 172.21.15.3:0/1682136657 >> 172.21.15.3:0/1682136657 conn(0x7f8c7006e740 msgr2=0x7f8c7006b250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:05.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:05.978+0000 7f8c77ed4640 1 -- 172.21.15.3:0/1682136657 shutdown_connections 2024-05-09T06:36:05.963 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:05.978+0000 7f8c77ed4640 1 -- 172.21.15.3:0/1682136657 wait complete. 2024-05-09T06:36:05.963 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:05.979+0000 7f8c77ed4640 1 Processor -- start 2024-05-09T06:36:05.963 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:05.979+0000 7f8c77ed4640 1 -- start start 2024-05-09T06:36:05.963 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:05.980+0000 7f8c77ed4640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8c700ce770 0x7f8c70139660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:05.963 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:05.980+0000 7f8c77ed4640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8c7013de10 0x7f8c70159090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:05.963 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:05.980+0000 7f8c77ed4640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8c7013e1f0 0x7f8c701595d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:05.963 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:05.980+0000 7f8c77ed4640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f8c7015b500 con 0x7f8c7013de10 2024-05-09T06:36:05.963 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:05.980+0000 7f8c77ed4640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f8c700cf120 con 0x7f8c7013e1f0 2024-05-09T06:36:05.963 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:05.980+0000 7f8c77ed4640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f8c7015b680 con 0x7f8c700ce770 2024-05-09T06:36:05.963 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:05.980+0000 7f8c7546f640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8c7013de10 0x7f8c70159090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:05.963 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:05.980+0000 7f8c7546f640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8c7013de10 0x7f8c70159090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45678/0 (socket says 172.21.15.3:45678) 2024-05-09T06:36:05.963 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:05.980+0000 7f8c7546f640 1 -- 172.21.15.3:0/862942843 learned_addr learned my addr 172.21.15.3:0/862942843 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:36:05.963 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:05.980+0000 7f8c75c70640 1 --2- 172.21.15.3:0/862942843 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8c700ce770 0x7f8c70139660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:05.963 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:05.980+0000 7f8c76471640 1 --2- 172.21.15.3:0/862942843 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8c7013e1f0 0x7f8c701595d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:05.963 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:05.980+0000 7f8c7546f640 1 -- 172.21.15.3:0/862942843 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8c700ce770 msgr2=0x7f8c70139660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:05.964 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:05.980+0000 7f8c7546f640 1 --2- 172.21.15.3:0/862942843 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8c700ce770 0x7f8c70139660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:05.964 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:05.980+0000 7f8c7546f640 1 -- 172.21.15.3:0/862942843 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8c7013e1f0 msgr2=0x7f8c701595d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:05.964 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:05.980+0000 7f8c7546f640 1 --2- 172.21.15.3:0/862942843 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8c7013e1f0 0x7f8c701595d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:05.964 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:05.980+0000 7f8c7546f640 1 -- 172.21.15.3:0/862942843 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c700b4ca0 con 0x7f8c7013de10 2024-05-09T06:36:05.964 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:05.980+0000 7f8c76471640 1 --2- 172.21.15.3:0/862942843 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8c7013e1f0 0x7f8c701595d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:36:05.964 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:05.980+0000 7f8c75c70640 1 --2- 172.21.15.3:0/862942843 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8c700ce770 0x7f8c70139660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:36:05.964 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:05.980+0000 7f8c7546f640 1 --2- 172.21.15.3:0/862942843 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8c7013de10 0x7f8c70159090 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f8c6000d5f0 tx=0x7f8c6000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:05.964 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:05.980+0000 7f8c667fc640 1 -- 172.21.15.3:0/862942843 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c600125f0 con 0x7f8c7013de10 2024-05-09T06:36:05.964 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:05.980+0000 7f8c667fc640 1 -- 172.21.15.3:0/862942843 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c60019070 con 0x7f8c7013de10 2024-05-09T06:36:05.964 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:05.980+0000 7f8c667fc640 1 -- 172.21.15.3:0/862942843 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c600129d0 con 0x7f8c7013de10 2024-05-09T06:36:05.964 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:05.980+0000 7f8c77ed4640 1 -- 172.21.15.3:0/862942843 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c7013a460 con 0x7f8c7013de10 2024-05-09T06:36:05.964 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:05.980+0000 7f8c77ed4640 1 -- 172.21.15.3:0/862942843 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c70139ba0 con 0x7f8c7013de10 2024-05-09T06:36:05.964 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:05.981+0000 7f8c77ed4640 1 -- 172.21.15.3:0/862942843 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c70069550 con 0x7f8c7013de10 2024-05-09T06:36:05.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:05.982+0000 7f8c667fc640 1 -- 172.21.15.3:0/862942843 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8c60012b70 con 0x7f8c7013de10 2024-05-09T06:36:05.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:05.982+0000 7f8c667fc640 1 --2- 172.21.15.3:0/862942843 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f8c5406be30 0x7f8c5406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:05.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:05.982+0000 7f8c667fc640 1 -- 172.21.15.3:0/862942843 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 5227+0+0 (secure 0 0 0) 0x7f8c6008b000 con 0x7f8c7013de10 2024-05-09T06:36:05.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:05.983+0000 7f8c75c70640 1 --2- 172.21.15.3:0/862942843 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f8c5406be30 0x7f8c5406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:05.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:05.983+0000 7f8c75c70640 1 --2- 172.21.15.3:0/862942843 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f8c5406be30 0x7f8c5406e2f0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f8c6c035000 tx=0x7f8c6c008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:05.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:05.987+0000 7f8c667fc640 1 -- 172.21.15.3:0/862942843 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f8c60059ba0 con 0x7f8c7013de10 2024-05-09T06:36:06.149 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:06.169+0000 7f8c77ed4640 1 -- 172.21.15.3:0/862942843 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f8c7015e290 con 0x7f8c7013de10 2024-05-09T06:36:06.228 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-09T06:36:06.231 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-09T06:36:06.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:06.841+0000 7f8c667fc640 1 -- 172.21.15.3:0/862942843 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v18) ==== 170+0+0 (secure 0 0 0) 0x7f8c600594a0 con 0x7f8c7013de10 2024-05-09T06:36:06.822 INFO:teuthology.orchestra.run.smithi003.stderr:pool 'cephfs_data' created 2024-05-09T06:36:06.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:06.844+0000 7f8c77ed4640 1 -- 172.21.15.3:0/862942843 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f8c5406be30 msgr2=0x7f8c5406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:06.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:06.844+0000 7f8c77ed4640 1 --2- 172.21.15.3:0/862942843 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f8c5406be30 0x7f8c5406e2f0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f8c6c035000 tx=0x7f8c6c008030 comp rx=0 tx=0).stop 2024-05-09T06:36:06.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:06.844+0000 7f8c77ed4640 1 -- 172.21.15.3:0/862942843 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8c7013de10 msgr2=0x7f8c70159090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:06.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:06.844+0000 7f8c77ed4640 1 --2- 172.21.15.3:0/862942843 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8c7013de10 0x7f8c70159090 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f8c6000d5f0 tx=0x7f8c6000dab0 comp rx=0 tx=0).stop 2024-05-09T06:36:06.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:06.845+0000 7f8c77ed4640 1 -- 172.21.15.3:0/862942843 shutdown_connections 2024-05-09T06:36:06.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:06.845+0000 7f8c77ed4640 1 --2- 172.21.15.3:0/862942843 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f8c5406be30 0x7f8c5406e2f0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:06.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:06.845+0000 7f8c77ed4640 1 --2- 172.21.15.3:0/862942843 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8c7013e1f0 0x7f8c701595d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:06.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:06.845+0000 7f8c77ed4640 1 --2- 172.21.15.3:0/862942843 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8c7013de10 0x7f8c70159090 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:06.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:06.845+0000 7f8c77ed4640 1 --2- 172.21.15.3:0/862942843 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8c700ce770 0x7f8c70139660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:06.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:06.845+0000 7f8c77ed4640 1 -- 172.21.15.3:0/862942843 >> 172.21.15.3:0/862942843 conn(0x7f8c7006e740 msgr2=0x7f8c7015a310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:06.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:06.845+0000 7f8c77ed4640 1 -- 172.21.15.3:0/862942843 shutdown_connections 2024-05-09T06:36:06.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:06.845+0000 7f8c77ed4640 1 -- 172.21.15.3:0/862942843 wait complete. 2024-05-09T06:36:06.853 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-05-09T06:36:06.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:06.984+0000 7f8913390640 1 Processor -- start 2024-05-09T06:36:06.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:06.984+0000 7f8913390640 1 -- start start 2024-05-09T06:36:06.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:06.985+0000 7f8913390640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f890c15e5c0 0x7f890c0d2800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:06.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:06.985+0000 7f8913390640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f890c0d1300 0x7f890c0b47e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:06.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:06.985+0000 7f8913390640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f890c15e9a0 con 0x7f890c0d4c40 2024-05-09T06:36:06.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:06.985+0000 7f8913390640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f890c0d31c0 con 0x7f890c0d1300 2024-05-09T06:36:06.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:06.985+0000 7f8913390640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f890c0d50a0 con 0x7f890c15e5c0 2024-05-09T06:36:06.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:06.985+0000 7f891092b640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f890c0d4c40 0x7f890c0d0bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:33054/0 (socket says 172.21.15.3:33054) 2024-05-09T06:36:06.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:06.985+0000 7f891092b640 1 -- 172.21.15.3:0/1678080348 learned_addr learned my addr 172.21.15.3:0/1678080348 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-05-09T06:36:06.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:06.986+0000 7f891112c640 1 --2- 172.21.15.3:0/1678080348 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f890c15e5c0 0x7f890c0d2800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:06.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:06.986+0000 7f891192d640 1 --2- 172.21.15.3:0/1678080348 >> v2:172.21.15.190:3300/0 conn(0x7f890c0d1300 0x7f890c0b47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:06.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:06.986+0000 7f890bfff640 1 -- 172.21.15.3:0/1678080348 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1871183017 0 0) 0x7f890c15e9a0 con 0x7f890c0d4c40 2024-05-09T06:36:06.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:06.986+0000 7f890bfff640 1 -- 172.21.15.3:0/1678080348 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f88f4003520 con 0x7f890c0d4c40 2024-05-09T06:36:06.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:06.986+0000 7f890bfff640 1 -- 172.21.15.3:0/1678080348 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2490711247 0 0) 0x7f88f4003520 con 0x7f890c0d4c40 2024-05-09T06:36:06.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:06.986+0000 7f890bfff640 1 -- 172.21.15.3:0/1678080348 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f890c15e5c0 msgr2=0x7f890c0d2800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:06.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:06.986+0000 7f890bfff640 1 --2- 172.21.15.3:0/1678080348 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f890c15e5c0 0x7f890c0d2800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:06.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:06.986+0000 7f890bfff640 1 -- 172.21.15.3:0/1678080348 >> v2:172.21.15.190:3300/0 conn(0x7f890c0d1300 msgr2=0x7f890c0b47e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:06.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:06.986+0000 7f890bfff640 1 --2- 172.21.15.3:0/1678080348 >> v2:172.21.15.190:3300/0 conn(0x7f890c0d1300 0x7f890c0b47e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:06.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:06.986+0000 7f890bfff640 1 -- 172.21.15.3:0/1678080348 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f890c0d2ee0 con 0x7f890c0d4c40 2024-05-09T06:36:06.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:06.986+0000 7f890bfff640 1 -- 172.21.15.3:0/1678080348 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 563312031 0 0) 0x7f8900001670 con 0x7f890c0d4c40 2024-05-09T06:36:06.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:06.986+0000 7f890bfff640 1 -- 172.21.15.3:0/1678080348 >> v1:172.21.15.3:6789/0 conn(0x7f890c0d4c40 legacy=0x7f890c0d0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:06.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:06.986+0000 7f890bfff640 1 --2- 172.21.15.3:0/1678080348 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f88f40039b0 0x7f88f4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:06.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:06.986+0000 7f890bfff640 1 --2- 172.21.15.3:0/1678080348 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f890c15e5c0 0x7f88f4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:06.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:06.986+0000 7f890bfff640 1 --2- 172.21.15.3:0/1678080348 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f88f4004d40 0x7f88f40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:06.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:06.986+0000 7f890bfff640 1 -- 172.21.15.3:0/1678080348 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f890c0d31c0 con 0x7f88f4004d40 2024-05-09T06:36:06.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:06.986+0000 7f890bfff640 1 -- 172.21.15.3:0/1678080348 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f890c0d50a0 con 0x7f88f40039b0 2024-05-09T06:36:06.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:06.986+0000 7f890bfff640 1 -- 172.21.15.3:0/1678080348 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f88f400b9c0 con 0x7f890c15e5c0 2024-05-09T06:36:06.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:06.986+0000 7f891192d640 1 --2- 172.21.15.3:0/1678080348 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f88f4004d40 0x7f88f40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:06.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:06.986+0000 7f891112c640 1 --2- 172.21.15.3:0/1678080348 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f88f40039b0 0x7f88f4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:06.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:06.986+0000 7f891092b640 1 --2- 172.21.15.3:0/1678080348 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f890c15e5c0 0x7f88f4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:06.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:06.986+0000 7f891192d640 1 -- 172.21.15.3:0/1678080348 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f890c15e5c0 msgr2=0x7f88f4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:06.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:06.986+0000 7f891192d640 1 --2- 172.21.15.3:0/1678080348 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f890c15e5c0 0x7f88f4004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:06.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:06.987+0000 7f891192d640 1 -- 172.21.15.3:0/1678080348 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f88f40039b0 msgr2=0x7f88f4003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:06.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:06.987+0000 7f891192d640 1 --2- 172.21.15.3:0/1678080348 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f88f40039b0 0x7f88f4003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:06.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:06.987+0000 7f891192d640 1 -- 172.21.15.3:0/1678080348 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88f4003520 con 0x7f88f4004d40 2024-05-09T06:36:06.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:06.987+0000 7f891192d640 1 --2- 172.21.15.3:0/1678080348 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f88f4004d40 0x7f88f40051c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f88fc009fc0 tx=0x7f88f4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=751b4f87d39955ae server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:06.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:06.987+0000 7f890bfff640 1 -- 172.21.15.3:0/1678080348 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88fc009c10 con 0x7f88f4004d40 2024-05-09T06:36:06.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:06.987+0000 7f890bfff640 1 -- 172.21.15.3:0/1678080348 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f88fc036070 con 0x7f88f4004d40 2024-05-09T06:36:06.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:06.987+0000 7f890bfff640 1 -- 172.21.15.3:0/1678080348 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88fc02d070 con 0x7f88f4004d40 2024-05-09T06:36:06.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:06.988+0000 7f8913390640 1 -- 172.21.15.3:0/1678080348 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f88f4004d40 msgr2=0x7f88f40051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:06.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:06.988+0000 7f8913390640 1 --2- 172.21.15.3:0/1678080348 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f88f4004d40 0x7f88f40051c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f88fc009fc0 tx=0x7f88f4005830 comp rx=0 tx=0).stop 2024-05-09T06:36:06.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:06.988+0000 7f891192d640 1 -- 172.21.15.3:0/1678080348 reap_dead start 2024-05-09T06:36:06.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:06.988+0000 7f8913390640 1 -- 172.21.15.3:0/1678080348 shutdown_connections 2024-05-09T06:36:06.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:06.988+0000 7f8913390640 1 -- 172.21.15.3:0/1678080348 >> 172.21.15.3:0/1678080348 conn(0x7f890c09d500 msgr2=0x7f890c0d4220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:06.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:06.988+0000 7f8913390640 1 -- 172.21.15.3:0/1678080348 shutdown_connections 2024-05-09T06:36:06.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:06.988+0000 7f8913390640 1 -- 172.21.15.3:0/1678080348 wait complete. 2024-05-09T06:36:06.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:06.989+0000 7f8913390640 1 Processor -- start 2024-05-09T06:36:06.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:06.989+0000 7f8913390640 1 -- start start 2024-05-09T06:36:06.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:06.990+0000 7f8913390640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f890c1612c0 0x7f890c1f4790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:06.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:06.990+0000 7f8913390640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f890c0ccfd0 0x7f890c1f7470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:06.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:06.990+0000 7f8913390640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f890c1616a0 0x7f890c1f79b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:06.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:06.990+0000 7f8913390640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f890c0b42c0 con 0x7f890c0ccfd0 2024-05-09T06:36:06.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:06.990+0000 7f8913390640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f890c0b3fe0 con 0x7f890c1616a0 2024-05-09T06:36:06.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:06.990+0000 7f8913390640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f890c0d2800 con 0x7f890c1612c0 2024-05-09T06:36:06.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:06.990+0000 7f891092b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f890c0ccfd0 0x7f890c1f7470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:06.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:06.990+0000 7f891092b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f890c0ccfd0 0x7f890c1f7470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45694/0 (socket says 172.21.15.3:45694) 2024-05-09T06:36:06.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:06.990+0000 7f891092b640 1 -- 172.21.15.3:0/947510538 learned_addr learned my addr 172.21.15.3:0/947510538 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:36:06.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:06.990+0000 7f891112c640 1 --2- 172.21.15.3:0/947510538 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f890c1612c0 0x7f890c1f4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:06.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:06.990+0000 7f891192d640 1 --2- 172.21.15.3:0/947510538 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f890c1616a0 0x7f890c1f79b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:06.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:06.990+0000 7f891092b640 1 -- 172.21.15.3:0/947510538 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f890c1612c0 msgr2=0x7f890c1f4790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:06.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:06.990+0000 7f891092b640 1 --2- 172.21.15.3:0/947510538 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f890c1612c0 0x7f890c1f4790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:06.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:06.990+0000 7f891092b640 1 -- 172.21.15.3:0/947510538 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f890c1616a0 msgr2=0x7f890c1f79b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:06.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:06.990+0000 7f891092b640 1 --2- 172.21.15.3:0/947510538 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f890c1616a0 0x7f890c1f79b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:06.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:06.990+0000 7f891092b640 1 -- 172.21.15.3:0/947510538 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f890c0cd470 con 0x7f890c0ccfd0 2024-05-09T06:36:06.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:06.991+0000 7f891092b640 1 --2- 172.21.15.3:0/947510538 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f890c0ccfd0 0x7f890c1f7470 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f8900007290 tx=0x7f8900007690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:06.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:06.991+0000 7f8909ffb640 1 -- 172.21.15.3:0/947510538 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f890000f9c0 con 0x7f890c0ccfd0 2024-05-09T06:36:06.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:06.991+0000 7f8909ffb640 1 -- 172.21.15.3:0/947510538 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8900018070 con 0x7f890c0ccfd0 2024-05-09T06:36:06.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:06.991+0000 7f8909ffb640 1 -- 172.21.15.3:0/947510538 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f890000fdc0 con 0x7f890c0ccfd0 2024-05-09T06:36:06.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:06.991+0000 7f8913390640 1 -- 172.21.15.3:0/947510538 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f890c0cd750 con 0x7f890c0ccfd0 2024-05-09T06:36:06.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:06.991+0000 7f8913390640 1 -- 172.21.15.3:0/947510538 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f890c0cdd10 con 0x7f890c0ccfd0 2024-05-09T06:36:06.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:06.992+0000 7f8913390640 1 -- 172.21.15.3:0/947510538 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f890c069550 con 0x7f890c0ccfd0 2024-05-09T06:36:06.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:06.992+0000 7f8909ffb640 1 -- 172.21.15.3:0/947510538 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f890001d460 con 0x7f890c0ccfd0 2024-05-09T06:36:06.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:06.993+0000 7f8909ffb640 1 --2- 172.21.15.3:0/947510538 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f88f806be30 0x7f88f806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:06.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:06.993+0000 7f8909ffb640 1 -- 172.21.15.3:0/947510538 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 5609+0+0 (secure 0 0 0) 0x7f890008af00 con 0x7f890c0ccfd0 2024-05-09T06:36:06.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:06.993+0000 7f891112c640 1 --2- 172.21.15.3:0/947510538 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f88f806be30 0x7f88f806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:06.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:06.993+0000 7f891112c640 1 --2- 172.21.15.3:0/947510538 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f88f806be30 0x7f88f806e2f0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f890c0b99a0 tx=0x7f8904009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:06.978 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:06.997+0000 7f8909ffb640 1 -- 172.21.15.3:0/947510538 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f8900059910 con 0x7f890c0ccfd0 2024-05-09T06:36:07.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:07.219+0000 7f8913390640 1 -- 172.21.15.3:0/947510538 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f890c15da80 con 0x7f890c0ccfd0 2024-05-09T06:36:07.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:07.845+0000 7f8909ffb640 1 -- 172.21.15.3:0/947510538 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) ==== 172+0+0 (secure 0 0 0) 0x7f8900059210 con 0x7f890c0ccfd0 2024-05-09T06:36:07.826 INFO:teuthology.orchestra.run.smithi003.stderr:new fs with metadata pool 2 and data pool 3 2024-05-09T06:36:07.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:07.848+0000 7f8913390640 1 -- 172.21.15.3:0/947510538 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f88f806be30 msgr2=0x7f88f806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:07.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:07.848+0000 7f8913390640 1 --2- 172.21.15.3:0/947510538 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f88f806be30 0x7f88f806e2f0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f890c0b99a0 tx=0x7f8904009190 comp rx=0 tx=0).stop 2024-05-09T06:36:07.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:07.848+0000 7f8913390640 1 -- 172.21.15.3:0/947510538 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f890c0ccfd0 msgr2=0x7f890c1f7470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:07.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:07.848+0000 7f8913390640 1 --2- 172.21.15.3:0/947510538 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f890c0ccfd0 0x7f890c1f7470 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f8900007290 tx=0x7f8900007690 comp rx=0 tx=0).stop 2024-05-09T06:36:07.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:07.848+0000 7f8913390640 1 -- 172.21.15.3:0/947510538 shutdown_connections 2024-05-09T06:36:07.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:07.848+0000 7f8913390640 1 --2- 172.21.15.3:0/947510538 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f88f806be30 0x7f88f806e2f0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:07.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:07.848+0000 7f8913390640 1 --2- 172.21.15.3:0/947510538 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f890c1616a0 0x7f890c1f79b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:07.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:07.848+0000 7f8913390640 1 --2- 172.21.15.3:0/947510538 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f890c0ccfd0 0x7f890c1f7470 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:07.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:07.848+0000 7f8913390640 1 --2- 172.21.15.3:0/947510538 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f890c1612c0 0x7f890c1f4790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:07.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:07.848+0000 7f8913390640 1 -- 172.21.15.3:0/947510538 >> 172.21.15.3:0/947510538 conn(0x7f890c09d500 msgr2=0x7f890c0cf550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:07.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:07.848+0000 7f8913390640 1 -- 172.21.15.3:0/947510538 shutdown_connections 2024-05-09T06:36:07.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:07.848+0000 7f8913390640 1 -- 172.21.15.3:0/947510538 wait complete. 2024-05-09T06:36:07.853 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T06:36:07.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:07.985+0000 7f1dc69a9640 1 Processor -- start 2024-05-09T06:36:07.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:07.985+0000 7f1dc69a9640 1 -- start start 2024-05-09T06:36:07.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:07.986+0000 7f1dc69a9640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f1dc015e480 0x7f1dc00d1dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:07.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:07.986+0000 7f1dc69a9640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f1dc00d2ed0 0x7f1dc01683a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:07.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:07.986+0000 7f1dc69a9640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1dc00d4e50 con 0x7f1dc00d2310 2024-05-09T06:36:07.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:07.986+0000 7f1dc69a9640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f1dc00d0270 con 0x7f1dc00d2ed0 2024-05-09T06:36:07.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:07.986+0000 7f1dc69a9640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f1dc00cf1d0 con 0x7f1dc015e480 2024-05-09T06:36:07.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:07.986+0000 7f1dc4f46640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f1dc00d2ed0 0x7f1dc01683a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:07.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:07.986+0000 7f1dc4f46640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f1dc00d2ed0 0x7f1dc01683a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45710/0 (socket says 172.21.15.3:45710) 2024-05-09T06:36:07.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:07.986+0000 7f1dc4f46640 1 -- 172.21.15.3:0/3352949290 learned_addr learned my addr 172.21.15.3:0/3352949290 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:36:07.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:07.986+0000 7f1dbffff640 1 --2- 172.21.15.3:0/3352949290 >> v2:172.21.15.190:3300/0 conn(0x7f1dc015e480 0x7f1dc00d1dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:07.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:07.986+0000 7f1dc4f46640 1 -- 172.21.15.3:0/3352949290 >> v2:172.21.15.190:3300/0 conn(0x7f1dc015e480 msgr2=0x7f1dc00d1dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:07.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:07.986+0000 7f1dc4f46640 1 --2- 172.21.15.3:0/3352949290 >> v2:172.21.15.190:3300/0 conn(0x7f1dc015e480 0x7f1dc00d1dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:07.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:07.986+0000 7f1dc4f46640 1 -- 172.21.15.3:0/3352949290 >> v1:172.21.15.190:6789/0 conn(0x7f1dc00d2310 legacy=0x7f1dc00d27c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:07.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:07.986+0000 7f1dc4f46640 1 -- 172.21.15.3:0/3352949290 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1dc00cf4b0 con 0x7f1dc00d2ed0 2024-05-09T06:36:07.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:07.987+0000 7f1dc4f46640 1 --2- 172.21.15.3:0/3352949290 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1dc00d2ed0 0x7f1dc01683a0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f1dc009da00 tx=0x7f1db802a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a066c2a5f351a60f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:07.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:07.987+0000 7f1dbeffd640 1 -- 172.21.15.3:0/3352949290 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1db8036070 con 0x7f1dc00d2ed0 2024-05-09T06:36:07.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:07.987+0000 7f1dbeffd640 1 -- 172.21.15.3:0/3352949290 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1db8031600 con 0x7f1dc00d2ed0 2024-05-09T06:36:07.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:07.987+0000 7f1dbeffd640 1 -- 172.21.15.3:0/3352949290 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1db80319d0 con 0x7f1dc00d2ed0 2024-05-09T06:36:07.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:07.988+0000 7f1dc69a9640 1 -- 172.21.15.3:0/3352949290 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1dc00d2ed0 msgr2=0x7f1dc01683a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:07.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:07.988+0000 7f1dc69a9640 1 --2- 172.21.15.3:0/3352949290 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1dc00d2ed0 0x7f1dc01683a0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f1dc009da00 tx=0x7f1db802a730 comp rx=0 tx=0).stop 2024-05-09T06:36:07.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:07.988+0000 7f1dc69a9640 1 -- 172.21.15.3:0/3352949290 shutdown_connections 2024-05-09T06:36:07.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:07.988+0000 7f1dc69a9640 1 --2- 172.21.15.3:0/3352949290 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1dc00d2ed0 0x7f1dc01683a0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:07.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:07.988+0000 7f1dc69a9640 1 --2- 172.21.15.3:0/3352949290 >> v2:172.21.15.190:3300/0 conn(0x7f1dc015e480 0x7f1dc00d1dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:07.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:07.988+0000 7f1dc69a9640 1 -- 172.21.15.3:0/3352949290 >> 172.21.15.3:0/3352949290 conn(0x7f1dc009d4d0 msgr2=0x7f1dc00d4200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:07.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:07.988+0000 7f1dc69a9640 1 -- 172.21.15.3:0/3352949290 shutdown_connections 2024-05-09T06:36:07.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:07.988+0000 7f1dc69a9640 1 -- 172.21.15.3:0/3352949290 wait complete. 2024-05-09T06:36:07.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:07.989+0000 7f1dc69a9640 1 Processor -- start 2024-05-09T06:36:07.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:07.989+0000 7f1dc69a9640 1 -- start start 2024-05-09T06:36:07.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:07.990+0000 7f1dc69a9640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1dc00d2310 0x7f1dc00c7f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:07.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:07.990+0000 7f1dc69a9640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1dc00d2ed0 0x7f1dc014e230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:07.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:07.990+0000 7f1dc69a9640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1dc015e480 0x7f1dc014e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:07.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:07.990+0000 7f1dc69a9640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f1dc00cfd50 con 0x7f1dc00d2310 2024-05-09T06:36:07.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:07.990+0000 7f1dc69a9640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f1dc00cfa70 con 0x7f1dc00d2ed0 2024-05-09T06:36:07.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:07.990+0000 7f1dc69a9640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f1dc00b5ab0 con 0x7f1dc015e480 2024-05-09T06:36:07.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:07.990+0000 7f1dbffff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1dc00d2310 0x7f1dc00c7f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:07.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:07.990+0000 7f1dbffff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1dc00d2310 0x7f1dc00c7f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45724/0 (socket says 172.21.15.3:45724) 2024-05-09T06:36:07.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:07.990+0000 7f1dbffff640 1 -- 172.21.15.3:0/1088331914 learned_addr learned my addr 172.21.15.3:0/1088331914 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:36:07.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:07.990+0000 7f1dbf7fe640 1 --2- 172.21.15.3:0/1088331914 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1dc00d2ed0 0x7f1dc014e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:07.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:07.990+0000 7f1dc4f46640 1 --2- 172.21.15.3:0/1088331914 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1dc015e480 0x7f1dc014e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:07.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:07.991+0000 7f1dbffff640 1 -- 172.21.15.3:0/1088331914 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1dc015e480 msgr2=0x7f1dc014e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:07.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:07.991+0000 7f1dbffff640 1 --2- 172.21.15.3:0/1088331914 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1dc015e480 0x7f1dc014e770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:07.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:07.991+0000 7f1dbffff640 1 -- 172.21.15.3:0/1088331914 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1dc00d2ed0 msgr2=0x7f1dc014e230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:07.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:07.991+0000 7f1dbffff640 1 --2- 172.21.15.3:0/1088331914 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1dc00d2ed0 0x7f1dc014e230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:07.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:07.991+0000 7f1dbffff640 1 -- 172.21.15.3:0/1088331914 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1dc01f5300 con 0x7f1dc00d2310 2024-05-09T06:36:07.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:07.991+0000 7f1dbffff640 1 --2- 172.21.15.3:0/1088331914 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1dc00d2310 0x7f1dc00c7f70 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f1db000e9b0 tx=0x7f1db000ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:07.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:07.991+0000 7f1dbcff9640 1 -- 172.21.15.3:0/1088331914 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1db0017070 con 0x7f1dc00d2310 2024-05-09T06:36:07.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:07.991+0000 7f1dbcff9640 1 -- 172.21.15.3:0/1088331914 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1db000c9a0 con 0x7f1dc00d2310 2024-05-09T06:36:07.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:07.991+0000 7f1dbcff9640 1 -- 172.21.15.3:0/1088331914 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1db000cd90 con 0x7f1dc00d2310 2024-05-09T06:36:07.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:07.991+0000 7f1dc69a9640 1 -- 172.21.15.3:0/1088331914 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1dc01f58e0 con 0x7f1dc00d2310 2024-05-09T06:36:07.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:07.991+0000 7f1dc69a9640 1 -- 172.21.15.3:0/1088331914 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1dc01f55e0 con 0x7f1dc00d2310 2024-05-09T06:36:07.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:07.992+0000 7f1dc69a9640 1 -- 172.21.15.3:0/1088331914 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1dc004f660 con 0x7f1dc00d2310 2024-05-09T06:36:07.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:07.993+0000 7f1dbcff9640 1 -- 172.21.15.3:0/1088331914 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1db001a070 con 0x7f1dc00d2310 2024-05-09T06:36:07.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:07.993+0000 7f1dbcff9640 1 --2- 172.21.15.3:0/1088331914 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f1dac06be30 0x7f1dac06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:07.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:07.993+0000 7f1dbcff9640 1 -- 172.21.15.3:0/1088331914 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 5709+0+0 (secure 0 0 0) 0x7f1db008ae30 con 0x7f1dc00d2310 2024-05-09T06:36:07.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:07.993+0000 7f1dbf7fe640 1 --2- 172.21.15.3:0/1088331914 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f1dac06be30 0x7f1dac06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:07.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:07.994+0000 7f1dbf7fe640 1 --2- 172.21.15.3:0/1088331914 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f1dac06be30 0x7f1dac06e2f0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f1dc015de90 tx=0x7f1db4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:07.978 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:07.997+0000 7f1dbcff9640 1 -- 172.21.15.3:0/1088331914 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f1db00591a0 con 0x7f1dc00d2310 2024-05-09T06:36:08.163 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:08.182+0000 7f1dc69a9640 1 -- 172.21.15.3:0/1088331914 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1dc00d1f40 con 0x7f1dc00d2310 2024-05-09T06:36:08.164 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:08.183+0000 7f1dbcff9640 1 -- 172.21.15.3:0/1088331914 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) ==== 74+0+16534 (secure 0 0 0) 0x7f1db005d4a0 con 0x7f1dc00d2310 2024-05-09T06:36:08.164 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-09T06:36:08.164 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":19,"fsid":"16ed737d-2665-431a-8ac6-ef242be3975a","created":"2024-05-09T06:35:42.541368+0000","modified":"2024-05-09T06:36:07.220747+0000","last_up_change":"2024-05-09T06:35:53.778274+0000","last_in_change":"2024-05-09T06:35:44.780586+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T06:35:54.819999+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T06:36:04.931314+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T06:36:06.200433+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fae3a8db-4826-4f3c-b33a-12c09bb5df6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":1897849827},{"type":"v1","addr":"172.21.15.3:6817","nonce":1897849827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":1897849827},{"type":"v1","addr":"172.21.15.3:6819","nonce":1897849827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":1897849827},{"type":"v1","addr":"172.21.15.3:6823","nonce":1897849827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":1897849827},{"type":"v1","addr":"172.21.15.3:6821","nonce":1897849827}]},"public_addr":"172.21.15.3:6817/1897849827","cluster_addr":"172.21.15.3:6819/1897849827","heartbeat_back_addr":"172.21.15.3:6823/1897849827","heartbeat_front_addr":"172.21.15.3:6821/1897849827","state":["exists","up"]},{"osd":1,"uuid":"646e882d-4ce4-4720-8c84-f71df563a35e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":926805498},{"type":"v1","addr":"172.21.15.3:6801","nonce":926805498}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":926805498},{"type":"v1","addr":"172.21.15.3:6803","nonce":926805498}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":926805498},{"type":"v1","addr":"172.21.15.3:6807","nonce":926805498}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":926805498},{"type":"v1","addr":"172.21.15.3:6805","nonce":926805498}]},"public_addr":"172.21.15.3:6801/926805498","cluster_addr":"172.21.15.3:6803/926805498","heartbeat_back_addr":"172.21.15.3:6807/926805498","heartbeat_front_addr":"172.21.15.3:6805/926805498","state":["exists","up"]},{"osd":2,"uuid":"1abbadb5-a570-4359-9643-acff3ede35a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":801295451},{"type":"v1","addr":"172.21.15.3:6809","nonce":801295451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":801295451},{"type":"v1","addr":"172.21.15.3:6811","nonce":801295451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":801295451},{"type":"v1","addr":"172.21.15.3:6815","nonce":801295451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":801295451},{"type":"v1","addr":"172.21.15.3:6813","nonce":801295451}]},"public_addr":"172.21.15.3:6809/801295451","cluster_addr":"172.21.15.3:6811/801295451","heartbeat_back_addr":"172.21.15.3:6815/801295451","heartbeat_front_addr":"172.21.15.3:6813/801295451","state":["exists","up"]},{"osd":3,"uuid":"62ac4dc1-2371-4dd2-acd3-e2dbb11aedf5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":1903389857},{"type":"v1","addr":"172.21.15.3:6825","nonce":1903389857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":1903389857},{"type":"v1","addr":"172.21.15.3:6827","nonce":1903389857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":1903389857},{"type":"v1","addr":"172.21.15.3:6831","nonce":1903389857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":1903389857},{"type":"v1","addr":"172.21.15.3:6829","nonce":1903389857}]},"public_addr":"172.21.15.3:6825/1903389857","cluster_addr":"172.21.15.3:6827/1903389857","heartbeat_back_addr":"172.21.15.3:6831/1903389857","heartbeat_front_addr":"172.21.15.3:6829/1903389857","state":["exists","up"]},{"osd":4,"uuid":"ae808398-8326-4d7c-9319-a2a0eafffb59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":2143624576},{"type":"v1","addr":"172.21.15.190:6801","nonce":2143624576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":2143624576},{"type":"v1","addr":"172.21.15.190:6803","nonce":2143624576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":2143624576},{"type":"v1","addr":"172.21.15.190:6807","nonce":2143624576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":2143624576},{"type":"v1","addr":"172.21.15.190:6805","nonce":2143624576}]},"public_addr":"172.21.15.190:6801/2143624576","cluster_addr":"172.21.15.190:6803/2143624576","heartbeat_back_addr":"172.21.15.190:6807/2143624576","heartbeat_front_addr":"172.21.15.190:6805/2143624576","state":["exists","up"]},{"osd":5,"uuid":"918adf50-c440-48ac-9a8d-4837562d7275","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":1766923126},{"type":"v1","addr":"172.21.15.190:6814","nonce":1766923126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":1766923126},{"type":"v1","addr":"172.21.15.190:6818","nonce":1766923126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":1766923126},{"type":"v1","addr":"172.21.15.190:6823","nonce":1766923126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":1766923126},{"type":"v1","addr":"172.21.15.190:6821","nonce":1766923126}]},"public_addr":"172.21.15.190:6814/1766923126","cluster_addr":"172.21.15.190:6818/1766923126","heartbeat_back_addr":"172.21.15.190:6823/1766923126","heartbeat_front_addr":"172.21.15.190:6821/1766923126","state":["exists","up"]},{"osd":6,"uuid":"ac2882bc-2a7c-463c-8f6c-76ff49828adc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":342414248},{"type":"v1","addr":"172.21.15.190:6825","nonce":342414248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":342414248},{"type":"v1","addr":"172.21.15.190:6827","nonce":342414248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":342414248},{"type":"v1","addr":"172.21.15.190:6831","nonce":342414248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":342414248},{"type":"v1","addr":"172.21.15.190:6829","nonce":342414248}]},"public_addr":"172.21.15.190:6825/342414248","cluster_addr":"172.21.15.190:6827/342414248","heartbeat_back_addr":"172.21.15.190:6831/342414248","heartbeat_front_addr":"172.21.15.190:6829/342414248","state":["exists","up"]},{"osd":7,"uuid":"dd4e767a-4730-4c70-8bd5-3f088140a5a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":537849891},{"type":"v1","addr":"172.21.15.190:6809","nonce":537849891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":537849891},{"type":"v1","addr":"172.21.15.190:6811","nonce":537849891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6817","nonce":537849891},{"type":"v1","addr":"172.21.15.190:6819","nonce":537849891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6813","nonce":537849891},{"type":"v1","addr":"172.21.15.190:6815","nonce":537849891}]},"public_addr":"172.21.15.190:6809/537849891","cluster_addr":"172.21.15.190:6811/537849891","heartbeat_back_addr":"172.21.15.190:6819/537849891","heartbeat_front_addr":"172.21.15.190:6815/537849891","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.889784+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.974002+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.973549+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.955396+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.928471+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.902418+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.958696+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.926474+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T06:36:08.166 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:08.185+0000 7f1dc69a9640 1 -- 172.21.15.3:0/1088331914 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f1dac06be30 msgr2=0x7f1dac06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:08.166 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:08.185+0000 7f1dc69a9640 1 --2- 172.21.15.3:0/1088331914 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f1dac06be30 0x7f1dac06e2f0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f1dc015de90 tx=0x7f1db4009040 comp rx=0 tx=0).stop 2024-05-09T06:36:08.166 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:08.185+0000 7f1dc69a9640 1 -- 172.21.15.3:0/1088331914 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1dc00d2310 msgr2=0x7f1dc00c7f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:08.166 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:08.185+0000 7f1dc69a9640 1 --2- 172.21.15.3:0/1088331914 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1dc00d2310 0x7f1dc00c7f70 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f1db000e9b0 tx=0x7f1db000ee70 comp rx=0 tx=0).stop 2024-05-09T06:36:08.166 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:08.185+0000 7f1dc69a9640 1 -- 172.21.15.3:0/1088331914 shutdown_connections 2024-05-09T06:36:08.166 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:08.185+0000 7f1dc69a9640 1 --2- 172.21.15.3:0/1088331914 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f1dac06be30 0x7f1dac06e2f0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:08.166 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:08.185+0000 7f1dc69a9640 1 --2- 172.21.15.3:0/1088331914 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1dc015e480 0x7f1dc014e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:08.167 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:08.185+0000 7f1dc69a9640 1 --2- 172.21.15.3:0/1088331914 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1dc00d2ed0 0x7f1dc014e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:08.167 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:08.185+0000 7f1dc69a9640 1 --2- 172.21.15.3:0/1088331914 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1dc00d2310 0x7f1dc00c7f70 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:08.167 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:08.185+0000 7f1dc69a9640 1 -- 172.21.15.3:0/1088331914 >> 172.21.15.3:0/1088331914 conn(0x7f1dc009d4d0 msgr2=0x7f1dc00b53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:08.167 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:08.185+0000 7f1dc69a9640 1 -- 172.21.15.3:0/1088331914 shutdown_connections 2024-05-09T06:36:08.167 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:08.185+0000 7f1dc69a9640 1 -- 172.21.15.3:0/1088331914 wait complete. 2024-05-09T06:36:08.178 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T06:36:08.281 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:08.300+0000 7f78ca1e2640 1 Processor -- start 2024-05-09T06:36:08.282 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:08.300+0000 7f78ca1e2640 1 -- start start 2024-05-09T06:36:08.285 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:08.301+0000 7f78ca1e2640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f78c40d27c0 0x7f78c40d2c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:08.285 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:08.301+0000 7f78ca1e2640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f78c40d4e50 con 0x7f78c4166390 2024-05-09T06:36:08.285 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:08.301+0000 7f78ca1e2640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f78c40d0270 con 0x7f78c415e480 2024-05-09T06:36:08.285 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:08.301+0000 7f78ca1e2640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f78c40d3160 con 0x7f78c40d27c0 2024-05-09T06:36:08.285 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:08.301+0000 7f78c3fff640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f78c4166390 0x7f78c4168780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:33058/0 (socket says 172.21.15.3:33058) 2024-05-09T06:36:08.286 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:08.301+0000 7f78c3fff640 1 -- 172.21.15.3:0/2158612026 learned_addr learned my addr 172.21.15.3:0/2158612026 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-05-09T06:36:08.286 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:08.302+0000 7f78c2ffd640 1 --2- 172.21.15.3:0/2158612026 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f78c40d27c0 0x7f78c40d2c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:08.286 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:08.302+0000 7f78c27fc640 1 -- 172.21.15.3:0/2158612026 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3751145991 0 0) 0x7f78bc001490 con 0x7f78c4166390 2024-05-09T06:36:08.286 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:08.302+0000 7f78c27fc640 1 -- 172.21.15.3:0/2158612026 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f78ac003520 con 0x7f78c4166390 2024-05-09T06:36:08.286 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:08.302+0000 7f78c27fc640 1 -- 172.21.15.3:0/2158612026 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1480207681 0 0) 0x7f78ac003520 con 0x7f78c4166390 2024-05-09T06:36:08.286 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:08.302+0000 7f78c27fc640 1 -- 172.21.15.3:0/2158612026 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f78c40d27c0 msgr2=0x7f78c40d2c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:08.286 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:08.302+0000 7f78c27fc640 1 --2- 172.21.15.3:0/2158612026 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f78c40d27c0 0x7f78c40d2c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:08.286 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:08.302+0000 7f78c27fc640 1 -- 172.21.15.3:0/2158612026 >> v1:172.21.15.190:6789/0 conn(0x7f78c415e480 legacy=0x7f78c40d1dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:08.286 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:08.302+0000 7f78c27fc640 1 -- 172.21.15.3:0/2158612026 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78c40cf1d0 con 0x7f78c4166390 2024-05-09T06:36:08.286 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:08.302+0000 7f78c27fc640 1 -- 172.21.15.3:0/2158612026 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 563312031 0 0) 0x7f78bc001bc0 con 0x7f78c4166390 2024-05-09T06:36:08.286 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:08.302+0000 7f78c27fc640 1 -- 172.21.15.3:0/2158612026 >> v1:172.21.15.3:6789/0 conn(0x7f78c4166390 legacy=0x7f78c4168780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:08.286 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:08.302+0000 7f78c27fc640 1 --2- 172.21.15.3:0/2158612026 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f78c40d27c0 0x7f78ac0039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:08.287 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:08.302+0000 7f78c27fc640 1 --2- 172.21.15.3:0/2158612026 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f78ac003f20 0x7f78ac004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:08.287 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:08.302+0000 7f78c27fc640 1 --2- 172.21.15.3:0/2158612026 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f78ac004d40 0x7f78ac0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:08.287 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:08.302+0000 7f78c27fc640 1 -- 172.21.15.3:0/2158612026 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f78ac00b9f0 con 0x7f78ac003f20 2024-05-09T06:36:08.287 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:08.302+0000 7f78c27fc640 1 -- 172.21.15.3:0/2158612026 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f78ac00bba0 con 0x7f78ac004d40 2024-05-09T06:36:08.287 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:08.302+0000 7f78c27fc640 1 -- 172.21.15.3:0/2158612026 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f78ac00bda0 con 0x7f78c40d27c0 2024-05-09T06:36:08.287 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:08.302+0000 7f78c2ffd640 1 --2- 172.21.15.3:0/2158612026 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f78ac003f20 0x7f78ac004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:08.287 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:08.302+0000 7f78c37fe640 1 --2- 172.21.15.3:0/2158612026 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f78c40d27c0 0x7f78ac0039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:08.287 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:08.302+0000 7f78c3fff640 1 --2- 172.21.15.3:0/2158612026 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f78ac004d40 0x7f78ac0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:08.287 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:08.302+0000 7f78c2ffd640 1 -- 172.21.15.3:0/2158612026 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f78c40d27c0 msgr2=0x7f78ac0039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:08.287 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:08.302+0000 7f78c2ffd640 1 --2- 172.21.15.3:0/2158612026 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f78c40d27c0 0x7f78ac0039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:08.287 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:08.302+0000 7f78c2ffd640 1 -- 172.21.15.3:0/2158612026 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f78ac004d40 msgr2=0x7f78ac0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:08.287 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:08.302+0000 7f78c2ffd640 1 --2- 172.21.15.3:0/2158612026 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f78ac004d40 0x7f78ac0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:08.288 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:08.302+0000 7f78c2ffd640 1 -- 172.21.15.3:0/2158612026 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78ac003520 con 0x7f78ac003f20 2024-05-09T06:36:08.288 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:08.303+0000 7f78c2ffd640 1 --2- 172.21.15.3:0/2158612026 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f78ac003f20 0x7f78ac004800 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f78b8009a90 tx=0x7f78ac005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2863574a0262c61e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:08.288 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:08.303+0000 7f78c27fc640 1 -- 172.21.15.3:0/2158612026 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78b8036070 con 0x7f78ac003f20 2024-05-09T06:36:08.288 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:08.303+0000 7f78c27fc640 1 -- 172.21.15.3:0/2158612026 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f78b8031400 con 0x7f78ac003f20 2024-05-09T06:36:08.288 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:08.303+0000 7f78c27fc640 1 -- 172.21.15.3:0/2158612026 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78b8009690 con 0x7f78ac003f20 2024-05-09T06:36:08.288 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:08.304+0000 7f78ca1e2640 1 -- 172.21.15.3:0/2158612026 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f78ac003f20 msgr2=0x7f78ac004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:08.288 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:08.304+0000 7f78ca1e2640 1 --2- 172.21.15.3:0/2158612026 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f78ac003f20 0x7f78ac004800 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f78b8009a90 tx=0x7f78ac005830 comp rx=0 tx=0).stop 2024-05-09T06:36:08.288 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:08.304+0000 7f78c3fff640 1 -- 172.21.15.3:0/2158612026 reap_dead start 2024-05-09T06:36:08.288 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:08.304+0000 7f78ca1e2640 1 -- 172.21.15.3:0/2158612026 shutdown_connections 2024-05-09T06:36:08.288 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:08.304+0000 7f78ca1e2640 1 -- 172.21.15.3:0/2158612026 >> 172.21.15.3:0/2158612026 conn(0x7f78c409d4d0 msgr2=0x7f78c40d4200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:08.289 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:08.304+0000 7f78ca1e2640 1 -- 172.21.15.3:0/2158612026 shutdown_connections 2024-05-09T06:36:08.289 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:08.304+0000 7f78ca1e2640 1 -- 172.21.15.3:0/2158612026 wait complete. 2024-05-09T06:36:08.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:08.305+0000 7f78ca1e2640 1 Processor -- start 2024-05-09T06:36:08.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:08.305+0000 7f78ca1e2640 1 -- start start 2024-05-09T06:36:08.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:08.306+0000 7f78ca1e2640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f78ac003f20 0x7f78c40b6700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:08.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:08.306+0000 7f78ca1e2640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f78c4157660 0x7f78c40cd4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:08.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:08.306+0000 7f78ca1e2640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f78c415f770 0x7f78c41f7580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:08.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:08.306+0000 7f78ca1e2640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f78c40cfa70 con 0x7f78ac003f20 2024-05-09T06:36:08.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:08.306+0000 7f78ca1e2640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f78c40cf790 con 0x7f78c415f770 2024-05-09T06:36:08.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:08.306+0000 7f78ca1e2640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f78c40cfd50 con 0x7f78c4157660 2024-05-09T06:36:08.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:08.306+0000 7f78c37fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f78ac003f20 0x7f78c40b6700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:08.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:08.306+0000 7f78c37fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f78ac003f20 0x7f78c40b6700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45736/0 (socket says 172.21.15.3:45736) 2024-05-09T06:36:08.293 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:08.306+0000 7f78c37fe640 1 -- 172.21.15.3:0/2336773380 learned_addr learned my addr 172.21.15.3:0/2336773380 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:36:08.293 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:08.306+0000 7f78c2ffd640 1 --2- 172.21.15.3:0/2336773380 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f78c4157660 0x7f78c40cd4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:08.293 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:08.306+0000 7f78c37fe640 1 -- 172.21.15.3:0/2336773380 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f78c4157660 msgr2=0x7f78c40cd4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:08.293 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:08.306+0000 7f78c37fe640 1 --2- 172.21.15.3:0/2336773380 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f78c4157660 0x7f78c40cd4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:08.293 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:08.306+0000 7f78c37fe640 1 -- 172.21.15.3:0/2336773380 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f78c415f770 msgr2=0x7f78c41f7580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:08.293 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:08.306+0000 7f78c37fe640 1 --2- 172.21.15.3:0/2336773380 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f78c415f770 0x7f78c41f7580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:08.293 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:08.306+0000 7f78c37fe640 1 -- 172.21.15.3:0/2336773380 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78c4157ac0 con 0x7f78ac003f20 2024-05-09T06:36:08.293 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:08.307+0000 7f78c37fe640 1 --2- 172.21.15.3:0/2336773380 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f78ac003f20 0x7f78c40b6700 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f78b400f7f0 tx=0x7f78b400fcb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:08.293 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:08.307+0000 7f78abfff640 1 -- 172.21.15.3:0/2336773380 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78b40052e0 con 0x7f78ac003f20 2024-05-09T06:36:08.293 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:08.307+0000 7f78abfff640 1 -- 172.21.15.3:0/2336773380 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f78b4018070 con 0x7f78ac003f20 2024-05-09T06:36:08.293 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:08.307+0000 7f78ca1e2640 1 -- 172.21.15.3:0/2336773380 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78c40cdc50 con 0x7f78ac003f20 2024-05-09T06:36:08.293 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:08.307+0000 7f78abfff640 1 -- 172.21.15.3:0/2336773380 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78b400d8f0 con 0x7f78ac003f20 2024-05-09T06:36:08.293 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:08.307+0000 7f78ca1e2640 1 -- 172.21.15.3:0/2336773380 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78c40b4940 con 0x7f78ac003f20 2024-05-09T06:36:08.293 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:08.308+0000 7f78ca1e2640 1 -- 172.21.15.3:0/2336773380 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78c404f660 con 0x7f78ac003f20 2024-05-09T06:36:08.294 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:08.308+0000 7f78abfff640 1 -- 172.21.15.3:0/2336773380 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f78b400da90 con 0x7f78ac003f20 2024-05-09T06:36:08.294 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:08.309+0000 7f78abfff640 1 --2- 172.21.15.3:0/2336773380 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f78b006be60 0x7f78b006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:08.294 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:08.309+0000 7f78abfff640 1 -- 172.21.15.3:0/2336773380 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 5709+0+0 (secure 0 0 0) 0x7f78b408ad90 con 0x7f78ac003f20 2024-05-09T06:36:08.294 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:08.309+0000 7f78c2ffd640 1 --2- 172.21.15.3:0/2336773380 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f78b006be60 0x7f78b006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:08.294 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:08.310+0000 7f78c2ffd640 1 --2- 172.21.15.3:0/2336773380 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f78b006be60 0x7f78b006e320 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f78b8033000 tx=0x7f78b8008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:08.294 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:08.313+0000 7f78abfff640 1 -- 172.21.15.3:0/2336773380 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f78b401a310 con 0x7f78ac003f20 2024-05-09T06:36:08.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:08.502+0000 7f78ca1e2640 1 -- 172.21.15.3:0/2336773380 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f78c4157cd0 con 0x7f78ac003f20 2024-05-09T06:36:08.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:08.503+0000 7f78abfff640 1 -- 172.21.15.3:0/2336773380 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) ==== 74+0+16534 (secure 0 0 0) 0x7f78b4059740 con 0x7f78ac003f20 2024-05-09T06:36:08.484 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-09T06:36:08.484 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":19,"fsid":"16ed737d-2665-431a-8ac6-ef242be3975a","created":"2024-05-09T06:35:42.541368+0000","modified":"2024-05-09T06:36:07.220747+0000","last_up_change":"2024-05-09T06:35:53.778274+0000","last_in_change":"2024-05-09T06:35:44.780586+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T06:35:54.819999+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T06:36:04.931314+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T06:36:06.200433+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fae3a8db-4826-4f3c-b33a-12c09bb5df6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":1897849827},{"type":"v1","addr":"172.21.15.3:6817","nonce":1897849827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":1897849827},{"type":"v1","addr":"172.21.15.3:6819","nonce":1897849827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":1897849827},{"type":"v1","addr":"172.21.15.3:6823","nonce":1897849827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":1897849827},{"type":"v1","addr":"172.21.15.3:6821","nonce":1897849827}]},"public_addr":"172.21.15.3:6817/1897849827","cluster_addr":"172.21.15.3:6819/1897849827","heartbeat_back_addr":"172.21.15.3:6823/1897849827","heartbeat_front_addr":"172.21.15.3:6821/1897849827","state":["exists","up"]},{"osd":1,"uuid":"646e882d-4ce4-4720-8c84-f71df563a35e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":926805498},{"type":"v1","addr":"172.21.15.3:6801","nonce":926805498}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":926805498},{"type":"v1","addr":"172.21.15.3:6803","nonce":926805498}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":926805498},{"type":"v1","addr":"172.21.15.3:6807","nonce":926805498}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":926805498},{"type":"v1","addr":"172.21.15.3:6805","nonce":926805498}]},"public_addr":"172.21.15.3:6801/926805498","cluster_addr":"172.21.15.3:6803/926805498","heartbeat_back_addr":"172.21.15.3:6807/926805498","heartbeat_front_addr":"172.21.15.3:6805/926805498","state":["exists","up"]},{"osd":2,"uuid":"1abbadb5-a570-4359-9643-acff3ede35a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":801295451},{"type":"v1","addr":"172.21.15.3:6809","nonce":801295451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":801295451},{"type":"v1","addr":"172.21.15.3:6811","nonce":801295451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":801295451},{"type":"v1","addr":"172.21.15.3:6815","nonce":801295451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":801295451},{"type":"v1","addr":"172.21.15.3:6813","nonce":801295451}]},"public_addr":"172.21.15.3:6809/801295451","cluster_addr":"172.21.15.3:6811/801295451","heartbeat_back_addr":"172.21.15.3:6815/801295451","heartbeat_front_addr":"172.21.15.3:6813/801295451","state":["exists","up"]},{"osd":3,"uuid":"62ac4dc1-2371-4dd2-acd3-e2dbb11aedf5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":1903389857},{"type":"v1","addr":"172.21.15.3:6825","nonce":1903389857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":1903389857},{"type":"v1","addr":"172.21.15.3:6827","nonce":1903389857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":1903389857},{"type":"v1","addr":"172.21.15.3:6831","nonce":1903389857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":1903389857},{"type":"v1","addr":"172.21.15.3:6829","nonce":1903389857}]},"public_addr":"172.21.15.3:6825/1903389857","cluster_addr":"172.21.15.3:6827/1903389857","heartbeat_back_addr":"172.21.15.3:6831/1903389857","heartbeat_front_addr":"172.21.15.3:6829/1903389857","state":["exists","up"]},{"osd":4,"uuid":"ae808398-8326-4d7c-9319-a2a0eafffb59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":2143624576},{"type":"v1","addr":"172.21.15.190:6801","nonce":2143624576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":2143624576},{"type":"v1","addr":"172.21.15.190:6803","nonce":2143624576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":2143624576},{"type":"v1","addr":"172.21.15.190:6807","nonce":2143624576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":2143624576},{"type":"v1","addr":"172.21.15.190:6805","nonce":2143624576}]},"public_addr":"172.21.15.190:6801/2143624576","cluster_addr":"172.21.15.190:6803/2143624576","heartbeat_back_addr":"172.21.15.190:6807/2143624576","heartbeat_front_addr":"172.21.15.190:6805/2143624576","state":["exists","up"]},{"osd":5,"uuid":"918adf50-c440-48ac-9a8d-4837562d7275","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":1766923126},{"type":"v1","addr":"172.21.15.190:6814","nonce":1766923126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":1766923126},{"type":"v1","addr":"172.21.15.190:6818","nonce":1766923126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":1766923126},{"type":"v1","addr":"172.21.15.190:6823","nonce":1766923126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":1766923126},{"type":"v1","addr":"172.21.15.190:6821","nonce":1766923126}]},"public_addr":"172.21.15.190:6814/1766923126","cluster_addr":"172.21.15.190:6818/1766923126","heartbeat_back_addr":"172.21.15.190:6823/1766923126","heartbeat_front_addr":"172.21.15.190:6821/1766923126","state":["exists","up"]},{"osd":6,"uuid":"ac2882bc-2a7c-463c-8f6c-76ff49828adc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":342414248},{"type":"v1","addr":"172.21.15.190:6825","nonce":342414248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":342414248},{"type":"v1","addr":"172.21.15.190:6827","nonce":342414248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":342414248},{"type":"v1","addr":"172.21.15.190:6831","nonce":342414248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":342414248},{"type":"v1","addr":"172.21.15.190:6829","nonce":342414248}]},"public_addr":"172.21.15.190:6825/342414248","cluster_addr":"172.21.15.190:6827/342414248","heartbeat_back_addr":"172.21.15.190:6831/342414248","heartbeat_front_addr":"172.21.15.190:6829/342414248","state":["exists","up"]},{"osd":7,"uuid":"dd4e767a-4730-4c70-8bd5-3f088140a5a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":537849891},{"type":"v1","addr":"172.21.15.190:6809","nonce":537849891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":537849891},{"type":"v1","addr":"172.21.15.190:6811","nonce":537849891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6817","nonce":537849891},{"type":"v1","addr":"172.21.15.190:6819","nonce":537849891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6813","nonce":537849891},{"type":"v1","addr":"172.21.15.190:6815","nonce":537849891}]},"public_addr":"172.21.15.190:6809/537849891","cluster_addr":"172.21.15.190:6811/537849891","heartbeat_back_addr":"172.21.15.190:6819/537849891","heartbeat_front_addr":"172.21.15.190:6815/537849891","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.889784+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.974002+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.973549+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.955396+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.928471+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.902418+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.958696+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.926474+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T06:36:08.486 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:08.505+0000 7f78ca1e2640 1 -- 172.21.15.3:0/2336773380 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f78b006be60 msgr2=0x7f78b006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:08.486 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:08.505+0000 7f78ca1e2640 1 --2- 172.21.15.3:0/2336773380 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f78b006be60 0x7f78b006e320 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f78b8033000 tx=0x7f78b8008030 comp rx=0 tx=0).stop 2024-05-09T06:36:08.487 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:08.505+0000 7f78ca1e2640 1 -- 172.21.15.3:0/2336773380 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f78ac003f20 msgr2=0x7f78c40b6700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:08.487 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:08.505+0000 7f78ca1e2640 1 --2- 172.21.15.3:0/2336773380 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f78ac003f20 0x7f78c40b6700 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f78b400f7f0 tx=0x7f78b400fcb0 comp rx=0 tx=0).stop 2024-05-09T06:36:08.487 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:08.505+0000 7f78ca1e2640 1 -- 172.21.15.3:0/2336773380 shutdown_connections 2024-05-09T06:36:08.487 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:08.505+0000 7f78ca1e2640 1 --2- 172.21.15.3:0/2336773380 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f78b006be60 0x7f78b006e320 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:08.487 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:08.505+0000 7f78ca1e2640 1 --2- 172.21.15.3:0/2336773380 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f78c415f770 0x7f78c41f7580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:08.487 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:08.505+0000 7f78ca1e2640 1 --2- 172.21.15.3:0/2336773380 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f78c4157660 0x7f78c40cd4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:08.487 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:08.505+0000 7f78ca1e2640 1 --2- 172.21.15.3:0/2336773380 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f78ac003f20 0x7f78c40b6700 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:08.487 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:08.505+0000 7f78ca1e2640 1 -- 172.21.15.3:0/2336773380 >> 172.21.15.3:0/2336773380 conn(0x7f78c409d4d0 msgr2=0x7f78c40b2ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:08.487 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:08.505+0000 7f78ca1e2640 1 -- 172.21.15.3:0/2336773380 shutdown_connections 2024-05-09T06:36:08.487 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:08.505+0000 7f78ca1e2640 1 -- 172.21.15.3:0/2336773380 wait complete. 2024-05-09T06:36:08.497 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-05-09T06:36:08.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:08.623+0000 7f3032453640 1 Processor -- start 2024-05-09T06:36:08.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:08.623+0000 7f3032453640 1 -- start start 2024-05-09T06:36:08.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:08.624+0000 7f3032453640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f302c15c310 0x7f302c0cfbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:08.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:08.624+0000 7f3032453640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f302c0d05c0 0x7f302c0d0a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:08.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:08.624+0000 7f3032453640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f302c0d0200 con 0x7f302c09b3f0 2024-05-09T06:36:08.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:08.624+0000 7f3032453640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f302c0d0f70 con 0x7f302c0d05c0 2024-05-09T06:36:08.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:08.624+0000 7f3032453640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f302c0cce60 con 0x7f302c15c310 2024-05-09T06:36:08.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:08.624+0000 7f302b7fe640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f302c09b3f0 0x7f302c0cd940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:33072/0 (socket says 172.21.15.3:33072) 2024-05-09T06:36:08.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:08.624+0000 7f302b7fe640 1 -- 172.21.15.3:0/905752000 learned_addr learned my addr 172.21.15.3:0/905752000 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-05-09T06:36:08.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:08.624+0000 7f30309f0640 1 --2- 172.21.15.3:0/905752000 >> v2:172.21.15.190:3300/0 conn(0x7f302c0d05c0 0x7f302c0d0a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:08.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:08.624+0000 7f302bfff640 1 --2- 172.21.15.3:0/905752000 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f302c15c310 0x7f302c0cfbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:08.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:08.624+0000 7f302affd640 1 -- 172.21.15.3:0/905752000 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1186719419 0 0) 0x7f302c0d0200 con 0x7f302c09b3f0 2024-05-09T06:36:08.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:08.624+0000 7f302affd640 1 -- 172.21.15.3:0/905752000 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3018003520 con 0x7f302c09b3f0 2024-05-09T06:36:08.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:08.625+0000 7f302bfff640 1 -- 172.21.15.3:0/905752000 >> v2:172.21.15.190:3300/0 conn(0x7f302c0d05c0 msgr2=0x7f302c0d0a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:08.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:08.625+0000 7f302bfff640 1 --2- 172.21.15.3:0/905752000 >> v2:172.21.15.190:3300/0 conn(0x7f302c0d05c0 0x7f302c0d0a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:08.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:08.625+0000 7f302bfff640 1 -- 172.21.15.3:0/905752000 >> v1:172.21.15.3:6789/0 conn(0x7f302c09b3f0 legacy=0x7f302c0cd940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:08.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:08.625+0000 7f302affd640 1 -- 172.21.15.3:0/905752000 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2551810132 0 0) 0x7f3018003520 con 0x7f302c09b3f0 2024-05-09T06:36:08.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:08.625+0000 7f302bfff640 1 -- 172.21.15.3:0/905752000 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f302c0cd140 con 0x7f302c15c310 2024-05-09T06:36:08.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:08.625+0000 7f302bfff640 1 --2- 172.21.15.3:0/905752000 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f302c15c310 0x7f302c0cfbd0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f3024009d90 tx=0x7f302402a560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f1286489e1237765 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:08.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:08.626+0000 7f302affd640 1 -- 172.21.15.3:0/905752000 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3014001690 con 0x7f302c15c310 2024-05-09T06:36:08.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:08.626+0000 7f302affd640 1 -- 172.21.15.3:0/905752000 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3024007540 con 0x7f302c15c310 2024-05-09T06:36:08.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:08.626+0000 7f302affd640 1 -- 172.21.15.3:0/905752000 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30240078f0 con 0x7f302c15c310 2024-05-09T06:36:08.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:08.626+0000 7f3032453640 1 -- 172.21.15.3:0/905752000 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f302c15c310 msgr2=0x7f302c0cfbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:08.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:08.626+0000 7f3032453640 1 --2- 172.21.15.3:0/905752000 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f302c15c310 0x7f302c0cfbd0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f3024009d90 tx=0x7f302402a560 comp rx=0 tx=0).stop 2024-05-09T06:36:08.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:08.626+0000 7f3032453640 1 -- 172.21.15.3:0/905752000 shutdown_connections 2024-05-09T06:36:08.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:08.626+0000 7f3032453640 1 --2- 172.21.15.3:0/905752000 >> v2:172.21.15.190:3300/0 conn(0x7f302c0d05c0 0x7f302c0d0a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:08.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:08.626+0000 7f3032453640 1 --2- 172.21.15.3:0/905752000 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f302c15c310 0x7f302c0cfbd0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:08.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:08.626+0000 7f3032453640 1 -- 172.21.15.3:0/905752000 >> 172.21.15.3:0/905752000 conn(0x7f302c0b3af0 msgr2=0x7f302c0b3f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:08.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:08.626+0000 7f3032453640 1 -- 172.21.15.3:0/905752000 shutdown_connections 2024-05-09T06:36:08.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:08.626+0000 7f3032453640 1 -- 172.21.15.3:0/905752000 wait complete. 2024-05-09T06:36:08.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:08.627+0000 7f3032453640 1 Processor -- start 2024-05-09T06:36:08.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:08.628+0000 7f3032453640 1 -- start start 2024-05-09T06:36:08.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:08.628+0000 7f3032453640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f302c09b3f0 0x7f302c0c9340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:08.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:08.628+0000 7f3032453640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f302c0d05c0 0x7f302c06d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:08.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:08.628+0000 7f3032453640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f302c15c310 0x7f302c06d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:08.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:08.628+0000 7f3032453640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f302c0ccb10 con 0x7f302c15c310 2024-05-09T06:36:08.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:08.628+0000 7f3032453640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f302c15c6f0 con 0x7f302c09b3f0 2024-05-09T06:36:08.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:08.628+0000 7f3032453640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f302c09b810 con 0x7f302c0d05c0 2024-05-09T06:36:08.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:08.628+0000 7f30309f0640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f302c15c310 0x7f302c06d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:08.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:08.628+0000 7f30309f0640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f302c15c310 0x7f302c06d950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45750/0 (socket says 172.21.15.3:45750) 2024-05-09T06:36:08.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:08.628+0000 7f30309f0640 1 -- 172.21.15.3:0/2743593102 learned_addr learned my addr 172.21.15.3:0/2743593102 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:36:08.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:08.629+0000 7f302b7fe640 1 --2- 172.21.15.3:0/2743593102 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f302c0d05c0 0x7f302c06d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:08.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:08.629+0000 7f302bfff640 1 --2- 172.21.15.3:0/2743593102 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f302c09b3f0 0x7f302c0c9340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:08.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:08.629+0000 7f30309f0640 1 -- 172.21.15.3:0/2743593102 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f302c0d05c0 msgr2=0x7f302c06d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:08.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:08.629+0000 7f30309f0640 1 --2- 172.21.15.3:0/2743593102 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f302c0d05c0 0x7f302c06d410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:08.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:08.629+0000 7f30309f0640 1 -- 172.21.15.3:0/2743593102 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f302c09b3f0 msgr2=0x7f302c0c9340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:08.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:08.629+0000 7f30309f0640 1 --2- 172.21.15.3:0/2743593102 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f302c09b3f0 0x7f302c0c9340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:08.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:08.629+0000 7f30309f0640 1 -- 172.21.15.3:0/2743593102 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f302c15d140 con 0x7f302c15c310 2024-05-09T06:36:08.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:08.629+0000 7f30309f0640 1 --2- 172.21.15.3:0/2743593102 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f302c15c310 0x7f302c06d950 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f301c00e820 tx=0x7f301c00ec20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:08.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:08.629+0000 7f3028ff9640 1 -- 172.21.15.3:0/2743593102 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f301c017070 con 0x7f302c15c310 2024-05-09T06:36:08.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:08.629+0000 7f3028ff9640 1 -- 172.21.15.3:0/2743593102 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f301c00ee70 con 0x7f302c15c310 2024-05-09T06:36:08.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:08.629+0000 7f3032453640 1 -- 172.21.15.3:0/2743593102 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f302c15d420 con 0x7f302c15c310 2024-05-09T06:36:08.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:08.629+0000 7f3028ff9640 1 -- 172.21.15.3:0/2743593102 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f301c00cb60 con 0x7f302c15c310 2024-05-09T06:36:08.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:08.629+0000 7f3032453640 1 -- 172.21.15.3:0/2743593102 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f302c15d9e0 con 0x7f302c15c310 2024-05-09T06:36:08.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:08.630+0000 7f3032453640 1 -- 172.21.15.3:0/2743593102 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f302c069550 con 0x7f302c15c310 2024-05-09T06:36:08.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:08.631+0000 7f3028ff9640 1 -- 172.21.15.3:0/2743593102 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f301c01a070 con 0x7f302c15c310 2024-05-09T06:36:08.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:08.631+0000 7f3028ff9640 1 --2- 172.21.15.3:0/2743593102 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f300c06be30 0x7f300c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:08.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:08.631+0000 7f3028ff9640 1 -- 172.21.15.3:0/2743593102 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 5709+0+0 (secure 0 0 0) 0x7f301c08af80 con 0x7f302c15c310 2024-05-09T06:36:08.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:08.631+0000 7f302bfff640 1 --2- 172.21.15.3:0/2743593102 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f300c06be30 0x7f300c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:08.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:08.632+0000 7f302bfff640 1 --2- 172.21.15.3:0/2743593102 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f300c06be30 0x7f300c06e2f0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f3024034000 tx=0x7f302402f000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:08.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:08.635+0000 7f3028ff9640 1 -- 172.21.15.3:0/2743593102 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f301c059370 con 0x7f302c15c310 2024-05-09T06:36:08.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:08.858+0000 7f3032453640 1 -- 172.21.15.3:0/2743593102 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f302c0734c0 con 0x7f302c15c310 2024-05-09T06:36:09.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:09.860+0000 7f3028ff9640 1 -- 172.21.15.3:0/2743593102 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v7) ==== 118+0+0 (secure 0 0 0) 0x7f301c05d670 con 0x7f302c15c310 2024-05-09T06:36:09.844 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:09.863+0000 7f3032453640 1 -- 172.21.15.3:0/2743593102 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f300c06be30 msgr2=0x7f300c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:09.844 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:09.863+0000 7f3032453640 1 --2- 172.21.15.3:0/2743593102 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f300c06be30 0x7f300c06e2f0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f3024034000 tx=0x7f302402f000 comp rx=0 tx=0).stop 2024-05-09T06:36:09.845 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:09.863+0000 7f3032453640 1 -- 172.21.15.3:0/2743593102 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f302c15c310 msgr2=0x7f302c06d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:09.845 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:09.863+0000 7f3032453640 1 --2- 172.21.15.3:0/2743593102 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f302c15c310 0x7f302c06d950 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f301c00e820 tx=0x7f301c00ec20 comp rx=0 tx=0).stop 2024-05-09T06:36:09.845 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:09.863+0000 7f3032453640 1 -- 172.21.15.3:0/2743593102 shutdown_connections 2024-05-09T06:36:09.845 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:09.863+0000 7f3032453640 1 --2- 172.21.15.3:0/2743593102 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f300c06be30 0x7f300c06e2f0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:09.845 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:09.863+0000 7f3032453640 1 --2- 172.21.15.3:0/2743593102 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f302c15c310 0x7f302c06d950 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:09.845 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:09.863+0000 7f3032453640 1 --2- 172.21.15.3:0/2743593102 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f302c0d05c0 0x7f302c06d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:09.845 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:09.863+0000 7f3032453640 1 --2- 172.21.15.3:0/2743593102 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f302c09b3f0 0x7f302c0c9340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:09.845 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:09.863+0000 7f3032453640 1 -- 172.21.15.3:0/2743593102 >> 172.21.15.3:0/2743593102 conn(0x7f302c0b3af0 msgr2=0x7f302c0c7be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:09.845 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:09.864+0000 7f3032453640 1 -- 172.21.15.3:0/2743593102 shutdown_connections 2024-05-09T06:36:09.845 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:09.864+0000 7f3032453640 1 -- 172.21.15.3:0/2743593102 wait complete. 2024-05-09T06:36:09.858 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-05-09T06:36:09.858 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-05-09T06:36:09.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:09.991+0000 7f7bbf3a5640 1 Processor -- start 2024-05-09T06:36:09.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:09.992+0000 7f7bbf3a5640 1 -- start start 2024-05-09T06:36:09.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:09.992+0000 7f7bbf3a5640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7bb815e5c0 0x7f7bb80d2800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:09.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:09.992+0000 7f7bbf3a5640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7bb815e9a0 con 0x7f7bb80d1300 2024-05-09T06:36:09.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:09.992+0000 7f7bbf3a5640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7bb80d31c0 con 0x7f7bb80d4c40 2024-05-09T06:36:09.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:09.992+0000 7f7bbf3a5640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f7bb80d50a0 con 0x7f7bb815e5c0 2024-05-09T06:36:09.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:09.993+0000 7f7bbd942640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f7bb80d1300 0x7f7bb80b47e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:33078/0 (socket says 172.21.15.3:33078) 2024-05-09T06:36:09.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:09.993+0000 7f7bbd942640 1 -- 172.21.15.3:0/2321979569 learned_addr learned my addr 172.21.15.3:0/2321979569 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-05-09T06:36:09.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:09.993+0000 7f7bbd141640 1 --2- 172.21.15.3:0/2321979569 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7bb815e5c0 0x7f7bb80d2800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:09.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:09.993+0000 7f7bb7fff640 1 -- 172.21.15.3:0/2321979569 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2429314528 0 0) 0x7f7bb815e9a0 con 0x7f7bb80d1300 2024-05-09T06:36:09.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:09.993+0000 7f7bb7fff640 1 -- 172.21.15.3:0/2321979569 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7ba0003520 con 0x7f7bb80d1300 2024-05-09T06:36:09.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:09.993+0000 7f7bb7fff640 1 -- 172.21.15.3:0/2321979569 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3587364685 0 0) 0x7f7bb80d31c0 con 0x7f7bb80d4c40 2024-05-09T06:36:09.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:09.993+0000 7f7bb7fff640 1 -- 172.21.15.3:0/2321979569 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7bb815e9a0 con 0x7f7bb80d4c40 2024-05-09T06:36:09.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:09.993+0000 7f7bb7fff640 1 -- 172.21.15.3:0/2321979569 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1347145045 0 0) 0x7f7ba0003520 con 0x7f7bb80d1300 2024-05-09T06:36:09.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:09.993+0000 7f7bbd141640 1 -- 172.21.15.3:0/2321979569 >> v1:172.21.15.190:6789/0 conn(0x7f7bb80d4c40 legacy=0x7f7bb80d0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:09.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:09.993+0000 7f7bbd141640 1 -- 172.21.15.3:0/2321979569 >> v1:172.21.15.3:6789/0 conn(0x7f7bb80d1300 legacy=0x7f7bb80b47e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:09.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:09.993+0000 7f7bbd141640 1 -- 172.21.15.3:0/2321979569 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bb80d2ee0 con 0x7f7bb815e5c0 2024-05-09T06:36:09.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:09.994+0000 7f7bbd141640 1 --2- 172.21.15.3:0/2321979569 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7bb815e5c0 0x7f7bb80d2800 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f7bb00097d0 tx=0x7f7bb002a1a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e39a89d0df35435e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:09.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:09.994+0000 7f7bb7fff640 1 -- 172.21.15.3:0/2321979569 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bb0037070 con 0x7f7bb815e5c0 2024-05-09T06:36:09.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:09.994+0000 7f7bb7fff640 1 -- 172.21.15.3:0/2321979569 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7bb002ac30 con 0x7f7bb815e5c0 2024-05-09T06:36:09.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:09.994+0000 7f7bb7fff640 1 -- 172.21.15.3:0/2321979569 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ba80019f0 con 0x7f7bb815e5c0 2024-05-09T06:36:09.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:09.994+0000 7f7bbf3a5640 1 -- 172.21.15.3:0/2321979569 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7bb815e5c0 msgr2=0x7f7bb80d2800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:09.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:09.995+0000 7f7bbf3a5640 1 --2- 172.21.15.3:0/2321979569 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7bb815e5c0 0x7f7bb80d2800 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f7bb00097d0 tx=0x7f7bb002a1a0 comp rx=0 tx=0).stop 2024-05-09T06:36:09.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:09.995+0000 7f7bbf3a5640 1 -- 172.21.15.3:0/2321979569 shutdown_connections 2024-05-09T06:36:09.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:09.995+0000 7f7bbf3a5640 1 --2- 172.21.15.3:0/2321979569 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7bb815e5c0 0x7f7bb80d2800 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:09.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:09.995+0000 7f7bbf3a5640 1 -- 172.21.15.3:0/2321979569 >> 172.21.15.3:0/2321979569 conn(0x7f7bb809d500 msgr2=0x7f7bb80d4220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:09.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:09.995+0000 7f7bbf3a5640 1 -- 172.21.15.3:0/2321979569 shutdown_connections 2024-05-09T06:36:09.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:09.995+0000 7f7bbf3a5640 1 -- 172.21.15.3:0/2321979569 wait complete. 2024-05-09T06:36:09.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:09.996+0000 7f7bbf3a5640 1 Processor -- start 2024-05-09T06:36:09.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:09.996+0000 7f7bbf3a5640 1 -- start start 2024-05-09T06:36:09.979 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:09.997+0000 7f7bbf3a5640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7bb80d1300 0x7f7bb81f4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:09.979 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:09.997+0000 7f7bbf3a5640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7bb80d4c40 0x7f7bb80b86f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:09.979 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:09.997+0000 7f7bbf3a5640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7bb815e5c0 0x7f7bb81f72f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:09.979 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:09.997+0000 7f7bbf3a5640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f7bb80b42c0 con 0x7f7bb80d4c40 2024-05-09T06:36:09.979 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:09.997+0000 7f7bbf3a5640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f7bb80b3fe0 con 0x7f7bb815e5c0 2024-05-09T06:36:09.979 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:09.997+0000 7f7bbf3a5640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f7bb80b4ef0 con 0x7f7bb80d1300 2024-05-09T06:36:09.979 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:09.997+0000 7f7bbc940640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7bb80d4c40 0x7f7bb80b86f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:09.979 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:09.997+0000 7f7bbc940640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7bb80d4c40 0x7f7bb80b86f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45754/0 (socket says 172.21.15.3:45754) 2024-05-09T06:36:09.979 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:09.997+0000 7f7bbc940640 1 -- 172.21.15.3:0/3835523641 learned_addr learned my addr 172.21.15.3:0/3835523641 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:36:09.979 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:09.997+0000 7f7bbd141640 1 --2- 172.21.15.3:0/3835523641 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7bb80d1300 0x7f7bb81f4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:09.980 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:09.997+0000 7f7bbd942640 1 --2- 172.21.15.3:0/3835523641 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7bb815e5c0 0x7f7bb81f72f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:09.980 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:09.997+0000 7f7bbc940640 1 -- 172.21.15.3:0/3835523641 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7bb80d1300 msgr2=0x7f7bb81f4ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:09.980 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:09.997+0000 7f7bbc940640 1 --2- 172.21.15.3:0/3835523641 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7bb80d1300 0x7f7bb81f4ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:09.980 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:09.997+0000 7f7bbc940640 1 -- 172.21.15.3:0/3835523641 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7bb815e5c0 msgr2=0x7f7bb81f72f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:09.980 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:09.997+0000 7f7bbc940640 1 --2- 172.21.15.3:0/3835523641 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7bb815e5c0 0x7f7bb81f72f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:09.980 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:09.997+0000 7f7bbc940640 1 -- 172.21.15.3:0/3835523641 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bb80cb540 con 0x7f7bb80d4c40 2024-05-09T06:36:09.980 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:09.997+0000 7f7bbc940640 1 --2- 172.21.15.3:0/3835523641 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7bb80d4c40 0x7f7bb80b86f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f7bac00c490 tx=0x7f7bac00c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:09.980 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:09.997+0000 7f7bb5ffb640 1 -- 172.21.15.3:0/3835523641 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bac0052d0 con 0x7f7bb80d4c40 2024-05-09T06:36:09.980 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:09.997+0000 7f7bb5ffb640 1 -- 172.21.15.3:0/3835523641 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7bac005470 con 0x7f7bb80d4c40 2024-05-09T06:36:09.980 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:09.997+0000 7f7bbf3a5640 1 -- 172.21.15.3:0/3835523641 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7bb80cb820 con 0x7f7bb80d4c40 2024-05-09T06:36:09.980 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:09.998+0000 7f7bb5ffb640 1 -- 172.21.15.3:0/3835523641 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bac012870 con 0x7f7bb80d4c40 2024-05-09T06:36:09.980 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:09.998+0000 7f7bbf3a5640 1 -- 172.21.15.3:0/3835523641 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7bb80cbde0 con 0x7f7bb80d4c40 2024-05-09T06:36:09.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:09.999+0000 7f7bbf3a5640 1 -- 172.21.15.3:0/3835523641 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7bb8072d50 con 0x7f7bb80d4c40 2024-05-09T06:36:09.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:09.999+0000 7f7bb5ffb640 1 -- 172.21.15.3:0/3835523641 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7bac012a10 con 0x7f7bb80d4c40 2024-05-09T06:36:09.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:10.000+0000 7f7bb5ffb640 1 --2- 172.21.15.3:0/3835523641 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f7ba406be30 0x7f7ba406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:09.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:10.000+0000 7f7bb5ffb640 1 -- 172.21.15.3:0/3835523641 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f7bac08a9a0 con 0x7f7bb80d4c40 2024-05-09T06:36:09.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:10.000+0000 7f7bbd141640 1 --2- 172.21.15.3:0/3835523641 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f7ba406be30 0x7f7ba406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:09.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:10.000+0000 7f7bbd141640 1 --2- 172.21.15.3:0/3835523641 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f7ba406be30 0x7f7ba406e2f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f7bb0034000 tx=0x7f7bb002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:09.985 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:10.004+0000 7f7bb5ffb640 1 -- 172.21.15.3:0/3835523641 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f7bb8072d50 con 0x7f7bb80d4c40 2024-05-09T06:36:10.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:10.229+0000 7f7bbf3a5640 1 -- 172.21.15.3:0/3835523641 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f7bb8069550 con 0x7f7bb80d4c40 2024-05-09T06:36:10.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:10.869+0000 7f7bb5ffb640 1 -- 172.21.15.3:0/3835523641 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v8) ==== 122+0+0 (secure 0 0 0) 0x7f7bac058d00 con 0x7f7bb80d4c40 2024-05-09T06:36:10.852 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:10.871+0000 7f7bbf3a5640 1 -- 172.21.15.3:0/3835523641 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f7ba406be30 msgr2=0x7f7ba406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:10.852 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:10.871+0000 7f7bbf3a5640 1 --2- 172.21.15.3:0/3835523641 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f7ba406be30 0x7f7ba406e2f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f7bb0034000 tx=0x7f7bb002e000 comp rx=0 tx=0).stop 2024-05-09T06:36:10.852 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:10.871+0000 7f7bbf3a5640 1 -- 172.21.15.3:0/3835523641 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7bb80d4c40 msgr2=0x7f7bb80b86f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:10.853 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:10.871+0000 7f7bbf3a5640 1 --2- 172.21.15.3:0/3835523641 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7bb80d4c40 0x7f7bb80b86f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f7bac00c490 tx=0x7f7bac00c950 comp rx=0 tx=0).stop 2024-05-09T06:36:10.853 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:10.871+0000 7f7bbf3a5640 1 -- 172.21.15.3:0/3835523641 shutdown_connections 2024-05-09T06:36:10.853 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:10.871+0000 7f7bbf3a5640 1 --2- 172.21.15.3:0/3835523641 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f7ba406be30 0x7f7ba406e2f0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:10.853 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:10.871+0000 7f7bbf3a5640 1 --2- 172.21.15.3:0/3835523641 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7bb815e5c0 0x7f7bb81f72f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:10.853 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:10.871+0000 7f7bbf3a5640 1 --2- 172.21.15.3:0/3835523641 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7bb80d4c40 0x7f7bb80b86f0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:10.853 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:10.871+0000 7f7bbf3a5640 1 --2- 172.21.15.3:0/3835523641 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7bb80d1300 0x7f7bb81f4ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:10.853 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:10.871+0000 7f7bbf3a5640 1 -- 172.21.15.3:0/3835523641 >> 172.21.15.3:0/3835523641 conn(0x7f7bb809d500 msgr2=0x7f7bb80cf100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:10.853 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:10.872+0000 7f7bbf3a5640 1 -- 172.21.15.3:0/3835523641 shutdown_connections 2024-05-09T06:36:10.853 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:10.872+0000 7f7bbf3a5640 1 -- 172.21.15.3:0/3835523641 wait complete. 2024-05-09T06:36:10.865 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-05-09T06:36:10.866 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T06:36:10.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:10.994+0000 7f3efb1f9640 1 Processor -- start 2024-05-09T06:36:10.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:10.994+0000 7f3efb1f9640 1 -- start start 2024-05-09T06:36:10.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:10.995+0000 7f3efb1f9640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f3ef4145ab0 0x7f3ef40d0760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:10.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:10.995+0000 7f3efb1f9640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3ef406f530 0x7f3ef406f970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:10.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:10.995+0000 7f3efb1f9640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3ef4160750 con 0x7f3ef40d0ca0 2024-05-09T06:36:10.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:10.995+0000 7f3efb1f9640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f3ef406feb0 con 0x7f3ef4145ab0 2024-05-09T06:36:10.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:10.995+0000 7f3efb1f9640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f3ef4160a30 con 0x7f3ef406f530 2024-05-09T06:36:10.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:10.995+0000 7f3ef8f95640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f3ef4145ab0 0x7f3ef40d0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:10.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:10.995+0000 7f3ef8f95640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f3ef4145ab0 0x7f3ef40d0760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49362/0 (socket says 172.21.15.3:49362) 2024-05-09T06:36:10.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:10.995+0000 7f3ef8f95640 1 -- 172.21.15.3:0/517215766 learned_addr learned my addr 172.21.15.3:0/517215766 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:36:10.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:10.996+0000 7f3ef9796640 1 --2- 172.21.15.3:0/517215766 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3ef406f530 0x7f3ef406f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:10.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:10.996+0000 7f3ef8f95640 1 -- 172.21.15.3:0/517215766 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3ef406f530 msgr2=0x7f3ef406f970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:10.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:10.996+0000 7f3ef8f95640 1 --2- 172.21.15.3:0/517215766 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3ef406f530 0x7f3ef406f970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:10.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:10.996+0000 7f3ef8f95640 1 -- 172.21.15.3:0/517215766 >> v1:172.21.15.190:6789/0 conn(0x7f3ef40d0ca0 legacy=0x7f3ef415fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:10.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:10.996+0000 7f3ef8f95640 1 -- 172.21.15.3:0/517215766 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ef40b5460 con 0x7f3ef4145ab0 2024-05-09T06:36:10.979 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:10.996+0000 7f3ef8f95640 1 --2- 172.21.15.3:0/517215766 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3ef4145ab0 0x7f3ef40d0760 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f3ee4009870 tx=0x7f3ee402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=866529071d3ec66c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:10.979 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:10.997+0000 7f3ef37fe640 1 -- 172.21.15.3:0/517215766 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ee4037070 con 0x7f3ef4145ab0 2024-05-09T06:36:10.979 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:10.997+0000 7f3ef37fe640 1 -- 172.21.15.3:0/517215766 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ee4032440 con 0x7f3ef4145ab0 2024-05-09T06:36:10.979 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:10.997+0000 7f3ef37fe640 1 -- 172.21.15.3:0/517215766 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ee4032810 con 0x7f3ef4145ab0 2024-05-09T06:36:10.979 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:10.997+0000 7f3efb1f9640 1 -- 172.21.15.3:0/517215766 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3ef4145ab0 msgr2=0x7f3ef40d0760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:10.979 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:10.997+0000 7f3efb1f9640 1 --2- 172.21.15.3:0/517215766 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3ef4145ab0 0x7f3ef40d0760 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f3ee4009870 tx=0x7f3ee402a270 comp rx=0 tx=0).stop 2024-05-09T06:36:10.979 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:10.997+0000 7f3efb1f9640 1 -- 172.21.15.3:0/517215766 shutdown_connections 2024-05-09T06:36:10.979 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:10.997+0000 7f3efb1f9640 1 --2- 172.21.15.3:0/517215766 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3ef406f530 0x7f3ef406f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:10.979 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:10.997+0000 7f3efb1f9640 1 --2- 172.21.15.3:0/517215766 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3ef4145ab0 0x7f3ef40d0760 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:10.979 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:10.997+0000 7f3efb1f9640 1 -- 172.21.15.3:0/517215766 >> 172.21.15.3:0/517215766 conn(0x7f3ef40d1f70 msgr2=0x7f3ef40d2350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:10.979 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:10.997+0000 7f3efb1f9640 1 -- 172.21.15.3:0/517215766 shutdown_connections 2024-05-09T06:36:10.979 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:10.997+0000 7f3efb1f9640 1 -- 172.21.15.3:0/517215766 wait complete. 2024-05-09T06:36:10.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:10.998+0000 7f3efb1f9640 1 Processor -- start 2024-05-09T06:36:10.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:10.999+0000 7f3efb1f9640 1 -- start start 2024-05-09T06:36:10.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:10.999+0000 7f3efb1f9640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3ef406f530 0x7f3ef41f1cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:10.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:10.999+0000 7f3efb1f9640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3ef40d0ca0 0x7f3ef40b3d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:10.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:10.999+0000 7f3efb1f9640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3ef4145ab0 0x7f3ef40b42a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:10.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:10.999+0000 7f3efb1f9640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f3ef40b70c0 con 0x7f3ef4145ab0 2024-05-09T06:36:10.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:10.999+0000 7f3efb1f9640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f3ef4145e90 con 0x7f3ef40d0ca0 2024-05-09T06:36:10.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:10.999+0000 7f3efb1f9640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f3ef41603a0 con 0x7f3ef406f530 2024-05-09T06:36:10.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:11.000+0000 7f3ef9796640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3ef4145ab0 0x7f3ef40b42a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:10.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:11.000+0000 7f3ef9796640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3ef4145ab0 0x7f3ef40b42a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49368/0 (socket says 172.21.15.3:49368) 2024-05-09T06:36:10.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:11.000+0000 7f3ef9796640 1 -- 172.21.15.3:0/383757646 learned_addr learned my addr 172.21.15.3:0/383757646 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:36:10.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:11.000+0000 7f3ef8f95640 1 --2- 172.21.15.3:0/383757646 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3ef406f530 0x7f3ef41f1cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:10.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:11.000+0000 7f3ef3fff640 1 --2- 172.21.15.3:0/383757646 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3ef40d0ca0 0x7f3ef40b3d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:10.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:11.000+0000 7f3ef9796640 1 -- 172.21.15.3:0/383757646 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3ef406f530 msgr2=0x7f3ef41f1cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:10.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:11.000+0000 7f3ef9796640 1 --2- 172.21.15.3:0/383757646 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3ef406f530 0x7f3ef41f1cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:10.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:11.000+0000 7f3ef9796640 1 -- 172.21.15.3:0/383757646 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3ef40d0ca0 msgr2=0x7f3ef40b3d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:10.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:11.000+0000 7f3ef9796640 1 --2- 172.21.15.3:0/383757646 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3ef40d0ca0 0x7f3ef40b3d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:10.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:11.000+0000 7f3ef9796640 1 -- 172.21.15.3:0/383757646 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ef41e67c0 con 0x7f3ef4145ab0 2024-05-09T06:36:10.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:11.000+0000 7f3ef9796640 1 --2- 172.21.15.3:0/383757646 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3ef4145ab0 0x7f3ef40b42a0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f3eec007e60 tx=0x7f3eec00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:10.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:11.000+0000 7f3ef17fa640 1 -- 172.21.15.3:0/383757646 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3eec014420 con 0x7f3ef4145ab0 2024-05-09T06:36:10.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:11.000+0000 7f3ef17fa640 1 -- 172.21.15.3:0/383757646 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3eec019070 con 0x7f3ef4145ab0 2024-05-09T06:36:10.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:11.000+0000 7f3efb1f9640 1 -- 172.21.15.3:0/383757646 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ef41f2500 con 0x7f3ef4145ab0 2024-05-09T06:36:10.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:11.000+0000 7f3ef17fa640 1 -- 172.21.15.3:0/383757646 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3eec014820 con 0x7f3ef4145ab0 2024-05-09T06:36:10.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:11.000+0000 7f3efb1f9640 1 -- 172.21.15.3:0/383757646 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ef41f2ac0 con 0x7f3ef4145ab0 2024-05-09T06:36:10.984 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:11.001+0000 7f3efb1f9640 1 -- 172.21.15.3:0/383757646 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ef415b720 con 0x7f3ef4145ab0 2024-05-09T06:36:10.984 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:11.002+0000 7f3ef17fa640 1 -- 172.21.15.3:0/383757646 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3eec0149c0 con 0x7f3ef4145ab0 2024-05-09T06:36:10.984 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:11.002+0000 7f3ef17fa640 1 --2- 172.21.15.3:0/383757646 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f3ee006be30 0x7f3ee006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:10.984 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:11.002+0000 7f3ef17fa640 1 -- 172.21.15.3:0/383757646 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f3eec08b600 con 0x7f3ef4145ab0 2024-05-09T06:36:10.984 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:11.002+0000 7f3ef8f95640 1 --2- 172.21.15.3:0/383757646 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f3ee006be30 0x7f3ee006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:10.984 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:11.003+0000 7f3ef8f95640 1 --2- 172.21.15.3:0/383757646 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f3ee006be30 0x7f3ee006e2f0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f3ee4034000 tx=0x7f3ee402e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:10.987 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:11.006+0000 7f3ef17fa640 1 -- 172.21.15.3:0/383757646 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f3ef415b720 con 0x7f3ef4145ab0 2024-05-09T06:36:11.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:11.229+0000 7f3efb1f9640 1 -- 172.21.15.3:0/383757646 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3ef41f2cc0 con 0x7f3ef4145ab0 2024-05-09T06:36:11.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:11.230+0000 7f3ef17fa640 1 -- 172.21.15.3:0/383757646 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5172 (secure 0 0 0) 0x7f3eec05a040 con 0x7f3ef4145ab0 2024-05-09T06:36:11.211 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-09T06:36:11.211 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4272,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6834/126273055","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":126273055},{"type":"v1","addr":"172.21.15.3:6834","nonce":126273055}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4273,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6836/3885692063","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":3885692063},{"type":"v1","addr":"172.21.15.190:6836","nonce":3885692063}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4275,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/3461228756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6833","nonce":3461228756},{"type":"v1","addr":"172.21.15.3:6835","nonce":3461228756}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-09T06:36:07.221126+0000","modified":"2024-05-09T06:36:10.231051+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4276},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4276":{"gid":4276,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.190:6837/4111472052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6835","nonce":4111472052},{"type":"v1","addr":"172.21.15.190:6837","nonce":4111472052}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4276,"qdb_cluster":[4276]},"id":1}]} 2024-05-09T06:36:11.211 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 8 2024-05-09T06:36:11.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:11.232+0000 7f3efb1f9640 1 -- 172.21.15.3:0/383757646 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f3ee006be30 msgr2=0x7f3ee006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:11.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:11.232+0000 7f3efb1f9640 1 --2- 172.21.15.3:0/383757646 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f3ee006be30 0x7f3ee006e2f0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f3ee4034000 tx=0x7f3ee402e000 comp rx=0 tx=0).stop 2024-05-09T06:36:11.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:11.232+0000 7f3efb1f9640 1 -- 172.21.15.3:0/383757646 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3ef4145ab0 msgr2=0x7f3ef40b42a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:11.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:11.232+0000 7f3efb1f9640 1 --2- 172.21.15.3:0/383757646 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3ef4145ab0 0x7f3ef40b42a0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f3eec007e60 tx=0x7f3eec00e4b0 comp rx=0 tx=0).stop 2024-05-09T06:36:11.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:11.232+0000 7f3efb1f9640 1 -- 172.21.15.3:0/383757646 shutdown_connections 2024-05-09T06:36:11.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:11.232+0000 7f3efb1f9640 1 --2- 172.21.15.3:0/383757646 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f3ee006be30 0x7f3ee006e2f0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:11.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:11.232+0000 7f3efb1f9640 1 --2- 172.21.15.3:0/383757646 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3ef4145ab0 0x7f3ef40b42a0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:11.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:11.232+0000 7f3efb1f9640 1 --2- 172.21.15.3:0/383757646 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3ef40d0ca0 0x7f3ef40b3d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:11.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:11.232+0000 7f3efb1f9640 1 --2- 172.21.15.3:0/383757646 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3ef406f530 0x7f3ef41f1cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:11.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:11.232+0000 7f3efb1f9640 1 -- 172.21.15.3:0/383757646 >> 172.21.15.3:0/383757646 conn(0x7f3ef40d1f70 msgr2=0x7f3ef415e920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:11.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:11.232+0000 7f3efb1f9640 1 -- 172.21.15.3:0/383757646 shutdown_connections 2024-05-09T06:36:11.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:11.232+0000 7f3efb1f9640 1 -- 172.21.15.3:0/383757646 wait complete. 2024-05-09T06:36:11.224 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T06:36:11.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:11.347+0000 7fda47f32640 1 Processor -- start 2024-05-09T06:36:11.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:11.347+0000 7fda47f32640 1 -- start start 2024-05-09T06:36:11.330 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:11.348+0000 7fda47f32640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fda4015e420 0x7fda4006b250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:11.330 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:11.348+0000 7fda47f32640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fda400b4d50 0x7fda400aeae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:11.331 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:11.348+0000 7fda47f32640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fda400ae600 con 0x7fda4006b790 2024-05-09T06:36:11.331 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:11.348+0000 7fda47f32640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fda4015e800 con 0x7fda400b4d50 2024-05-09T06:36:11.331 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:11.348+0000 7fda47f32640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7fda400b5290 con 0x7fda4015e420 2024-05-09T06:36:11.331 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:11.348+0000 7fda464cf640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fda400b4d50 0x7fda400aeae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:11.331 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:11.348+0000 7fda454cd640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fda4006b790 0x7fda400b4640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:49182/0 (socket says 172.21.15.3:49182) 2024-05-09T06:36:11.331 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:11.348+0000 7fda454cd640 1 -- 172.21.15.3:0/2924704065 learned_addr learned my addr 172.21.15.3:0/2924704065 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-05-09T06:36:11.331 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:11.348+0000 7fda45cce640 1 --2- 172.21.15.3:0/2924704065 >> v2:172.21.15.190:3300/0 conn(0x7fda4015e420 0x7fda4006b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:11.331 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:11.348+0000 7fda464cf640 1 -- 172.21.15.3:0/2924704065 >> v2:172.21.15.190:3300/0 conn(0x7fda4015e420 msgr2=0x7fda4006b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:11.331 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:11.348+0000 7fda464cf640 1 --2- 172.21.15.3:0/2924704065 >> v2:172.21.15.190:3300/0 conn(0x7fda4015e420 0x7fda4006b250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:11.331 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:11.348+0000 7fda464cf640 1 -- 172.21.15.3:0/2924704065 >> v1:172.21.15.3:6789/0 conn(0x7fda4006b790 legacy=0x7fda400b4640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:11.331 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:11.348+0000 7fda44ccc640 1 -- 172.21.15.3:0/2924704065 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1362231192 0 0) 0x7fda400ae600 con 0x7fda4006b790 2024-05-09T06:36:11.331 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:11.348+0000 7fda464cf640 1 -- 172.21.15.3:0/2924704065 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda400b5570 con 0x7fda400b4d50 2024-05-09T06:36:11.331 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:11.348+0000 7fda45cce640 1 --2- 172.21.15.3:0/2924704065 >> v2:172.21.15.190:3300/0 conn(0x7fda4015e420 0x7fda4006b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:36:11.331 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:11.349+0000 7fda464cf640 1 --2- 172.21.15.3:0/2924704065 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fda400b4d50 0x7fda400aeae0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fda40070d40 tx=0x7fda3802a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80e7c61d5d626b26 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:11.331 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:11.349+0000 7fda44ccc640 1 -- 172.21.15.3:0/2924704065 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda38007740 con 0x7fda400b4d50 2024-05-09T06:36:11.332 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:11.349+0000 7fda44ccc640 1 -- 172.21.15.3:0/2924704065 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda380078e0 con 0x7fda400b4d50 2024-05-09T06:36:11.332 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:11.349+0000 7fda44ccc640 1 -- 172.21.15.3:0/2924704065 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda38007cb0 con 0x7fda400b4d50 2024-05-09T06:36:11.332 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:11.350+0000 7fda47f32640 1 -- 172.21.15.3:0/2924704065 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fda400b4d50 msgr2=0x7fda400aeae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:11.332 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:11.350+0000 7fda47f32640 1 --2- 172.21.15.3:0/2924704065 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fda400b4d50 0x7fda400aeae0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fda40070d40 tx=0x7fda3802a5f0 comp rx=0 tx=0).stop 2024-05-09T06:36:11.332 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:11.350+0000 7fda47f32640 1 -- 172.21.15.3:0/2924704065 shutdown_connections 2024-05-09T06:36:11.332 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:11.350+0000 7fda47f32640 1 --2- 172.21.15.3:0/2924704065 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fda400b4d50 0x7fda400aeae0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:11.332 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:11.350+0000 7fda47f32640 1 --2- 172.21.15.3:0/2924704065 >> v2:172.21.15.190:3300/0 conn(0x7fda4015e420 0x7fda4006b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:11.332 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:11.350+0000 7fda47f32640 1 -- 172.21.15.3:0/2924704065 >> 172.21.15.3:0/2924704065 conn(0x7fda4006e3d0 msgr2=0x7fda4006e7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:11.332 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:11.350+0000 7fda47f32640 1 -- 172.21.15.3:0/2924704065 shutdown_connections 2024-05-09T06:36:11.332 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:11.350+0000 7fda47f32640 1 -- 172.21.15.3:0/2924704065 wait complete. 2024-05-09T06:36:11.335 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:11.351+0000 7fda47f32640 1 Processor -- start 2024-05-09T06:36:11.335 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:11.351+0000 7fda47f32640 1 -- start start 2024-05-09T06:36:11.335 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:11.352+0000 7fda47f32640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fda4006b790 0x7fda40152100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:11.335 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:11.352+0000 7fda47f32640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fda400b4d50 0x7fda401603e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:11.335 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:11.352+0000 7fda47f32640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fda4015e420 0x7fda40160920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:11.335 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:11.352+0000 7fda47f32640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fda400af020 con 0x7fda4015e420 2024-05-09T06:36:11.335 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:11.352+0000 7fda47f32640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fda400b5b30 con 0x7fda4006b790 2024-05-09T06:36:11.335 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:11.352+0000 7fda47f32640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fda400af1a0 con 0x7fda400b4d50 2024-05-09T06:36:11.335 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:11.352+0000 7fda464cf640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fda4015e420 0x7fda40160920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:11.335 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:11.352+0000 7fda464cf640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fda4015e420 0x7fda40160920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49380/0 (socket says 172.21.15.3:49380) 2024-05-09T06:36:11.336 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:11.352+0000 7fda464cf640 1 -- 172.21.15.3:0/2774945163 learned_addr learned my addr 172.21.15.3:0/2774945163 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:36:11.336 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:11.352+0000 7fda45cce640 1 --2- 172.21.15.3:0/2774945163 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fda4006b790 0x7fda40152100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:11.336 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:11.352+0000 7fda454cd640 1 --2- 172.21.15.3:0/2774945163 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fda400b4d50 0x7fda401603e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:11.336 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:11.352+0000 7fda464cf640 1 -- 172.21.15.3:0/2774945163 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fda400b4d50 msgr2=0x7fda401603e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:11.336 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:11.352+0000 7fda464cf640 1 --2- 172.21.15.3:0/2774945163 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fda400b4d50 0x7fda401603e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:11.336 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:11.352+0000 7fda464cf640 1 -- 172.21.15.3:0/2774945163 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fda4006b790 msgr2=0x7fda40152100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:11.336 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:11.352+0000 7fda464cf640 1 --2- 172.21.15.3:0/2774945163 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fda4006b790 0x7fda40152100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:11.336 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:11.352+0000 7fda464cf640 1 -- 172.21.15.3:0/2774945163 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda40155e60 con 0x7fda4015e420 2024-05-09T06:36:11.336 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:11.352+0000 7fda454cd640 1 --2- 172.21.15.3:0/2774945163 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fda400b4d50 0x7fda401603e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:36:11.336 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:11.352+0000 7fda464cf640 1 --2- 172.21.15.3:0/2774945163 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fda4015e420 0x7fda40160920 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fda3802abe0 tx=0x7fda38004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:11.336 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:11.352+0000 7fda367fc640 1 -- 172.21.15.3:0/2774945163 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda38040070 con 0x7fda4015e420 2024-05-09T06:36:11.336 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:11.352+0000 7fda367fc640 1 -- 172.21.15.3:0/2774945163 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda38031bc0 con 0x7fda4015e420 2024-05-09T06:36:11.336 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:11.352+0000 7fda367fc640 1 -- 172.21.15.3:0/2774945163 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda38045460 con 0x7fda4015e420 2024-05-09T06:36:11.336 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:11.352+0000 7fda47f32640 1 -- 172.21.15.3:0/2774945163 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda40152750 con 0x7fda4015e420 2024-05-09T06:36:11.336 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:11.353+0000 7fda47f32640 1 -- 172.21.15.3:0/2774945163 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda40154e80 con 0x7fda4015e420 2024-05-09T06:36:11.336 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:11.353+0000 7fda47f32640 1 -- 172.21.15.3:0/2774945163 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda4004f660 con 0x7fda4015e420 2024-05-09T06:36:11.337 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:11.354+0000 7fda367fc640 1 -- 172.21.15.3:0/2774945163 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fda38039040 con 0x7fda4015e420 2024-05-09T06:36:11.337 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:11.354+0000 7fda367fc640 1 --2- 172.21.15.3:0/2774945163 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fda2406be60 0x7fda2406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:11.337 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:11.354+0000 7fda367fc640 1 -- 172.21.15.3:0/2774945163 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fda380ac7d0 con 0x7fda4015e420 2024-05-09T06:36:11.337 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:11.355+0000 7fda45cce640 1 --2- 172.21.15.3:0/2774945163 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fda2406be60 0x7fda2406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:11.337 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:11.355+0000 7fda45cce640 1 --2- 172.21.15.3:0/2774945163 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fda2406be60 0x7fda2406e320 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fda3c00aac0 tx=0x7fda3c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:11.339 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:11.359+0000 7fda367fc640 1 -- 172.21.15.3:0/2774945163 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fda3807b180 con 0x7fda4015e420 2024-05-09T06:36:11.356 INFO:tasks.ceph.mds.d.smithi190.stderr:2024-05-09T06:36:11.376+0000 7f33d478a640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-09T06:36:11.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:11.542+0000 7fda47f32640 1 -- 172.21.15.3:0/2774945163 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fda40155050 con 0x7fda4015e420 2024-05-09T06:36:11.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:11.543+0000 7fda367fc640 1 -- 172.21.15.3:0/2774945163 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16521 (secure 0 0 0) 0x7fda3807aa80 con 0x7fda4015e420 2024-05-09T06:36:11.524 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-09T06:36:11.524 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":20,"fsid":"16ed737d-2665-431a-8ac6-ef242be3975a","created":"2024-05-09T06:35:42.541368+0000","modified":"2024-05-09T06:36:08.856391+0000","last_up_change":"2024-05-09T06:35:53.778274+0000","last_in_change":"2024-05-09T06:35:44.780586+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T06:35:54.819999+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T06:36:04.931314+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T06:36:06.200433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fae3a8db-4826-4f3c-b33a-12c09bb5df6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":1897849827},{"type":"v1","addr":"172.21.15.3:6817","nonce":1897849827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":1897849827},{"type":"v1","addr":"172.21.15.3:6819","nonce":1897849827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":1897849827},{"type":"v1","addr":"172.21.15.3:6823","nonce":1897849827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":1897849827},{"type":"v1","addr":"172.21.15.3:6821","nonce":1897849827}]},"public_addr":"172.21.15.3:6817/1897849827","cluster_addr":"172.21.15.3:6819/1897849827","heartbeat_back_addr":"172.21.15.3:6823/1897849827","heartbeat_front_addr":"172.21.15.3:6821/1897849827","state":["exists","up"]},{"osd":1,"uuid":"646e882d-4ce4-4720-8c84-f71df563a35e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":926805498},{"type":"v1","addr":"172.21.15.3:6801","nonce":926805498}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":926805498},{"type":"v1","addr":"172.21.15.3:6803","nonce":926805498}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":926805498},{"type":"v1","addr":"172.21.15.3:6807","nonce":926805498}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":926805498},{"type":"v1","addr":"172.21.15.3:6805","nonce":926805498}]},"public_addr":"172.21.15.3:6801/926805498","cluster_addr":"172.21.15.3:6803/926805498","heartbeat_back_addr":"172.21.15.3:6807/926805498","heartbeat_front_addr":"172.21.15.3:6805/926805498","state":["exists","up"]},{"osd":2,"uuid":"1abbadb5-a570-4359-9643-acff3ede35a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":801295451},{"type":"v1","addr":"172.21.15.3:6809","nonce":801295451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":801295451},{"type":"v1","addr":"172.21.15.3:6811","nonce":801295451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":801295451},{"type":"v1","addr":"172.21.15.3:6815","nonce":801295451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":801295451},{"type":"v1","addr":"172.21.15.3:6813","nonce":801295451}]},"public_addr":"172.21.15.3:6809/801295451","cluster_addr":"172.21.15.3:6811/801295451","heartbeat_back_addr":"172.21.15.3:6815/801295451","heartbeat_front_addr":"172.21.15.3:6813/801295451","state":["exists","up"]},{"osd":3,"uuid":"62ac4dc1-2371-4dd2-acd3-e2dbb11aedf5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":1903389857},{"type":"v1","addr":"172.21.15.3:6825","nonce":1903389857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":1903389857},{"type":"v1","addr":"172.21.15.3:6827","nonce":1903389857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":1903389857},{"type":"v1","addr":"172.21.15.3:6831","nonce":1903389857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":1903389857},{"type":"v1","addr":"172.21.15.3:6829","nonce":1903389857}]},"public_addr":"172.21.15.3:6825/1903389857","cluster_addr":"172.21.15.3:6827/1903389857","heartbeat_back_addr":"172.21.15.3:6831/1903389857","heartbeat_front_addr":"172.21.15.3:6829/1903389857","state":["exists","up"]},{"osd":4,"uuid":"ae808398-8326-4d7c-9319-a2a0eafffb59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":2143624576},{"type":"v1","addr":"172.21.15.190:6801","nonce":2143624576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":2143624576},{"type":"v1","addr":"172.21.15.190:6803","nonce":2143624576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":2143624576},{"type":"v1","addr":"172.21.15.190:6807","nonce":2143624576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":2143624576},{"type":"v1","addr":"172.21.15.190:6805","nonce":2143624576}]},"public_addr":"172.21.15.190:6801/2143624576","cluster_addr":"172.21.15.190:6803/2143624576","heartbeat_back_addr":"172.21.15.190:6807/2143624576","heartbeat_front_addr":"172.21.15.190:6805/2143624576","state":["exists","up"]},{"osd":5,"uuid":"918adf50-c440-48ac-9a8d-4837562d7275","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":1766923126},{"type":"v1","addr":"172.21.15.190:6814","nonce":1766923126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":1766923126},{"type":"v1","addr":"172.21.15.190:6818","nonce":1766923126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":1766923126},{"type":"v1","addr":"172.21.15.190:6823","nonce":1766923126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":1766923126},{"type":"v1","addr":"172.21.15.190:6821","nonce":1766923126}]},"public_addr":"172.21.15.190:6814/1766923126","cluster_addr":"172.21.15.190:6818/1766923126","heartbeat_back_addr":"172.21.15.190:6823/1766923126","heartbeat_front_addr":"172.21.15.190:6821/1766923126","state":["exists","up"]},{"osd":6,"uuid":"ac2882bc-2a7c-463c-8f6c-76ff49828adc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":342414248},{"type":"v1","addr":"172.21.15.190:6825","nonce":342414248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":342414248},{"type":"v1","addr":"172.21.15.190:6827","nonce":342414248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":342414248},{"type":"v1","addr":"172.21.15.190:6831","nonce":342414248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":342414248},{"type":"v1","addr":"172.21.15.190:6829","nonce":342414248}]},"public_addr":"172.21.15.190:6825/342414248","cluster_addr":"172.21.15.190:6827/342414248","heartbeat_back_addr":"172.21.15.190:6831/342414248","heartbeat_front_addr":"172.21.15.190:6829/342414248","state":["exists","up"]},{"osd":7,"uuid":"dd4e767a-4730-4c70-8bd5-3f088140a5a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":537849891},{"type":"v1","addr":"172.21.15.190:6809","nonce":537849891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":537849891},{"type":"v1","addr":"172.21.15.190:6811","nonce":537849891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6817","nonce":537849891},{"type":"v1","addr":"172.21.15.190:6819","nonce":537849891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6813","nonce":537849891},{"type":"v1","addr":"172.21.15.190:6815","nonce":537849891}]},"public_addr":"172.21.15.190:6809/537849891","cluster_addr":"172.21.15.190:6811/537849891","heartbeat_back_addr":"172.21.15.190:6819/537849891","heartbeat_front_addr":"172.21.15.190:6815/537849891","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.889784+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.974002+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.973549+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.955396+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.928471+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.902418+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.958696+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.926474+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T06:36:11.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:11.545+0000 7fda47f32640 1 -- 172.21.15.3:0/2774945163 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fda2406be60 msgr2=0x7fda2406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:11.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:11.545+0000 7fda47f32640 1 --2- 172.21.15.3:0/2774945163 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fda2406be60 0x7fda2406e320 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fda3c00aac0 tx=0x7fda3c008040 comp rx=0 tx=0).stop 2024-05-09T06:36:11.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:11.545+0000 7fda47f32640 1 -- 172.21.15.3:0/2774945163 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fda4015e420 msgr2=0x7fda40160920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:11.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:11.545+0000 7fda47f32640 1 --2- 172.21.15.3:0/2774945163 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fda4015e420 0x7fda40160920 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fda3802abe0 tx=0x7fda38004a40 comp rx=0 tx=0).stop 2024-05-09T06:36:11.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:11.545+0000 7fda47f32640 1 -- 172.21.15.3:0/2774945163 shutdown_connections 2024-05-09T06:36:11.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:11.545+0000 7fda47f32640 1 --2- 172.21.15.3:0/2774945163 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fda2406be60 0x7fda2406e320 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:11.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:11.545+0000 7fda47f32640 1 --2- 172.21.15.3:0/2774945163 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fda4015e420 0x7fda40160920 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:11.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:11.545+0000 7fda47f32640 1 --2- 172.21.15.3:0/2774945163 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fda400b4d50 0x7fda401603e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:11.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:11.545+0000 7fda47f32640 1 --2- 172.21.15.3:0/2774945163 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fda4006b790 0x7fda40152100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:11.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:11.545+0000 7fda47f32640 1 -- 172.21.15.3:0/2774945163 >> 172.21.15.3:0/2774945163 conn(0x7fda4006e3d0 msgr2=0x7fda400b3580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:11.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:11.545+0000 7fda47f32640 1 -- 172.21.15.3:0/2774945163 shutdown_connections 2024-05-09T06:36:11.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:11.545+0000 7fda47f32640 1 -- 172.21.15.3:0/2774945163 wait complete. 2024-05-09T06:36:11.537 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-05-09T06:36:11.537 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-09T06:36:11.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:11.661+0000 7f7e63909640 1 Processor -- start 2024-05-09T06:36:11.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:11.661+0000 7f7e63909640 1 -- start start 2024-05-09T06:36:11.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:11.662+0000 7f7e63909640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f7e5c15bd90 0x7f7e5c06f530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:11.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:11.662+0000 7f7e63909640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7e5c070c10 con 0x7f7e5c15f300 2024-05-09T06:36:11.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:11.662+0000 7f7e63909640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7e5c070ef0 con 0x7f7e5c06fa70 2024-05-09T06:36:11.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:11.662+0000 7f7e63909640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f7e5c0719d0 con 0x7f7e5c15bd90 2024-05-09T06:36:11.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:11.662+0000 7f7e61ea6640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f7e5c15f300 0x7f7e5c06c870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:49190/0 (socket says 172.21.15.3:49190) 2024-05-09T06:36:11.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:11.662+0000 7f7e61ea6640 1 -- 172.21.15.3:0/2027766438 learned_addr learned my addr 172.21.15.3:0/2027766438 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-05-09T06:36:11.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:11.662+0000 7f7e616a5640 1 --2- 172.21.15.3:0/2027766438 >> v2:172.21.15.190:3300/0 conn(0x7f7e5c15bd90 0x7f7e5c06f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:11.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:11.662+0000 7f7e53fff640 1 -- 172.21.15.3:0/2027766438 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2984923890 0 0) 0x7f7e5c070c10 con 0x7f7e5c15f300 2024-05-09T06:36:11.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:11.662+0000 7f7e53fff640 1 -- 172.21.15.3:0/2027766438 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7e48003520 con 0x7f7e5c15f300 2024-05-09T06:36:11.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:11.662+0000 7f7e53fff640 1 -- 172.21.15.3:0/2027766438 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2440908599 0 0) 0x7f7e48003520 con 0x7f7e5c15f300 2024-05-09T06:36:11.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:11.662+0000 7f7e53fff640 1 -- 172.21.15.3:0/2027766438 >> v2:172.21.15.190:3300/0 conn(0x7f7e5c15bd90 msgr2=0x7f7e5c06f530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:11.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:11.662+0000 7f7e53fff640 1 --2- 172.21.15.3:0/2027766438 >> v2:172.21.15.190:3300/0 conn(0x7f7e5c15bd90 0x7f7e5c06f530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:11.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:11.662+0000 7f7e53fff640 1 -- 172.21.15.3:0/2027766438 >> v1:172.21.15.190:6789/0 conn(0x7f7e5c06fa70 legacy=0x7f7e5c15ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:11.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:11.662+0000 7f7e53fff640 1 -- 172.21.15.3:0/2027766438 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e5c0711d0 con 0x7f7e5c15f300 2024-05-09T06:36:11.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:11.663+0000 7f7e53fff640 1 -- 172.21.15.3:0/2027766438 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 563312031 0 0) 0x7f7e540019f0 con 0x7f7e5c15f300 2024-05-09T06:36:11.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:11.663+0000 7f7e53fff640 1 -- 172.21.15.3:0/2027766438 >> v1:172.21.15.3:6789/0 conn(0x7f7e5c15f300 legacy=0x7f7e5c06c870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:11.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:11.663+0000 7f7e53fff640 1 --2- 172.21.15.3:0/2027766438 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7e480039b0 0x7f7e48003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:11.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:11.663+0000 7f7e53fff640 1 --2- 172.21.15.3:0/2027766438 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7e48004800 0x7f7e48004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:11.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:11.663+0000 7f7e53fff640 1 --2- 172.21.15.3:0/2027766438 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7e480051c0 0x7f7e4800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:11.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:11.663+0000 7f7e53fff640 1 -- 172.21.15.3:0/2027766438 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f7e4800c130 con 0x7f7e480051c0 2024-05-09T06:36:11.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:11.663+0000 7f7e53fff640 1 -- 172.21.15.3:0/2027766438 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f7e4800c2e0 con 0x7f7e48004800 2024-05-09T06:36:11.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:11.663+0000 7f7e53fff640 1 -- 172.21.15.3:0/2027766438 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f7e4800c4e0 con 0x7f7e480039b0 2024-05-09T06:36:11.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:11.663+0000 7f7e61ea6640 1 --2- 172.21.15.3:0/2027766438 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7e480051c0 0x7f7e4800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:11.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:11.663+0000 7f7e60ea4640 1 --2- 172.21.15.3:0/2027766438 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7e48004800 0x7f7e48004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:11.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:11.663+0000 7f7e616a5640 1 --2- 172.21.15.3:0/2027766438 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7e480039b0 0x7f7e48003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:11.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:11.663+0000 7f7e61ea6640 1 -- 172.21.15.3:0/2027766438 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7e480039b0 msgr2=0x7f7e48003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:11.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:11.663+0000 7f7e61ea6640 1 --2- 172.21.15.3:0/2027766438 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7e480039b0 0x7f7e48003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:11.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:11.663+0000 7f7e61ea6640 1 -- 172.21.15.3:0/2027766438 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7e48004800 msgr2=0x7f7e48004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:11.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:11.663+0000 7f7e61ea6640 1 --2- 172.21.15.3:0/2027766438 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7e48004800 0x7f7e48004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:11.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:11.663+0000 7f7e61ea6640 1 -- 172.21.15.3:0/2027766438 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e48003520 con 0x7f7e480051c0 2024-05-09T06:36:11.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:11.664+0000 7f7e61ea6640 1 -- 172.21.15.3:0/2027766438 reap_dead start 2024-05-09T06:36:11.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:11.664+0000 7f7e61ea6640 1 --2- 172.21.15.3:0/2027766438 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7e480051c0 0x7f7e4800b9c0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f7e5400a860 tx=0x7f7e48005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4afb40913eac306c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:11.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:11.664+0000 7f7e53fff640 1 -- 172.21.15.3:0/2027766438 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e5400a480 con 0x7f7e480051c0 2024-05-09T06:36:11.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:11.664+0000 7f7e53fff640 1 -- 172.21.15.3:0/2027766438 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e5402f070 con 0x7f7e480051c0 2024-05-09T06:36:11.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:11.664+0000 7f7e53fff640 1 -- 172.21.15.3:0/2027766438 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e5402bcc0 con 0x7f7e480051c0 2024-05-09T06:36:11.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:11.664+0000 7f7e63909640 1 -- 172.21.15.3:0/2027766438 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7e480051c0 msgr2=0x7f7e4800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:11.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:11.664+0000 7f7e63909640 1 --2- 172.21.15.3:0/2027766438 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7e480051c0 0x7f7e4800b9c0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f7e5400a860 tx=0x7f7e48005830 comp rx=0 tx=0).stop 2024-05-09T06:36:11.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:11.664+0000 7f7e63909640 1 -- 172.21.15.3:0/2027766438 shutdown_connections 2024-05-09T06:36:11.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:11.664+0000 7f7e63909640 1 --2- 172.21.15.3:0/2027766438 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7e480051c0 0x7f7e4800b9c0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:11.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:11.664+0000 7f7e63909640 1 -- 172.21.15.3:0/2027766438 >> 172.21.15.3:0/2027766438 conn(0x7f7e5c0d1f70 msgr2=0x7f7e5c0d2350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:11.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:11.665+0000 7f7e63909640 1 -- 172.21.15.3:0/2027766438 shutdown_connections 2024-05-09T06:36:11.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:11.665+0000 7f7e63909640 1 -- 172.21.15.3:0/2027766438 wait complete. 2024-05-09T06:36:11.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:11.666+0000 7f7e63909640 1 Processor -- start 2024-05-09T06:36:11.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:11.666+0000 7f7e63909640 1 -- start start 2024-05-09T06:36:11.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:11.666+0000 7f7e63909640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7e480051c0 0x7f7e5c14acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:11.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:11.666+0000 7f7e63909640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7e5c14b590 0x7f7e5c156650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:11.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:11.666+0000 7f7e63909640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7e5c156b90 0x7f7e5c0b3d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:11.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:11.667+0000 7f7e63909640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f7e5c15c170 con 0x7f7e480051c0 2024-05-09T06:36:11.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:11.667+0000 7f7e63909640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f7e5c15f730 con 0x7f7e5c14b590 2024-05-09T06:36:11.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:11.667+0000 7f7e63909640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f7e5c15c2f0 con 0x7f7e5c156b90 2024-05-09T06:36:11.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:11.667+0000 7f7e616a5640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7e480051c0 0x7f7e5c14acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:11.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:11.667+0000 7f7e616a5640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7e480051c0 0x7f7e5c14acb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49394/0 (socket says 172.21.15.3:49394) 2024-05-09T06:36:11.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:11.667+0000 7f7e616a5640 1 -- 172.21.15.3:0/879622886 learned_addr learned my addr 172.21.15.3:0/879622886 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:36:11.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:11.667+0000 7f7e60ea4640 1 --2- 172.21.15.3:0/879622886 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7e5c14b590 0x7f7e5c156650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:11.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:11.667+0000 7f7e61ea6640 1 --2- 172.21.15.3:0/879622886 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7e5c156b90 0x7f7e5c0b3d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:11.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:11.667+0000 7f7e616a5640 1 -- 172.21.15.3:0/879622886 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7e5c156b90 msgr2=0x7f7e5c0b3d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:11.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:11.667+0000 7f7e616a5640 1 --2- 172.21.15.3:0/879622886 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7e5c156b90 0x7f7e5c0b3d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:11.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:11.667+0000 7f7e616a5640 1 -- 172.21.15.3:0/879622886 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7e5c14b590 msgr2=0x7f7e5c156650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:11.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:11.667+0000 7f7e616a5640 1 --2- 172.21.15.3:0/879622886 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7e5c14b590 0x7f7e5c156650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:11.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:11.667+0000 7f7e616a5640 1 -- 172.21.15.3:0/879622886 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e5c0cc920 con 0x7f7e480051c0 2024-05-09T06:36:11.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:11.667+0000 7f7e616a5640 1 --2- 172.21.15.3:0/879622886 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7e480051c0 0x7f7e5c14acb0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f7e580118e0 tx=0x7f7e58011da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:11.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:11.667+0000 7f7e51ffb640 1 -- 172.21.15.3:0/879622886 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e5800f9d0 con 0x7f7e480051c0 2024-05-09T06:36:11.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:11.667+0000 7f7e51ffb640 1 -- 172.21.15.3:0/879622886 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e5800fb70 con 0x7f7e480051c0 2024-05-09T06:36:11.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:11.667+0000 7f7e51ffb640 1 -- 172.21.15.3:0/879622886 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e58014440 con 0x7f7e480051c0 2024-05-09T06:36:11.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:11.667+0000 7f7e63909640 1 -- 172.21.15.3:0/879622886 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e5c0ccc00 con 0x7f7e480051c0 2024-05-09T06:36:11.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:11.667+0000 7f7e63909640 1 -- 172.21.15.3:0/879622886 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e5c0cd1c0 con 0x7f7e480051c0 2024-05-09T06:36:11.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:11.668+0000 7f7e63909640 1 -- 172.21.15.3:0/879622886 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e5c04f660 con 0x7f7e480051c0 2024-05-09T06:36:11.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:11.669+0000 7f7e51ffb640 1 -- 172.21.15.3:0/879622886 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7e58012040 con 0x7f7e480051c0 2024-05-09T06:36:11.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:11.669+0000 7f7e51ffb640 1 --2- 172.21.15.3:0/879622886 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f7e4006be30 0x7f7e4006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:11.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:11.669+0000 7f7e51ffb640 1 -- 172.21.15.3:0/879622886 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f7e5808b300 con 0x7f7e480051c0 2024-05-09T06:36:11.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:11.669+0000 7f7e60ea4640 1 --2- 172.21.15.3:0/879622886 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f7e4006be30 0x7f7e4006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:11.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:11.670+0000 7f7e60ea4640 1 --2- 172.21.15.3:0/879622886 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f7e4006be30 0x7f7e4006e2f0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f7e4c000c00 tx=0x7f7e4c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:11.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:11.674+0000 7f7e51ffb640 1 -- 172.21.15.3:0/879622886 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f7e58019030 con 0x7f7e480051c0 2024-05-09T06:36:11.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:11.852+0000 7f7e63909640 1 -- 172.21.15.3:0/879622886 --> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f7e5c0702e0 con 0x7f7e4006be30 2024-05-09T06:36:11.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:11.859+0000 7f7e51ffb640 1 -- 172.21.15.3:0/879622886 <== mgr.4105 v2:172.21.15.190:6832/975893321 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251882 (secure 0 0 0) 0x7f7e5c0702e0 con 0x7f7e4006be30 2024-05-09T06:36:11.843 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-09T06:36:11.845 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2024-05-09T06:36:11.846 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:11.862+0000 7f7e63909640 1 -- 172.21.15.3:0/879622886 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f7e4006be30 msgr2=0x7f7e4006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:11.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:11.862+0000 7f7e63909640 1 --2- 172.21.15.3:0/879622886 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f7e4006be30 0x7f7e4006e2f0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f7e4c000c00 tx=0x7f7e4c009040 comp rx=0 tx=0).stop 2024-05-09T06:36:11.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:11.862+0000 7f7e63909640 1 -- 172.21.15.3:0/879622886 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7e480051c0 msgr2=0x7f7e5c14acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:11.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:11.862+0000 7f7e63909640 1 --2- 172.21.15.3:0/879622886 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7e480051c0 0x7f7e5c14acb0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f7e580118e0 tx=0x7f7e58011da0 comp rx=0 tx=0).stop 2024-05-09T06:36:11.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:11.862+0000 7f7e63909640 1 -- 172.21.15.3:0/879622886 shutdown_connections 2024-05-09T06:36:11.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:11.862+0000 7f7e63909640 1 --2- 172.21.15.3:0/879622886 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f7e4006be30 0x7f7e4006e2f0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:11.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:11.862+0000 7f7e63909640 1 --2- 172.21.15.3:0/879622886 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7e5c156b90 0x7f7e5c0b3d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:11.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:11.862+0000 7f7e63909640 1 --2- 172.21.15.3:0/879622886 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7e5c14b590 0x7f7e5c156650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:11.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:11.862+0000 7f7e63909640 1 --2- 172.21.15.3:0/879622886 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7e480051c0 0x7f7e5c14acb0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:11.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:11.862+0000 7f7e63909640 1 -- 172.21.15.3:0/879622886 >> 172.21.15.3:0/879622886 conn(0x7f7e5c0d1f70 msgr2=0x7f7e5c0b27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:11.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:11.862+0000 7f7e63909640 1 -- 172.21.15.3:0/879622886 shutdown_connections 2024-05-09T06:36:11.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:11.862+0000 7f7e63909640 1 -- 172.21.15.3:0/879622886 wait complete. 2024-05-09T06:36:11.854 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2024-05-09T06:36:10.776481+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":135,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218892,"kb_used_data":2728,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755828,"statfs":{"total":773094113280,"available":772869967872,"internally_reserved":0,"allocated":2793472,"data_stored":1702628,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.937739"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T06:36:07.874676+0000","last_change":"2024-05-09T06:36:07.870322+0000","last_active":"2024-05-09T06:36:07.874676+0000","last_peered":"2024-05-09T06:36:07.874676+0000","last_clean":"2024-05-09T06:36:07.874676+0000","last_became_active":"2024-05-09T06:36:07.870000+0000","last_became_peered":"2024-05-09T06:36:07.870000+0000","last_unstale":"2024-05-09T06:36:07.874676+0000","last_undegraded":"2024-05-09T06:36:07.874676+0000","last_fullsized":"2024-05-09T06:36:07.874676+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:24:51.534066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T06:36:07.872110+0000","last_change":"2024-05-09T06:36:07.865401+0000","last_active":"2024-05-09T06:36:07.872110+0000","last_peered":"2024-05-09T06:36:07.872110+0000","last_clean":"2024-05-09T06:36:07.872110+0000","last_became_active":"2024-05-09T06:36:07.865119+0000","last_became_peered":"2024-05-09T06:36:07.865119+0000","last_unstale":"2024-05-09T06:36:07.872110+0000","last_undegraded":"2024-05-09T06:36:07.872110+0000","last_fullsized":"2024-05-09T06:36:07.872110+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:39:07.691670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T06:36:07.874890+0000","last_change":"2024-05-09T06:36:07.865504+0000","last_active":"2024-05-09T06:36:07.874890+0000","last_peered":"2024-05-09T06:36:07.874890+0000","last_clean":"2024-05-09T06:36:07.874890+0000","last_became_active":"2024-05-09T06:36:07.864109+0000","last_became_peered":"2024-05-09T06:36:07.864109+0000","last_unstale":"2024-05-09T06:36:07.874890+0000","last_undegraded":"2024-05-09T06:36:07.874890+0000","last_fullsized":"2024-05-09T06:36:07.874890+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:23:14.982108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T06:36:07.881369+0000","last_change":"2024-05-09T06:36:07.879922+0000","last_active":"2024-05-09T06:36:07.881369+0000","last_peered":"2024-05-09T06:36:07.881369+0000","last_clean":"2024-05-09T06:36:07.881369+0000","last_became_active":"2024-05-09T06:36:07.879559+0000","last_became_peered":"2024-05-09T06:36:07.879559+0000","last_unstale":"2024-05-09T06:36:07.881369+0000","last_undegraded":"2024-05-09T06:36:07.881369+0000","last_fullsized":"2024-05-09T06:36:07.881369+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:44:00.455366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T06:36:07.869262+0000","last_change":"2024-05-09T06:36:07.867057+0000","last_active":"2024-05-09T06:36:07.869262+0000","last_peered":"2024-05-09T06:36:07.869262+0000","last_clean":"2024-05-09T06:36:07.869262+0000","last_became_active":"2024-05-09T06:36:07.866657+0000","last_became_peered":"2024-05-09T06:36:07.866657+0000","last_unstale":"2024-05-09T06:36:07.869262+0000","last_undegraded":"2024-05-09T06:36:07.869262+0000","last_fullsized":"2024-05-09T06:36:07.869262+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:39:07.691670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T06:36:07.884464+0000","last_change":"2024-05-09T06:36:07.883141+0000","last_active":"2024-05-09T06:36:07.884464+0000","last_peered":"2024-05-09T06:36:07.884464+0000","last_clean":"2024-05-09T06:36:07.884464+0000","last_became_active":"2024-05-09T06:36:07.882826+0000","last_became_peered":"2024-05-09T06:36:07.882826+0000","last_unstale":"2024-05-09T06:36:07.884464+0000","last_undegraded":"2024-05-09T06:36:07.884464+0000","last_fullsized":"2024-05-09T06:36:07.884464+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:50:02.245694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T06:36:07.882132+0000","last_change":"2024-05-09T06:36:07.876998+0000","last_active":"2024-05-09T06:36:07.882132+0000","last_peered":"2024-05-09T06:36:07.882132+0000","last_clean":"2024-05-09T06:36:07.882132+0000","last_became_active":"2024-05-09T06:36:07.876672+0000","last_became_peered":"2024-05-09T06:36:07.876672+0000","last_unstale":"2024-05-09T06:36:07.882132+0000","last_undegraded":"2024-05-09T06:36:07.882132+0000","last_fullsized":"2024-05-09T06:36:07.882132+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:50:02.245694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T06:36:07.882731+0000","last_change":"2024-05-09T06:36:07.878020+0000","last_active":"2024-05-09T06:36:07.882731+0000","last_peered":"2024-05-09T06:36:07.882731+0000","last_clean":"2024-05-09T06:36:07.882731+0000","last_became_active":"2024-05-09T06:36:07.877767+0000","last_became_peered":"2024-05-09T06:36:07.877767+0000","last_unstale":"2024-05-09T06:36:07.882731+0000","last_undegraded":"2024-05-09T06:36:07.882731+0000","last_fullsized":"2024-05-09T06:36:07.882731+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:41:33.459622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T06:36:07.882156+0000","last_change":"2024-05-09T06:36:07.877214+0000","last_active":"2024-05-09T06:36:07.882156+0000","last_peered":"2024-05-09T06:36:07.882156+0000","last_clean":"2024-05-09T06:36:07.882156+0000","last_became_active":"2024-05-09T06:36:07.876924+0000","last_became_peered":"2024-05-09T06:36:07.876924+0000","last_unstale":"2024-05-09T06:36:07.882156+0000","last_undegraded":"2024-05-09T06:36:07.882156+0000","last_fullsized":"2024-05-09T06:36:07.882156+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:51:24.938206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T06:36:07.882483+0000","last_change":"2024-05-09T06:36:07.879229+0000","last_active":"2024-05-09T06:36:07.882483+0000","last_peered":"2024-05-09T06:36:07.882483+0000","last_clean":"2024-05-09T06:36:07.882483+0000","last_became_active":"2024-05-09T06:36:07.879000+0000","last_became_peered":"2024-05-09T06:36:07.879000+0000","last_unstale":"2024-05-09T06:36:07.882483+0000","last_undegraded":"2024-05-09T06:36:07.882483+0000","last_fullsized":"2024-05-09T06:36:07.882483+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:41:33.459622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T06:36:07.882187+0000","last_change":"2024-05-09T06:36:07.878671+0000","last_active":"2024-05-09T06:36:07.882187+0000","last_peered":"2024-05-09T06:36:07.882187+0000","last_clean":"2024-05-09T06:36:07.882187+0000","last_became_active":"2024-05-09T06:36:07.878389+0000","last_became_peered":"2024-05-09T06:36:07.878389+0000","last_unstale":"2024-05-09T06:36:07.882187+0000","last_undegraded":"2024-05-09T06:36:07.882187+0000","last_fullsized":"2024-05-09T06:36:07.882187+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:47:56.779394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T06:36:07.868118+0000","last_change":"2024-05-09T06:36:07.859955+0000","last_active":"2024-05-09T06:36:07.868118+0000","last_peered":"2024-05-09T06:36:07.868118+0000","last_clean":"2024-05-09T06:36:07.868118+0000","last_became_active":"2024-05-09T06:36:07.859649+0000","last_became_peered":"2024-05-09T06:36:07.859649+0000","last_unstale":"2024-05-09T06:36:07.868118+0000","last_undegraded":"2024-05-09T06:36:07.868118+0000","last_fullsized":"2024-05-09T06:36:07.868118+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:05:59.095770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T06:36:07.877186+0000","last_change":"2024-05-09T06:36:07.870408+0000","last_active":"2024-05-09T06:36:07.877186+0000","last_peered":"2024-05-09T06:36:07.877186+0000","last_clean":"2024-05-09T06:36:07.877186+0000","last_became_active":"2024-05-09T06:36:07.868679+0000","last_became_peered":"2024-05-09T06:36:07.868679+0000","last_unstale":"2024-05-09T06:36:07.877186+0000","last_undegraded":"2024-05-09T06:36:07.877186+0000","last_fullsized":"2024-05-09T06:36:07.877186+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:16:35.564277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T06:36:07.882449+0000","last_change":"2024-05-09T06:36:07.872146+0000","last_active":"2024-05-09T06:36:07.882449+0000","last_peered":"2024-05-09T06:36:07.882449+0000","last_clean":"2024-05-09T06:36:07.882449+0000","last_became_active":"2024-05-09T06:36:07.871858+0000","last_became_peered":"2024-05-09T06:36:07.871858+0000","last_unstale":"2024-05-09T06:36:07.882449+0000","last_undegraded":"2024-05-09T06:36:07.882449+0000","last_fullsized":"2024-05-09T06:36:07.882449+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:15:50.253604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T06:36:07.871548+0000","last_change":"2024-05-09T06:36:07.864771+0000","last_active":"2024-05-09T06:36:07.871548+0000","last_peered":"2024-05-09T06:36:07.871548+0000","last_clean":"2024-05-09T06:36:07.871548+0000","last_became_active":"2024-05-09T06:36:07.864352+0000","last_became_peered":"2024-05-09T06:36:07.864352+0000","last_unstale":"2024-05-09T06:36:07.871548+0000","last_undegraded":"2024-05-09T06:36:07.871548+0000","last_fullsized":"2024-05-09T06:36:07.871548+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:59:57.900963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T06:36:07.884449+0000","last_change":"2024-05-09T06:36:07.883189+0000","last_active":"2024-05-09T06:36:07.884449+0000","last_peered":"2024-05-09T06:36:07.884449+0000","last_clean":"2024-05-09T06:36:07.884449+0000","last_became_active":"2024-05-09T06:36:07.882906+0000","last_became_peered":"2024-05-09T06:36:07.882906+0000","last_unstale":"2024-05-09T06:36:07.884449+0000","last_undegraded":"2024-05-09T06:36:07.884449+0000","last_fullsized":"2024-05-09T06:36:07.884449+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:51:24.938206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T06:36:07.881323+0000","last_change":"2024-05-09T06:36:07.878966+0000","last_active":"2024-05-09T06:36:07.881323+0000","last_peered":"2024-05-09T06:36:07.881323+0000","last_clean":"2024-05-09T06:36:07.881323+0000","last_became_active":"2024-05-09T06:36:07.878678+0000","last_became_peered":"2024-05-09T06:36:07.878678+0000","last_unstale":"2024-05-09T06:36:07.881323+0000","last_undegraded":"2024-05-09T06:36:07.881323+0000","last_fullsized":"2024-05-09T06:36:07.881323+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:53:50.215704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T06:36:07.868140+0000","last_change":"2024-05-09T06:36:07.866493+0000","last_active":"2024-05-09T06:36:07.868140+0000","last_peered":"2024-05-09T06:36:07.868140+0000","last_clean":"2024-05-09T06:36:07.868140+0000","last_became_active":"2024-05-09T06:36:07.866157+0000","last_became_peered":"2024-05-09T06:36:07.866157+0000","last_unstale":"2024-05-09T06:36:07.868140+0000","last_undegraded":"2024-05-09T06:36:07.868140+0000","last_fullsized":"2024-05-09T06:36:07.868140+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:40:42.090918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T06:36:07.869466+0000","last_change":"2024-05-09T06:36:07.866352+0000","last_active":"2024-05-09T06:36:07.869466+0000","last_peered":"2024-05-09T06:36:07.869466+0000","last_clean":"2024-05-09T06:36:07.869466+0000","last_became_active":"2024-05-09T06:36:07.866066+0000","last_became_peered":"2024-05-09T06:36:07.866066+0000","last_unstale":"2024-05-09T06:36:07.869466+0000","last_undegraded":"2024-05-09T06:36:07.869466+0000","last_fullsized":"2024-05-09T06:36:07.869466+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:59:57.900963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T06:36:07.865622+0000","last_change":"2024-05-09T06:36:07.858555+0000","last_active":"2024-05-09T06:36:07.865622+0000","last_peered":"2024-05-09T06:36:07.865622+0000","last_clean":"2024-05-09T06:36:07.865622+0000","last_became_active":"2024-05-09T06:36:07.858259+0000","last_became_peered":"2024-05-09T06:36:07.858259+0000","last_unstale":"2024-05-09T06:36:07.865622+0000","last_undegraded":"2024-05-09T06:36:07.865622+0000","last_fullsized":"2024-05-09T06:36:07.865622+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:16:35.564277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T06:36:07.883670+0000","last_change":"2024-05-09T06:36:07.875125+0000","last_active":"2024-05-09T06:36:07.883670+0000","last_peered":"2024-05-09T06:36:07.883670+0000","last_clean":"2024-05-09T06:36:07.883670+0000","last_became_active":"2024-05-09T06:36:07.874432+0000","last_became_peered":"2024-05-09T06:36:07.874432+0000","last_unstale":"2024-05-09T06:36:07.883670+0000","last_undegraded":"2024-05-09T06:36:07.883670+0000","last_fullsized":"2024-05-09T06:36:07.883670+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:16:35.564277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T06:36:07.881068+0000","last_change":"2024-05-09T06:36:07.876701+0000","last_active":"2024-05-09T06:36:07.881068+0000","last_peered":"2024-05-09T06:36:07.881068+0000","last_clean":"2024-05-09T06:36:07.881068+0000","last_became_active":"2024-05-09T06:36:07.876416+0000","last_became_peered":"2024-05-09T06:36:07.876416+0000","last_unstale":"2024-05-09T06:36:07.881068+0000","last_undegraded":"2024-05-09T06:36:07.881068+0000","last_fullsized":"2024-05-09T06:36:07.881068+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:47:56.779394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T06:36:07.875346+0000","last_change":"2024-05-09T06:36:07.868404+0000","last_active":"2024-05-09T06:36:07.875346+0000","last_peered":"2024-05-09T06:36:07.875346+0000","last_clean":"2024-05-09T06:36:07.875346+0000","last_became_active":"2024-05-09T06:36:07.867909+0000","last_became_peered":"2024-05-09T06:36:07.867909+0000","last_unstale":"2024-05-09T06:36:07.875346+0000","last_undegraded":"2024-05-09T06:36:07.875346+0000","last_fullsized":"2024-05-09T06:36:07.875346+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:40:42.090918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T06:36:07.881396+0000","last_change":"2024-05-09T06:36:07.880411+0000","last_active":"2024-05-09T06:36:07.881396+0000","last_peered":"2024-05-09T06:36:07.881396+0000","last_clean":"2024-05-09T06:36:07.881396+0000","last_became_active":"2024-05-09T06:36:07.879887+0000","last_became_peered":"2024-05-09T06:36:07.879887+0000","last_unstale":"2024-05-09T06:36:07.881396+0000","last_undegraded":"2024-05-09T06:36:07.881396+0000","last_fullsized":"2024-05-09T06:36:07.881396+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:34:13.971998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T06:36:07.875405+0000","last_change":"2024-05-09T06:36:07.868320+0000","last_active":"2024-05-09T06:36:07.875405+0000","last_peered":"2024-05-09T06:36:07.875405+0000","last_clean":"2024-05-09T06:36:07.875405+0000","last_became_active":"2024-05-09T06:36:07.867969+0000","last_became_peered":"2024-05-09T06:36:07.867969+0000","last_unstale":"2024-05-09T06:36:07.875405+0000","last_undegraded":"2024-05-09T06:36:07.875405+0000","last_fullsized":"2024-05-09T06:36:07.875405+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:15:50.253604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T06:36:07.873521+0000","last_change":"2024-05-09T06:36:07.869956+0000","last_active":"2024-05-09T06:36:07.873521+0000","last_peered":"2024-05-09T06:36:07.873521+0000","last_clean":"2024-05-09T06:36:07.873521+0000","last_became_active":"2024-05-09T06:36:07.869695+0000","last_became_peered":"2024-05-09T06:36:07.869695+0000","last_unstale":"2024-05-09T06:36:07.873521+0000","last_undegraded":"2024-05-09T06:36:07.873521+0000","last_fullsized":"2024-05-09T06:36:07.873521+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:39:07.691670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T06:36:07.871772+0000","last_change":"2024-05-09T06:36:07.867102+0000","last_active":"2024-05-09T06:36:07.871772+0000","last_peered":"2024-05-09T06:36:07.871772+0000","last_clean":"2024-05-09T06:36:07.871772+0000","last_became_active":"2024-05-09T06:36:07.866785+0000","last_became_peered":"2024-05-09T06:36:07.866785+0000","last_unstale":"2024-05-09T06:36:07.871772+0000","last_undegraded":"2024-05-09T06:36:07.871772+0000","last_fullsized":"2024-05-09T06:36:07.871772+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:53:50.215704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T06:36:07.875611+0000","last_change":"2024-05-09T06:36:07.871788+0000","last_active":"2024-05-09T06:36:07.875611+0000","last_peered":"2024-05-09T06:36:07.875611+0000","last_clean":"2024-05-09T06:36:07.875611+0000","last_became_active":"2024-05-09T06:36:07.871500+0000","last_became_peered":"2024-05-09T06:36:07.871500+0000","last_unstale":"2024-05-09T06:36:07.875611+0000","last_undegraded":"2024-05-09T06:36:07.875611+0000","last_fullsized":"2024-05-09T06:36:07.875611+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:39:07.691670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T06:36:07.883742+0000","last_change":"2024-05-09T06:36:07.876623+0000","last_active":"2024-05-09T06:36:07.883742+0000","last_peered":"2024-05-09T06:36:07.883742+0000","last_clean":"2024-05-09T06:36:07.883742+0000","last_became_active":"2024-05-09T06:36:07.876358+0000","last_became_peered":"2024-05-09T06:36:07.876358+0000","last_unstale":"2024-05-09T06:36:07.883742+0000","last_undegraded":"2024-05-09T06:36:07.883742+0000","last_fullsized":"2024-05-09T06:36:07.883742+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:58:19.164284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T06:36:07.881325+0000","last_change":"2024-05-09T06:36:07.879149+0000","last_active":"2024-05-09T06:36:07.881325+0000","last_peered":"2024-05-09T06:36:07.881325+0000","last_clean":"2024-05-09T06:36:07.881325+0000","last_became_active":"2024-05-09T06:36:07.878868+0000","last_became_peered":"2024-05-09T06:36:07.878868+0000","last_unstale":"2024-05-09T06:36:07.881325+0000","last_undegraded":"2024-05-09T06:36:07.881325+0000","last_fullsized":"2024-05-09T06:36:07.881325+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:24:51.534066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T06:36:07.879373+0000","last_change":"2024-05-09T06:36:07.876443+0000","last_active":"2024-05-09T06:36:07.879373+0000","last_peered":"2024-05-09T06:36:07.879373+0000","last_clean":"2024-05-09T06:36:07.879373+0000","last_became_active":"2024-05-09T06:36:07.876122+0000","last_became_peered":"2024-05-09T06:36:07.876122+0000","last_unstale":"2024-05-09T06:36:07.879373+0000","last_undegraded":"2024-05-09T06:36:07.879373+0000","last_fullsized":"2024-05-09T06:36:07.879373+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:59:30.741519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T06:36:07.870539+0000","last_change":"2024-05-09T06:36:07.867899+0000","last_active":"2024-05-09T06:36:07.870539+0000","last_peered":"2024-05-09T06:36:07.870539+0000","last_clean":"2024-05-09T06:36:07.870539+0000","last_became_active":"2024-05-09T06:36:07.867371+0000","last_became_peered":"2024-05-09T06:36:07.867371+0000","last_unstale":"2024-05-09T06:36:07.870539+0000","last_undegraded":"2024-05-09T06:36:07.870539+0000","last_fullsized":"2024-05-09T06:36:07.870539+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:03:39.564328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T06:36:07.874468+0000","last_change":"2024-05-09T06:36:07.871355+0000","last_active":"2024-05-09T06:36:07.874468+0000","last_peered":"2024-05-09T06:36:07.874468+0000","last_clean":"2024-05-09T06:36:07.874468+0000","last_became_active":"2024-05-09T06:36:07.871025+0000","last_became_peered":"2024-05-09T06:36:07.871025+0000","last_unstale":"2024-05-09T06:36:07.874468+0000","last_undegraded":"2024-05-09T06:36:07.874468+0000","last_fullsized":"2024-05-09T06:36:07.874468+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:15:50.253604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T06:36:07.871608+0000","last_change":"2024-05-09T06:36:06.865248+0000","last_active":"2024-05-09T06:36:07.871608+0000","last_peered":"2024-05-09T06:36:07.871608+0000","last_clean":"2024-05-09T06:36:07.871608+0000","last_became_active":"2024-05-09T06:36:06.864940+0000","last_became_peered":"2024-05-09T06:36:06.864940+0000","last_unstale":"2024-05-09T06:36:07.871608+0000","last_undegraded":"2024-05-09T06:36:07.871608+0000","last_fullsized":"2024-05-09T06:36:07.871608+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_clean_scrub_stamp":"2024-05-09T06:36:05.833229+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:08:20.004456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T06:36:07.882647+0000","last_change":"2024-05-09T06:36:07.874223+0000","last_active":"2024-05-09T06:36:07.882647+0000","last_peered":"2024-05-09T06:36:07.882647+0000","last_clean":"2024-05-09T06:36:07.882647+0000","last_became_active":"2024-05-09T06:36:07.873787+0000","last_became_peered":"2024-05-09T06:36:07.873787+0000","last_unstale":"2024-05-09T06:36:07.882647+0000","last_undegraded":"2024-05-09T06:36:07.882647+0000","last_fullsized":"2024-05-09T06:36:07.882647+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:59:30.741519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T06:36:07.868643+0000","last_change":"2024-05-09T06:36:06.862586+0000","last_active":"2024-05-09T06:36:07.868643+0000","last_peered":"2024-05-09T06:36:07.868643+0000","last_clean":"2024-05-09T06:36:07.868643+0000","last_became_active":"2024-05-09T06:36:06.861545+0000","last_became_peered":"2024-05-09T06:36:06.861545+0000","last_unstale":"2024-05-09T06:36:07.868643+0000","last_undegraded":"2024-05-09T06:36:07.868643+0000","last_fullsized":"2024-05-09T06:36:07.868643+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_clean_scrub_stamp":"2024-05-09T06:36:05.833229+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:15:49.248390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T06:36:07.859545+0000","last_change":"2024-05-09T06:36:06.867498+0000","last_active":"2024-05-09T06:36:07.859545+0000","last_peered":"2024-05-09T06:36:07.859545+0000","last_clean":"2024-05-09T06:36:07.859545+0000","last_became_active":"2024-05-09T06:36:06.867177+0000","last_became_peered":"2024-05-09T06:36:06.867177+0000","last_unstale":"2024-05-09T06:36:07.859545+0000","last_undegraded":"2024-05-09T06:36:07.859545+0000","last_fullsized":"2024-05-09T06:36:07.859545+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_clean_scrub_stamp":"2024-05-09T06:36:05.833229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:23:13.976894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T06:36:07.875735+0000","last_change":"2024-05-09T06:36:07.871392+0000","last_active":"2024-05-09T06:36:07.875735+0000","last_peered":"2024-05-09T06:36:07.875735+0000","last_clean":"2024-05-09T06:36:07.875735+0000","last_became_active":"2024-05-09T06:36:07.871103+0000","last_became_peered":"2024-05-09T06:36:07.871103+0000","last_unstale":"2024-05-09T06:36:07.875735+0000","last_undegraded":"2024-05-09T06:36:07.875735+0000","last_fullsized":"2024-05-09T06:36:07.875735+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:59:57.900963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T06:36:07.871825+0000","last_change":"2024-05-09T06:36:06.865012+0000","last_active":"2024-05-09T06:36:07.871825+0000","last_peered":"2024-05-09T06:36:07.871825+0000","last_clean":"2024-05-09T06:36:07.871825+0000","last_became_active":"2024-05-09T06:36:06.864434+0000","last_became_peered":"2024-05-09T06:36:06.864434+0000","last_unstale":"2024-05-09T06:36:07.871825+0000","last_undegraded":"2024-05-09T06:36:07.871825+0000","last_fullsized":"2024-05-09T06:36:07.871825+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_clean_scrub_stamp":"2024-05-09T06:36:05.833229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:16:34.559063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T06:36:07.879368+0000","last_change":"2024-05-09T06:36:07.870720+0000","last_active":"2024-05-09T06:36:07.879368+0000","last_peered":"2024-05-09T06:36:07.879368+0000","last_clean":"2024-05-09T06:36:07.879368+0000","last_became_active":"2024-05-09T06:36:07.870431+0000","last_became_peered":"2024-05-09T06:36:07.870431+0000","last_unstale":"2024-05-09T06:36:07.879368+0000","last_undegraded":"2024-05-09T06:36:07.879368+0000","last_fullsized":"2024-05-09T06:36:07.879368+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:24:51.534066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T06:36:07.857540+0000","last_change":"2024-05-09T06:36:06.866523+0000","last_active":"2024-05-09T06:36:07.857540+0000","last_peered":"2024-05-09T06:36:07.857540+0000","last_clean":"2024-05-09T06:36:07.857540+0000","last_became_active":"2024-05-09T06:36:06.866220+0000","last_became_peered":"2024-05-09T06:36:06.866220+0000","last_unstale":"2024-05-09T06:36:07.857540+0000","last_undegraded":"2024-05-09T06:36:07.857540+0000","last_fullsized":"2024-05-09T06:36:07.857540+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_clean_scrub_stamp":"2024-05-09T06:36:05.833229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:40:41.085704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T06:36:07.881032+0000","last_change":"2024-05-09T06:36:07.877537+0000","last_active":"2024-05-09T06:36:07.881032+0000","last_peered":"2024-05-09T06:36:07.881032+0000","last_clean":"2024-05-09T06:36:07.881032+0000","last_became_active":"2024-05-09T06:36:07.877261+0000","last_became_peered":"2024-05-09T06:36:07.877261+0000","last_unstale":"2024-05-09T06:36:07.881032+0000","last_undegraded":"2024-05-09T06:36:07.881032+0000","last_fullsized":"2024-05-09T06:36:07.881032+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:41:33.459622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T06:36:07.866215+0000","last_change":"2024-05-09T06:36:06.864863+0000","last_active":"2024-05-09T06:36:07.866215+0000","last_peered":"2024-05-09T06:36:07.866215+0000","last_clean":"2024-05-09T06:36:07.866215+0000","last_became_active":"2024-05-09T06:36:06.864300+0000","last_became_peered":"2024-05-09T06:36:06.864300+0000","last_unstale":"2024-05-09T06:36:07.866215+0000","last_undegraded":"2024-05-09T06:36:07.866215+0000","last_fullsized":"2024-05-09T06:36:07.866215+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_clean_scrub_stamp":"2024-05-09T06:36:05.833229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:15:49.248390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T06:36:07.883809+0000","last_change":"2024-05-09T06:36:07.877642+0000","last_active":"2024-05-09T06:36:07.883809+0000","last_peered":"2024-05-09T06:36:07.883809+0000","last_clean":"2024-05-09T06:36:07.883809+0000","last_became_active":"2024-05-09T06:36:07.877358+0000","last_became_peered":"2024-05-09T06:36:07.877358+0000","last_unstale":"2024-05-09T06:36:07.883809+0000","last_undegraded":"2024-05-09T06:36:07.883809+0000","last_fullsized":"2024-05-09T06:36:07.883809+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:03:39.564328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":15,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T06:36:07.879614+0000","last_change":"2024-05-09T06:36:06.865905+0000","last_active":"2024-05-09T06:36:07.879614+0000","last_peered":"2024-05-09T06:36:07.879614+0000","last_clean":"2024-05-09T06:36:07.879614+0000","last_became_active":"2024-05-09T06:36:06.865528+0000","last_became_peered":"2024-05-09T06:36:06.865528+0000","last_unstale":"2024-05-09T06:36:07.879614+0000","last_undegraded":"2024-05-09T06:36:07.879614+0000","last_fullsized":"2024-05-09T06:36:07.879614+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_clean_scrub_stamp":"2024-05-09T06:36:05.833229+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:40:41.085704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T06:36:07.868158+0000","last_change":"2024-05-09T06:36:07.864517+0000","last_active":"2024-05-09T06:36:07.868158+0000","last_peered":"2024-05-09T06:36:07.868158+0000","last_clean":"2024-05-09T06:36:07.868158+0000","last_became_active":"2024-05-09T06:36:07.864076+0000","last_became_peered":"2024-05-09T06:36:07.864076+0000","last_unstale":"2024-05-09T06:36:07.868158+0000","last_undegraded":"2024-05-09T06:36:07.868158+0000","last_fullsized":"2024-05-09T06:36:07.868158+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:08:21.009670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":18,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T06:36:07.887025+0000","last_change":"2024-05-09T06:36:06.862854+0000","last_active":"2024-05-09T06:36:07.887025+0000","last_peered":"2024-05-09T06:36:07.887025+0000","last_clean":"2024-05-09T06:36:07.887025+0000","last_became_active":"2024-05-09T06:36:06.862538+0000","last_became_peered":"2024-05-09T06:36:06.862538+0000","last_unstale":"2024-05-09T06:36:07.887025+0000","last_undegraded":"2024-05-09T06:36:07.887025+0000","last_fullsized":"2024-05-09T06:36:07.887025+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_clean_scrub_stamp":"2024-05-09T06:36:05.833229+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:08:20.004456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T06:36:07.873153+0000","last_change":"2024-05-09T06:36:07.872477+0000","last_active":"2024-05-09T06:36:07.873153+0000","last_peered":"2024-05-09T06:36:07.873153+0000","last_clean":"2024-05-09T06:36:07.873153+0000","last_became_active":"2024-05-09T06:36:07.872181+0000","last_became_peered":"2024-05-09T06:36:07.872181+0000","last_unstale":"2024-05-09T06:36:07.873153+0000","last_undegraded":"2024-05-09T06:36:07.873153+0000","last_fullsized":"2024-05-09T06:36:07.873153+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:44:00.455366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T06:36:07.874818+0000","last_change":"2024-05-09T06:36:06.861757+0000","last_active":"2024-05-09T06:36:07.874818+0000","last_peered":"2024-05-09T06:36:07.874818+0000","last_clean":"2024-05-09T06:36:07.874818+0000","last_became_active":"2024-05-09T06:36:06.861434+0000","last_became_peered":"2024-05-09T06:36:06.861434+0000","last_unstale":"2024-05-09T06:36:07.874818+0000","last_undegraded":"2024-05-09T06:36:07.874818+0000","last_fullsized":"2024-05-09T06:36:07.874818+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_clean_scrub_stamp":"2024-05-09T06:36:05.833229+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:56:44.324093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T06:36:07.882619+0000","last_change":"2024-05-09T06:36:07.873646+0000","last_active":"2024-05-09T06:36:07.882619+0000","last_peered":"2024-05-09T06:36:07.882619+0000","last_clean":"2024-05-09T06:36:07.882619+0000","last_became_active":"2024-05-09T06:36:07.872272+0000","last_became_peered":"2024-05-09T06:36:07.872272+0000","last_unstale":"2024-05-09T06:36:07.882619+0000","last_undegraded":"2024-05-09T06:36:07.882619+0000","last_fullsized":"2024-05-09T06:36:07.882619+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:40:42.090918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T06:36:07.871242+0000","last_change":"2024-05-09T06:36:06.865709+0000","last_active":"2024-05-09T06:36:07.871242+0000","last_peered":"2024-05-09T06:36:07.871242+0000","last_clean":"2024-05-09T06:36:07.871242+0000","last_became_active":"2024-05-09T06:36:06.864907+0000","last_became_peered":"2024-05-09T06:36:06.864907+0000","last_unstale":"2024-05-09T06:36:07.871242+0000","last_undegraded":"2024-05-09T06:36:07.871242+0000","last_fullsized":"2024-05-09T06:36:07.871242+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_clean_scrub_stamp":"2024-05-09T06:36:05.833229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:40:41.085704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T06:36:07.875323+0000","last_change":"2024-05-09T06:36:07.873795+0000","last_active":"2024-05-09T06:36:07.875323+0000","last_peered":"2024-05-09T06:36:07.875323+0000","last_clean":"2024-05-09T06:36:07.875323+0000","last_became_active":"2024-05-09T06:36:07.873491+0000","last_became_peered":"2024-05-09T06:36:07.873491+0000","last_unstale":"2024-05-09T06:36:07.875323+0000","last_undegraded":"2024-05-09T06:36:07.875323+0000","last_fullsized":"2024-05-09T06:36:07.875323+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:34:13.971998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T06:36:07.855945+0000","last_change":"2024-05-09T06:36:06.862067+0000","last_active":"2024-05-09T06:36:07.855945+0000","last_peered":"2024-05-09T06:36:07.855945+0000","last_clean":"2024-05-09T06:36:07.855945+0000","last_became_active":"2024-05-09T06:36:06.861791+0000","last_became_peered":"2024-05-09T06:36:06.861791+0000","last_unstale":"2024-05-09T06:36:07.855945+0000","last_undegraded":"2024-05-09T06:36:07.855945+0000","last_fullsized":"2024-05-09T06:36:07.855945+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_clean_scrub_stamp":"2024-05-09T06:36:05.833229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:03:17.189398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T06:36:07.870516+0000","last_change":"2024-05-09T06:36:07.867637+0000","last_active":"2024-05-09T06:36:07.870516+0000","last_peered":"2024-05-09T06:36:07.870516+0000","last_clean":"2024-05-09T06:36:07.870516+0000","last_became_active":"2024-05-09T06:36:07.867028+0000","last_became_peered":"2024-05-09T06:36:07.867028+0000","last_unstale":"2024-05-09T06:36:07.870516+0000","last_undegraded":"2024-05-09T06:36:07.870516+0000","last_fullsized":"2024-05-09T06:36:07.870516+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:58:19.164284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":20,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T06:36:07.884338+0000","last_change":"2024-05-09T06:36:06.852528+0000","last_active":"2024-05-09T06:36:07.884338+0000","last_peered":"2024-05-09T06:36:07.884338+0000","last_clean":"2024-05-09T06:36:07.884338+0000","last_became_active":"2024-05-09T06:36:06.852202+0000","last_became_peered":"2024-05-09T06:36:06.852202+0000","last_unstale":"2024-05-09T06:36:07.884338+0000","last_undegraded":"2024-05-09T06:36:07.884338+0000","last_fullsized":"2024-05-09T06:36:07.884338+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_clean_scrub_stamp":"2024-05-09T06:36:05.833229+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:08:20.004456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T06:36:07.869370+0000","last_change":"2024-05-09T06:36:07.867532+0000","last_active":"2024-05-09T06:36:07.869370+0000","last_peered":"2024-05-09T06:36:07.869370+0000","last_clean":"2024-05-09T06:36:07.869370+0000","last_became_active":"2024-05-09T06:36:07.867262+0000","last_became_peered":"2024-05-09T06:36:07.867262+0000","last_unstale":"2024-05-09T06:36:07.869370+0000","last_undegraded":"2024-05-09T06:36:07.869370+0000","last_fullsized":"2024-05-09T06:36:07.869370+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:47:56.779394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T06:36:07.866872+0000","last_change":"2024-05-09T06:36:06.863522+0000","last_active":"2024-05-09T06:36:07.866872+0000","last_peered":"2024-05-09T06:36:07.866872+0000","last_clean":"2024-05-09T06:36:07.866872+0000","last_became_active":"2024-05-09T06:36:06.863099+0000","last_became_peered":"2024-05-09T06:36:06.863099+0000","last_unstale":"2024-05-09T06:36:07.866872+0000","last_undegraded":"2024-05-09T06:36:07.866872+0000","last_fullsized":"2024-05-09T06:36:07.866872+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_clean_scrub_stamp":"2024-05-09T06:36:05.833229+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:56:44.324093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T06:36:07.869347+0000","last_change":"2024-05-09T06:36:07.868102+0000","last_active":"2024-05-09T06:36:07.869347+0000","last_peered":"2024-05-09T06:36:07.869347+0000","last_clean":"2024-05-09T06:36:07.869347+0000","last_became_active":"2024-05-09T06:36:07.867800+0000","last_became_peered":"2024-05-09T06:36:07.867800+0000","last_unstale":"2024-05-09T06:36:07.869347+0000","last_undegraded":"2024-05-09T06:36:07.869347+0000","last_fullsized":"2024-05-09T06:36:07.869347+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:41:33.459622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T06:36:07.883152+0000","last_change":"2024-05-09T06:36:06.861458+0000","last_active":"2024-05-09T06:36:07.883152+0000","last_peered":"2024-05-09T06:36:07.883152+0000","last_clean":"2024-05-09T06:36:07.883152+0000","last_became_active":"2024-05-09T06:36:06.861098+0000","last_became_peered":"2024-05-09T06:36:06.861098+0000","last_unstale":"2024-05-09T06:36:07.883152+0000","last_undegraded":"2024-05-09T06:36:07.883152+0000","last_fullsized":"2024-05-09T06:36:07.883152+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_clean_scrub_stamp":"2024-05-09T06:36:05.833229+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:15:49.248390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T06:36:07.874654+0000","last_change":"2024-05-09T06:36:07.868487+0000","last_active":"2024-05-09T06:36:07.874654+0000","last_peered":"2024-05-09T06:36:07.874654+0000","last_clean":"2024-05-09T06:36:07.874654+0000","last_became_active":"2024-05-09T06:36:07.867705+0000","last_became_peered":"2024-05-09T06:36:07.867705+0000","last_unstale":"2024-05-09T06:36:07.874654+0000","last_undegraded":"2024-05-09T06:36:07.874654+0000","last_fullsized":"2024-05-09T06:36:07.874654+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:53:50.215704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T06:36:07.879521+0000","last_change":"2024-05-09T06:36:06.858041+0000","last_active":"2024-05-09T06:36:07.879521+0000","last_peered":"2024-05-09T06:36:07.879521+0000","last_clean":"2024-05-09T06:36:07.879521+0000","last_became_active":"2024-05-09T06:36:06.857745+0000","last_became_peered":"2024-05-09T06:36:06.857745+0000","last_unstale":"2024-05-09T06:36:07.879521+0000","last_undegraded":"2024-05-09T06:36:07.879521+0000","last_fullsized":"2024-05-09T06:36:07.879521+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_clean_scrub_stamp":"2024-05-09T06:36:05.833229+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:58:18.159070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T06:36:07.880969+0000","last_change":"2024-05-09T06:36:07.876459+0000","last_active":"2024-05-09T06:36:07.880969+0000","last_peered":"2024-05-09T06:36:07.880969+0000","last_clean":"2024-05-09T06:36:07.880969+0000","last_became_active":"2024-05-09T06:36:07.876151+0000","last_became_peered":"2024-05-09T06:36:07.876151+0000","last_unstale":"2024-05-09T06:36:07.880969+0000","last_undegraded":"2024-05-09T06:36:07.880969+0000","last_fullsized":"2024-05-09T06:36:07.880969+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:39:07.691670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T06:36:07.856918+0000","last_change":"2024-05-09T06:36:06.866239+0000","last_active":"2024-05-09T06:36:07.856918+0000","last_peered":"2024-05-09T06:36:07.856918+0000","last_clean":"2024-05-09T06:36:07.856918+0000","last_became_active":"2024-05-09T06:36:06.865929+0000","last_became_peered":"2024-05-09T06:36:06.865929+0000","last_unstale":"2024-05-09T06:36:07.856918+0000","last_undegraded":"2024-05-09T06:36:07.856918+0000","last_fullsized":"2024-05-09T06:36:07.856918+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_clean_scrub_stamp":"2024-05-09T06:36:05.833229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:03:17.189398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T06:36:07.885088+0000","last_change":"2024-05-09T06:36:07.877578+0000","last_active":"2024-05-09T06:36:07.885088+0000","last_peered":"2024-05-09T06:36:07.885088+0000","last_clean":"2024-05-09T06:36:07.885088+0000","last_became_active":"2024-05-09T06:36:07.877319+0000","last_became_peered":"2024-05-09T06:36:07.877319+0000","last_unstale":"2024-05-09T06:36:07.885088+0000","last_undegraded":"2024-05-09T06:36:07.885088+0000","last_fullsized":"2024-05-09T06:36:07.885088+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:19:58.370458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":168,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T06:36:07.858464+0000","last_change":"2024-05-09T06:35:55.895214+0000","last_active":"2024-05-09T06:36:07.858464+0000","last_peered":"2024-05-09T06:36:07.858464+0000","last_clean":"2024-05-09T06:36:07.858464+0000","last_became_active":"2024-05-09T06:35:55.894087+0000","last_became_peered":"2024-05-09T06:35:55.894087+0000","last_unstale":"2024-05-09T06:36:07.858464+0000","last_undegraded":"2024-05-09T06:36:07.858464+0000","last_fullsized":"2024-05-09T06:36:07.858464+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:35:54.820102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:35:54.820102+0000","last_clean_scrub_stamp":"2024-05-09T06:35:54.820102+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:29:20.408099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T06:36:07.859959+0000","last_change":"2024-05-09T06:36:06.864900+0000","last_active":"2024-05-09T06:36:07.859959+0000","last_peered":"2024-05-09T06:36:07.859959+0000","last_clean":"2024-05-09T06:36:07.859959+0000","last_became_active":"2024-05-09T06:36:06.864169+0000","last_became_peered":"2024-05-09T06:36:06.864169+0000","last_unstale":"2024-05-09T06:36:07.859959+0000","last_undegraded":"2024-05-09T06:36:07.859959+0000","last_fullsized":"2024-05-09T06:36:07.859959+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_clean_scrub_stamp":"2024-05-09T06:36:05.833229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:03:17.189398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T06:36:07.886527+0000","last_change":"2024-05-09T06:36:07.880436+0000","last_active":"2024-05-09T06:36:07.886527+0000","last_peered":"2024-05-09T06:36:07.886527+0000","last_clean":"2024-05-09T06:36:07.886527+0000","last_became_active":"2024-05-09T06:36:07.879931+0000","last_became_peered":"2024-05-09T06:36:07.879931+0000","last_unstale":"2024-05-09T06:36:07.886527+0000","last_undegraded":"2024-05-09T06:36:07.886527+0000","last_fullsized":"2024-05-09T06:36:07.886527+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:53:50.215704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T06:36:07.858392+0000","last_change":"2024-05-09T06:36:06.860567+0000","last_active":"2024-05-09T06:36:07.858392+0000","last_peered":"2024-05-09T06:36:07.858392+0000","last_clean":"2024-05-09T06:36:07.858392+0000","last_became_active":"2024-05-09T06:36:06.860284+0000","last_became_peered":"2024-05-09T06:36:06.860284+0000","last_unstale":"2024-05-09T06:36:07.858392+0000","last_undegraded":"2024-05-09T06:36:07.858392+0000","last_fullsized":"2024-05-09T06:36:07.858392+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_clean_scrub_stamp":"2024-05-09T06:36:05.833229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:03:38.559114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T06:36:07.874795+0000","last_change":"2024-05-09T06:36:07.865957+0000","last_active":"2024-05-09T06:36:07.874795+0000","last_peered":"2024-05-09T06:36:07.874795+0000","last_clean":"2024-05-09T06:36:07.874795+0000","last_became_active":"2024-05-09T06:36:07.865683+0000","last_became_peered":"2024-05-09T06:36:07.865683+0000","last_unstale":"2024-05-09T06:36:07.874795+0000","last_undegraded":"2024-05-09T06:36:07.874795+0000","last_fullsized":"2024-05-09T06:36:07.874795+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:08:21.009670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T06:36:07.856303+0000","last_change":"2024-05-09T06:36:06.862232+0000","last_active":"2024-05-09T06:36:07.856303+0000","last_peered":"2024-05-09T06:36:07.856303+0000","last_clean":"2024-05-09T06:36:07.856303+0000","last_became_active":"2024-05-09T06:36:06.861918+0000","last_became_peered":"2024-05-09T06:36:06.861918+0000","last_unstale":"2024-05-09T06:36:07.856303+0000","last_undegraded":"2024-05-09T06:36:07.856303+0000","last_fullsized":"2024-05-09T06:36:07.856303+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_clean_scrub_stamp":"2024-05-09T06:36:05.833229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:23:13.976894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T06:36:07.871659+0000","last_change":"2024-05-09T06:36:07.865850+0000","last_active":"2024-05-09T06:36:07.871659+0000","last_peered":"2024-05-09T06:36:07.871659+0000","last_clean":"2024-05-09T06:36:07.871659+0000","last_became_active":"2024-05-09T06:36:07.865520+0000","last_became_peered":"2024-05-09T06:36:07.865520+0000","last_unstale":"2024-05-09T06:36:07.871659+0000","last_undegraded":"2024-05-09T06:36:07.871659+0000","last_fullsized":"2024-05-09T06:36:07.871659+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:41:33.459622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T06:36:07.871402+0000","last_change":"2024-05-09T06:36:06.865545+0000","last_active":"2024-05-09T06:36:07.871402+0000","last_peered":"2024-05-09T06:36:07.871402+0000","last_clean":"2024-05-09T06:36:07.871402+0000","last_became_active":"2024-05-09T06:36:06.865255+0000","last_became_peered":"2024-05-09T06:36:06.865255+0000","last_unstale":"2024-05-09T06:36:07.871402+0000","last_undegraded":"2024-05-09T06:36:07.871402+0000","last_fullsized":"2024-05-09T06:36:07.871402+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_clean_scrub_stamp":"2024-05-09T06:36:05.833229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:59:29.736305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T06:36:07.870022+0000","last_change":"2024-05-09T06:36:07.867264+0000","last_active":"2024-05-09T06:36:07.870022+0000","last_peered":"2024-05-09T06:36:07.870022+0000","last_clean":"2024-05-09T06:36:07.870022+0000","last_became_active":"2024-05-09T06:36:07.866981+0000","last_became_peered":"2024-05-09T06:36:07.866981+0000","last_unstale":"2024-05-09T06:36:07.870022+0000","last_undegraded":"2024-05-09T06:36:07.870022+0000","last_fullsized":"2024-05-09T06:36:07.870022+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:03:18.194613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T06:36:07.852112+0000","last_change":"2024-05-09T06:36:06.853399+0000","last_active":"2024-05-09T06:36:07.852112+0000","last_peered":"2024-05-09T06:36:07.852112+0000","last_clean":"2024-05-09T06:36:07.852112+0000","last_became_active":"2024-05-09T06:36:06.852409+0000","last_became_peered":"2024-05-09T06:36:06.852409+0000","last_unstale":"2024-05-09T06:36:07.852112+0000","last_undegraded":"2024-05-09T06:36:07.852112+0000","last_fullsized":"2024-05-09T06:36:07.852112+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_clean_scrub_stamp":"2024-05-09T06:36:05.833229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:15:49.248390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T06:36:07.880913+0000","last_change":"2024-05-09T06:36:07.875455+0000","last_active":"2024-05-09T06:36:07.880913+0000","last_peered":"2024-05-09T06:36:07.880913+0000","last_clean":"2024-05-09T06:36:07.880913+0000","last_became_active":"2024-05-09T06:36:07.875153+0000","last_became_peered":"2024-05-09T06:36:07.875153+0000","last_unstale":"2024-05-09T06:36:07.880913+0000","last_undegraded":"2024-05-09T06:36:07.880913+0000","last_fullsized":"2024-05-09T06:36:07.880913+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:03:39.564328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T06:36:07.856427+0000","last_change":"2024-05-09T06:36:06.861152+0000","last_active":"2024-05-09T06:36:07.856427+0000","last_peered":"2024-05-09T06:36:07.856427+0000","last_clean":"2024-05-09T06:36:07.856427+0000","last_became_active":"2024-05-09T06:36:06.860905+0000","last_became_peered":"2024-05-09T06:36:06.860905+0000","last_unstale":"2024-05-09T06:36:07.856427+0000","last_undegraded":"2024-05-09T06:36:07.856427+0000","last_fullsized":"2024-05-09T06:36:07.856427+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_clean_scrub_stamp":"2024-05-09T06:36:05.833229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:59:29.736305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T06:36:07.871597+0000","last_change":"2024-05-09T06:36:07.865602+0000","last_active":"2024-05-09T06:36:07.871597+0000","last_peered":"2024-05-09T06:36:07.871597+0000","last_clean":"2024-05-09T06:36:07.871597+0000","last_became_active":"2024-05-09T06:36:07.865083+0000","last_became_peered":"2024-05-09T06:36:07.865083+0000","last_unstale":"2024-05-09T06:36:07.871597+0000","last_undegraded":"2024-05-09T06:36:07.871597+0000","last_fullsized":"2024-05-09T06:36:07.871597+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:47:56.779394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T06:36:07.852890+0000","last_change":"2024-05-09T06:36:06.855928+0000","last_active":"2024-05-09T06:36:07.852890+0000","last_peered":"2024-05-09T06:36:07.852890+0000","last_clean":"2024-05-09T06:36:07.852890+0000","last_became_active":"2024-05-09T06:36:06.855322+0000","last_became_peered":"2024-05-09T06:36:06.855322+0000","last_unstale":"2024-05-09T06:36:07.852890+0000","last_undegraded":"2024-05-09T06:36:07.852890+0000","last_fullsized":"2024-05-09T06:36:07.852890+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_clean_scrub_stamp":"2024-05-09T06:36:05.833229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:03:38.559114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T06:36:07.875474+0000","last_change":"2024-05-09T06:36:07.870976+0000","last_active":"2024-05-09T06:36:07.875474+0000","last_peered":"2024-05-09T06:36:07.875474+0000","last_clean":"2024-05-09T06:36:07.875474+0000","last_became_active":"2024-05-09T06:36:07.870713+0000","last_became_peered":"2024-05-09T06:36:07.870713+0000","last_unstale":"2024-05-09T06:36:07.875474+0000","last_undegraded":"2024-05-09T06:36:07.875474+0000","last_fullsized":"2024-05-09T06:36:07.875474+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:03:39.564328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T06:36:07.874941+0000","last_change":"2024-05-09T06:36:06.863328+0000","last_active":"2024-05-09T06:36:07.874941+0000","last_peered":"2024-05-09T06:36:07.874941+0000","last_clean":"2024-05-09T06:36:07.874941+0000","last_became_active":"2024-05-09T06:36:06.862686+0000","last_became_peered":"2024-05-09T06:36:06.862686+0000","last_unstale":"2024-05-09T06:36:07.874941+0000","last_undegraded":"2024-05-09T06:36:07.874941+0000","last_fullsized":"2024-05-09T06:36:07.874941+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_clean_scrub_stamp":"2024-05-09T06:36:05.833229+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:59:29.736305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T06:36:07.882438+0000","last_change":"2024-05-09T06:36:07.868926+0000","last_active":"2024-05-09T06:36:07.882438+0000","last_peered":"2024-05-09T06:36:07.882438+0000","last_clean":"2024-05-09T06:36:07.882438+0000","last_became_active":"2024-05-09T06:36:07.868613+0000","last_became_peered":"2024-05-09T06:36:07.868613+0000","last_unstale":"2024-05-09T06:36:07.882438+0000","last_undegraded":"2024-05-09T06:36:07.882438+0000","last_fullsized":"2024-05-09T06:36:07.882438+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:03:18.194613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T06:36:07.858225+0000","last_change":"2024-05-09T06:36:06.861104+0000","last_active":"2024-05-09T06:36:07.858225+0000","last_peered":"2024-05-09T06:36:07.858225+0000","last_clean":"2024-05-09T06:36:07.858225+0000","last_became_active":"2024-05-09T06:36:06.860799+0000","last_became_peered":"2024-05-09T06:36:06.860799+0000","last_unstale":"2024-05-09T06:36:07.858225+0000","last_undegraded":"2024-05-09T06:36:07.858225+0000","last_fullsized":"2024-05-09T06:36:07.858225+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_clean_scrub_stamp":"2024-05-09T06:36:05.833229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:58:18.159070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T06:36:07.866537+0000","last_change":"2024-05-09T06:36:07.863846+0000","last_active":"2024-05-09T06:36:07.866537+0000","last_peered":"2024-05-09T06:36:07.866537+0000","last_clean":"2024-05-09T06:36:07.866537+0000","last_became_active":"2024-05-09T06:36:07.863161+0000","last_became_peered":"2024-05-09T06:36:07.863161+0000","last_unstale":"2024-05-09T06:36:07.866537+0000","last_undegraded":"2024-05-09T06:36:07.866537+0000","last_fullsized":"2024-05-09T06:36:07.866537+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:23:14.982108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T06:36:07.852530+0000","last_change":"2024-05-09T06:36:06.861276+0000","last_active":"2024-05-09T06:36:07.852530+0000","last_peered":"2024-05-09T06:36:07.852530+0000","last_clean":"2024-05-09T06:36:07.852530+0000","last_became_active":"2024-05-09T06:36:06.860977+0000","last_became_peered":"2024-05-09T06:36:06.860977+0000","last_unstale":"2024-05-09T06:36:07.852530+0000","last_undegraded":"2024-05-09T06:36:07.852530+0000","last_fullsized":"2024-05-09T06:36:07.852530+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_clean_scrub_stamp":"2024-05-09T06:36:05.833229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:16:34.559063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T06:36:07.875756+0000","last_change":"2024-05-09T06:36:07.870964+0000","last_active":"2024-05-09T06:36:07.875756+0000","last_peered":"2024-05-09T06:36:07.875756+0000","last_clean":"2024-05-09T06:36:07.875756+0000","last_became_active":"2024-05-09T06:36:07.870669+0000","last_became_peered":"2024-05-09T06:36:07.870669+0000","last_unstale":"2024-05-09T06:36:07.875756+0000","last_undegraded":"2024-05-09T06:36:07.875756+0000","last_fullsized":"2024-05-09T06:36:07.875756+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:58:19.164284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T06:36:07.852209+0000","last_change":"2024-05-09T06:36:06.861162+0000","last_active":"2024-05-09T06:36:07.852209+0000","last_peered":"2024-05-09T06:36:07.852209+0000","last_clean":"2024-05-09T06:36:07.852209+0000","last_became_active":"2024-05-09T06:36:06.860880+0000","last_became_peered":"2024-05-09T06:36:06.860880+0000","last_unstale":"2024-05-09T06:36:07.852209+0000","last_undegraded":"2024-05-09T06:36:07.852209+0000","last_fullsized":"2024-05-09T06:36:07.852209+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_clean_scrub_stamp":"2024-05-09T06:36:05.833229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:59:29.736305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T06:36:07.877210+0000","last_change":"2024-05-09T06:36:07.872872+0000","last_active":"2024-05-09T06:36:07.877210+0000","last_peered":"2024-05-09T06:36:07.877210+0000","last_clean":"2024-05-09T06:36:07.877210+0000","last_became_active":"2024-05-09T06:36:07.872556+0000","last_became_peered":"2024-05-09T06:36:07.872556+0000","last_unstale":"2024-05-09T06:36:07.877210+0000","last_undegraded":"2024-05-09T06:36:07.877210+0000","last_fullsized":"2024-05-09T06:36:07.877210+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:47:56.779394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T06:36:07.877875+0000","last_change":"2024-05-09T06:36:06.862160+0000","last_active":"2024-05-09T06:36:07.877875+0000","last_peered":"2024-05-09T06:36:07.877875+0000","last_clean":"2024-05-09T06:36:07.877875+0000","last_became_active":"2024-05-09T06:36:06.861881+0000","last_became_peered":"2024-05-09T06:36:06.861881+0000","last_unstale":"2024-05-09T06:36:07.877875+0000","last_undegraded":"2024-05-09T06:36:07.877875+0000","last_fullsized":"2024-05-09T06:36:07.877875+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_clean_scrub_stamp":"2024-05-09T06:36:05.833229+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:16:34.559063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T06:36:07.873367+0000","last_change":"2024-05-09T06:36:07.869000+0000","last_active":"2024-05-09T06:36:07.873367+0000","last_peered":"2024-05-09T06:36:07.873367+0000","last_clean":"2024-05-09T06:36:07.873367+0000","last_became_active":"2024-05-09T06:36:07.868522+0000","last_became_peered":"2024-05-09T06:36:07.868522+0000","last_unstale":"2024-05-09T06:36:07.873367+0000","last_undegraded":"2024-05-09T06:36:07.873367+0000","last_fullsized":"2024-05-09T06:36:07.873367+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:59:57.900963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T06:36:07.880954+0000","last_change":"2024-05-09T06:36:07.875476+0000","last_active":"2024-05-09T06:36:07.880954+0000","last_peered":"2024-05-09T06:36:07.880954+0000","last_clean":"2024-05-09T06:36:07.880954+0000","last_became_active":"2024-05-09T06:36:07.875170+0000","last_became_peered":"2024-05-09T06:36:07.875170+0000","last_unstale":"2024-05-09T06:36:07.880954+0000","last_undegraded":"2024-05-09T06:36:07.880954+0000","last_fullsized":"2024-05-09T06:36:07.880954+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:58:19.164284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T06:36:07.880546+0000","last_change":"2024-05-09T06:36:06.866061+0000","last_active":"2024-05-09T06:36:07.880546+0000","last_peered":"2024-05-09T06:36:07.880546+0000","last_clean":"2024-05-09T06:36:07.880546+0000","last_became_active":"2024-05-09T06:36:06.865783+0000","last_became_peered":"2024-05-09T06:36:06.865783+0000","last_unstale":"2024-05-09T06:36:07.880546+0000","last_undegraded":"2024-05-09T06:36:07.880546+0000","last_fullsized":"2024-05-09T06:36:07.880546+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_clean_scrub_stamp":"2024-05-09T06:36:05.833229+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:15:49.248390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T06:36:07.880933+0000","last_change":"2024-05-09T06:36:07.876132+0000","last_active":"2024-05-09T06:36:07.880933+0000","last_peered":"2024-05-09T06:36:07.880933+0000","last_clean":"2024-05-09T06:36:07.880933+0000","last_became_active":"2024-05-09T06:36:07.874679+0000","last_became_peered":"2024-05-09T06:36:07.874679+0000","last_unstale":"2024-05-09T06:36:07.880933+0000","last_undegraded":"2024-05-09T06:36:07.880933+0000","last_fullsized":"2024-05-09T06:36:07.880933+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:59:57.900963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T06:36:07.874964+0000","last_change":"2024-05-09T06:36:06.860619+0000","last_active":"2024-05-09T06:36:07.874964+0000","last_peered":"2024-05-09T06:36:07.874964+0000","last_clean":"2024-05-09T06:36:07.874964+0000","last_became_active":"2024-05-09T06:36:06.860323+0000","last_became_peered":"2024-05-09T06:36:06.860323+0000","last_unstale":"2024-05-09T06:36:07.874964+0000","last_undegraded":"2024-05-09T06:36:07.874964+0000","last_fullsized":"2024-05-09T06:36:07.874964+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_clean_scrub_stamp":"2024-05-09T06:36:05.833229+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:40:41.085704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T06:36:07.883896+0000","last_change":"2024-05-09T06:36:07.880245+0000","last_active":"2024-05-09T06:36:07.883896+0000","last_peered":"2024-05-09T06:36:07.883896+0000","last_clean":"2024-05-09T06:36:07.883896+0000","last_became_active":"2024-05-09T06:36:07.879926+0000","last_became_peered":"2024-05-09T06:36:07.879926+0000","last_unstale":"2024-05-09T06:36:07.883896+0000","last_undegraded":"2024-05-09T06:36:07.883896+0000","last_fullsized":"2024-05-09T06:36:07.883896+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:59:57.900963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T06:36:07.878622+0000","last_change":"2024-05-09T06:36:06.864934+0000","last_active":"2024-05-09T06:36:07.878622+0000","last_peered":"2024-05-09T06:36:07.878622+0000","last_clean":"2024-05-09T06:36:07.878622+0000","last_became_active":"2024-05-09T06:36:06.864680+0000","last_became_peered":"2024-05-09T06:36:06.864680+0000","last_unstale":"2024-05-09T06:36:07.878622+0000","last_undegraded":"2024-05-09T06:36:07.878622+0000","last_fullsized":"2024-05-09T06:36:07.878622+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_clean_scrub_stamp":"2024-05-09T06:36:05.833229+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:03:17.189398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T06:36:07.877563+0000","last_change":"2024-05-09T06:36:07.870194+0000","last_active":"2024-05-09T06:36:07.877563+0000","last_peered":"2024-05-09T06:36:07.877563+0000","last_clean":"2024-05-09T06:36:07.877563+0000","last_became_active":"2024-05-09T06:36:07.869895+0000","last_became_peered":"2024-05-09T06:36:07.869895+0000","last_unstale":"2024-05-09T06:36:07.877563+0000","last_undegraded":"2024-05-09T06:36:07.877563+0000","last_fullsized":"2024-05-09T06:36:07.877563+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:59:30.741519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":16,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T06:36:07.875880+0000","last_change":"2024-05-09T06:36:06.861090+0000","last_active":"2024-05-09T06:36:07.875880+0000","last_peered":"2024-05-09T06:36:07.875880+0000","last_clean":"2024-05-09T06:36:07.875880+0000","last_became_active":"2024-05-09T06:36:06.860774+0000","last_became_peered":"2024-05-09T06:36:06.860774+0000","last_unstale":"2024-05-09T06:36:07.875880+0000","last_undegraded":"2024-05-09T06:36:07.875880+0000","last_fullsized":"2024-05-09T06:36:07.875880+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_clean_scrub_stamp":"2024-05-09T06:36:05.833229+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:05:58.090556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27792,"kb_used_data":772,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344048,"statfs":{"total":96636764160,"available":96608305152,"internally_reserved":0,"allocated":790528,"data_stored":655584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574854,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":204800,"data_stored":66246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574853,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27212,"kb_used_data":192,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344628,"statfs":{"total":96636764160,"available":96608899072,"internally_reserved":0,"allocated":196608,"data_stored":64724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574853,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":204800,"data_stored":64760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":204800,"data_stored":66246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574853,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27208,"kb_used_data":188,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344632,"statfs":{"total":96636764160,"available":96608903168,"internally_reserved":0,"allocated":192512,"data_stored":64634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574853,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27216,"kb_used_data":196,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344624,"statfs":{"total":96636764160,"available":96608894976,"internally_reserved":0,"allocated":200704,"data_stored":64794,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27804,"kb_used_data":780,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344036,"statfs":{"total":96636764160,"available":96608292864,"internally_reserved":0,"allocated":798720,"data_stored":655640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-09T06:36:11.858 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-09T06:36:11.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.017+0000 7f03faf29640 1 Processor -- start 2024-05-09T06:36:11.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.017+0000 7f03faf29640 1 -- start start 2024-05-09T06:36:11.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.018+0000 7f03faf29640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f03f40ae9c0 0x7f03f406bc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:11.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.018+0000 7f03faf29640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f03f406c1a0 0x7f03f40b4ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:12.000 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.018+0000 7f03faf29640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f03f406cc30 con 0x7f03f415e290 2024-05-09T06:36:12.000 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.018+0000 7f03faf29640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f03f40b4630 con 0x7f03f406c1a0 2024-05-09T06:36:12.000 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.018+0000 7f03faf29640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f03f415e670 con 0x7f03f40ae9c0 2024-05-09T06:36:12.000 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.018+0000 7f03f94c6640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f03f406c1a0 0x7f03f40b4ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:12.000 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.018+0000 7f03f94c6640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f03f406c1a0 0x7f03f40b4ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.3:47546/0 (socket says 172.21.15.3:47546) 2024-05-09T06:36:12.000 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.018+0000 7f03f8cc5640 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7f03f415e290 0x7f03f40ae2b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.3:58308/0 (socket says 172.21.15.3:58308) 2024-05-09T06:36:12.000 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.018+0000 7f03ebfff640 1 --2- 172.21.15.3:0/872054162 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f03f40ae9c0 0x7f03f406bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:12.000 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.018+0000 7f03f94c6640 1 -- 172.21.15.3:0/872054162 learned_addr learned my addr 172.21.15.3:0/872054162 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:36:12.000 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.018+0000 7f03ebfff640 1 -- 172.21.15.3:0/872054162 >> v2:172.21.15.190:3300/0 conn(0x7f03f406c1a0 msgr2=0x7f03f40b4ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:12.000 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.019+0000 7f03ebfff640 1 --2- 172.21.15.3:0/872054162 >> v2:172.21.15.190:3300/0 conn(0x7f03f406c1a0 0x7f03f40b4ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:12.000 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.019+0000 7f03ebfff640 1 -- 172.21.15.3:0/872054162 >> v1:172.21.15.190:6789/0 conn(0x7f03f415e290 legacy=0x7f03f40ae2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:12.000 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.019+0000 7f03eb7fe640 1 -- 172.21.15.3:0/872054162 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 864525089 0 0) 0x7f03f406cc30 con 0x7f03f415e290 2024-05-09T06:36:12.000 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.019+0000 7f03ebfff640 1 -- 172.21.15.3:0/872054162 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03f406c950 con 0x7f03f40ae9c0 2024-05-09T06:36:12.000 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.019+0000 7f03f94c6640 1 --2- 172.21.15.3:0/872054162 >> v2:172.21.15.190:3300/0 conn(0x7f03f406c1a0 0x7f03f40b4ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T06:36:12.002 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.019+0000 7f03ebfff640 1 --2- 172.21.15.3:0/872054162 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f03f40ae9c0 0x7f03f406bc60 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f03e40097a0 tx=0x7f03e402a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a98e6fe8e6734f77 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:12.002 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.019+0000 7f03eb7fe640 1 -- 172.21.15.3:0/872054162 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03e4037070 con 0x7f03f40ae9c0 2024-05-09T06:36:12.002 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.020+0000 7f03eb7fe640 1 -- 172.21.15.3:0/872054162 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03e402ae50 con 0x7f03f40ae9c0 2024-05-09T06:36:12.002 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.020+0000 7f03eb7fe640 1 -- 172.21.15.3:0/872054162 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03e4032660 con 0x7f03f40ae9c0 2024-05-09T06:36:12.002 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.020+0000 7f03faf29640 1 -- 172.21.15.3:0/872054162 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f03f40ae9c0 msgr2=0x7f03f406bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:12.002 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.020+0000 7f03faf29640 1 --2- 172.21.15.3:0/872054162 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f03f40ae9c0 0x7f03f406bc60 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f03e40097a0 tx=0x7f03e402a100 comp rx=0 tx=0).stop 2024-05-09T06:36:12.002 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.020+0000 7f03faf29640 1 -- 172.21.15.3:0/872054162 shutdown_connections 2024-05-09T06:36:12.002 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.020+0000 7f03faf29640 1 --2- 172.21.15.3:0/872054162 >> v2:172.21.15.190:3300/0 conn(0x7f03f406c1a0 0x7f03f40b4ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:12.002 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.020+0000 7f03faf29640 1 --2- 172.21.15.3:0/872054162 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f03f40ae9c0 0x7f03f406bc60 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:12.002 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.020+0000 7f03faf29640 1 -- 172.21.15.3:0/872054162 >> 172.21.15.3:0/872054162 conn(0x7f03f40d05d0 msgr2=0x7f03f40d09e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:12.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.020+0000 7f03faf29640 1 -- 172.21.15.3:0/872054162 shutdown_connections 2024-05-09T06:36:12.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.020+0000 7f03faf29640 1 -- 172.21.15.3:0/872054162 wait complete. 2024-05-09T06:36:12.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.021+0000 7f03faf29640 1 Processor -- start 2024-05-09T06:36:12.005 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.021+0000 7f03faf29640 1 -- start start 2024-05-09T06:36:12.005 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.022+0000 7f03faf29640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f03f406c1a0 0x7f03f41f8150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:12.005 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.022+0000 7f03faf29640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f03f40ae9c0 0x7f03f414def0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:12.005 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.022+0000 7f03faf29640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f03f415e290 0x7f03f40ce2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:12.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.022+0000 7f03faf29640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f03f406c630 con 0x7f03f406c1a0 2024-05-09T06:36:12.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.022+0000 7f03faf29640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f03f40b42c0 con 0x7f03f415e290 2024-05-09T06:36:12.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.022+0000 7f03faf29640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f03f406d8f0 con 0x7f03f40ae9c0 2024-05-09T06:36:12.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.022+0000 7f03f8cc5640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f03f406c1a0 0x7f03f41f8150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:12.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.022+0000 7f03f8cc5640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f03f406c1a0 0x7f03f41f8150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49398/0 (socket says 172.21.15.3:49398) 2024-05-09T06:36:12.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.022+0000 7f03f8cc5640 1 -- 172.21.15.3:0/2334889435 learned_addr learned my addr 172.21.15.3:0/2334889435 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:36:12.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.023+0000 7f03ebfff640 1 --2- 172.21.15.3:0/2334889435 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f03f40ae9c0 0x7f03f414def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:12.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.023+0000 7f03f94c6640 1 --2- 172.21.15.3:0/2334889435 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f03f415e290 0x7f03f40ce2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:12.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.023+0000 7f03f8cc5640 1 -- 172.21.15.3:0/2334889435 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f03f40ae9c0 msgr2=0x7f03f414def0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:12.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.023+0000 7f03f8cc5640 1 --2- 172.21.15.3:0/2334889435 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f03f40ae9c0 0x7f03f414def0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:12.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.023+0000 7f03f8cc5640 1 -- 172.21.15.3:0/2334889435 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f03f415e290 msgr2=0x7f03f40ce2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:12.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.023+0000 7f03f8cc5640 1 --2- 172.21.15.3:0/2334889435 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f03f415e290 0x7f03f40ce2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:12.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.023+0000 7f03f8cc5640 1 -- 172.21.15.3:0/2334889435 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03f414e430 con 0x7f03f406c1a0 2024-05-09T06:36:12.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.023+0000 7f03ebfff640 1 --2- 172.21.15.3:0/2334889435 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f03f40ae9c0 0x7f03f414def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:36:12.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.023+0000 7f03f8cc5640 1 --2- 172.21.15.3:0/2334889435 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f03f406c1a0 0x7f03f41f8150 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f03f000c4d0 tx=0x7f03f000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:12.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.023+0000 7f03e97fa640 1 -- 172.21.15.3:0/2334889435 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03f0018070 con 0x7f03f406c1a0 2024-05-09T06:36:12.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.023+0000 7f03e97fa640 1 -- 172.21.15.3:0/2334889435 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03f0007500 con 0x7f03f406c1a0 2024-05-09T06:36:12.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.023+0000 7f03faf29640 1 -- 172.21.15.3:0/2334889435 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03f40ced30 con 0x7f03f406c1a0 2024-05-09T06:36:12.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.023+0000 7f03e97fa640 1 -- 172.21.15.3:0/2334889435 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03f0004a20 con 0x7f03f406c1a0 2024-05-09T06:36:12.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.023+0000 7f03faf29640 1 -- 172.21.15.3:0/2334889435 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03f41f7670 con 0x7f03f406c1a0 2024-05-09T06:36:12.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.024+0000 7f03faf29640 1 -- 172.21.15.3:0/2334889435 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03f4067380 con 0x7f03f406c1a0 2024-05-09T06:36:12.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.025+0000 7f03e97fa640 1 -- 172.21.15.3:0/2334889435 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f03f001b070 con 0x7f03f406c1a0 2024-05-09T06:36:12.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.025+0000 7f03e97fa640 1 --2- 172.21.15.3:0/2334889435 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f03e006be60 0x7f03e006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:12.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.025+0000 7f03e97fa640 1 -- 172.21.15.3:0/2334889435 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f03f008a100 con 0x7f03f406c1a0 2024-05-09T06:36:12.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.025+0000 7f03ebfff640 1 --2- 172.21.15.3:0/2334889435 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f03e006be60 0x7f03e006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:12.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.026+0000 7f03ebfff640 1 --2- 172.21.15.3:0/2334889435 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f03e006be60 0x7f03e006e320 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f03e4034000 tx=0x7f03e402e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:12.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.029+0000 7f03e97fa640 1 -- 172.21.15.3:0/2334889435 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f03f0058ab0 con 0x7f03f406c1a0 2024-05-09T06:36:12.194 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.213+0000 7f03faf29640 1 -- 172.21.15.3:0/2334889435 --> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f03f404f660 con 0x7f03e006be60 2024-05-09T06:36:12.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.220+0000 7f03e97fa640 1 -- 172.21.15.3:0/2334889435 <== mgr.4105 v2:172.21.15.190:6832/975893321 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251882 (secure 0 0 0) 0x7f03f404f660 con 0x7f03e006be60 2024-05-09T06:36:12.204 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-09T06:36:12.205 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2024-05-09T06:36:12.207 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.223+0000 7f03faf29640 1 -- 172.21.15.3:0/2334889435 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f03e006be60 msgr2=0x7f03e006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:12.207 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.223+0000 7f03faf29640 1 --2- 172.21.15.3:0/2334889435 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f03e006be60 0x7f03e006e320 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f03e4034000 tx=0x7f03e402e000 comp rx=0 tx=0).stop 2024-05-09T06:36:12.207 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.223+0000 7f03faf29640 1 -- 172.21.15.3:0/2334889435 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f03f406c1a0 msgr2=0x7f03f41f8150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:12.207 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.223+0000 7f03faf29640 1 --2- 172.21.15.3:0/2334889435 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f03f406c1a0 0x7f03f41f8150 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f03f000c4d0 tx=0x7f03f000c990 comp rx=0 tx=0).stop 2024-05-09T06:36:12.207 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.223+0000 7f03faf29640 1 -- 172.21.15.3:0/2334889435 shutdown_connections 2024-05-09T06:36:12.207 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.223+0000 7f03faf29640 1 --2- 172.21.15.3:0/2334889435 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f03e006be60 0x7f03e006e320 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:12.207 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.223+0000 7f03faf29640 1 --2- 172.21.15.3:0/2334889435 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f03f415e290 0x7f03f40ce2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:12.207 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.223+0000 7f03faf29640 1 --2- 172.21.15.3:0/2334889435 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f03f40ae9c0 0x7f03f414def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:12.207 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.223+0000 7f03faf29640 1 --2- 172.21.15.3:0/2334889435 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f03f406c1a0 0x7f03f41f8150 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:12.207 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.223+0000 7f03faf29640 1 -- 172.21.15.3:0/2334889435 >> 172.21.15.3:0/2334889435 conn(0x7f03f40d05d0 msgr2=0x7f03f406b730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:12.207 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.223+0000 7f03faf29640 1 -- 172.21.15.3:0/2334889435 shutdown_connections 2024-05-09T06:36:12.207 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.223+0000 7f03faf29640 1 -- 172.21.15.3:0/2334889435 wait complete. 2024-05-09T06:36:12.215 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2024-05-09T06:36:10.776481+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":135,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218892,"kb_used_data":2728,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755828,"statfs":{"total":773094113280,"available":772869967872,"internally_reserved":0,"allocated":2793472,"data_stored":1702628,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.937739"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T06:36:07.874676+0000","last_change":"2024-05-09T06:36:07.870322+0000","last_active":"2024-05-09T06:36:07.874676+0000","last_peered":"2024-05-09T06:36:07.874676+0000","last_clean":"2024-05-09T06:36:07.874676+0000","last_became_active":"2024-05-09T06:36:07.870000+0000","last_became_peered":"2024-05-09T06:36:07.870000+0000","last_unstale":"2024-05-09T06:36:07.874676+0000","last_undegraded":"2024-05-09T06:36:07.874676+0000","last_fullsized":"2024-05-09T06:36:07.874676+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:24:51.534066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T06:36:07.872110+0000","last_change":"2024-05-09T06:36:07.865401+0000","last_active":"2024-05-09T06:36:07.872110+0000","last_peered":"2024-05-09T06:36:07.872110+0000","last_clean":"2024-05-09T06:36:07.872110+0000","last_became_active":"2024-05-09T06:36:07.865119+0000","last_became_peered":"2024-05-09T06:36:07.865119+0000","last_unstale":"2024-05-09T06:36:07.872110+0000","last_undegraded":"2024-05-09T06:36:07.872110+0000","last_fullsized":"2024-05-09T06:36:07.872110+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:39:07.691670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T06:36:07.874890+0000","last_change":"2024-05-09T06:36:07.865504+0000","last_active":"2024-05-09T06:36:07.874890+0000","last_peered":"2024-05-09T06:36:07.874890+0000","last_clean":"2024-05-09T06:36:07.874890+0000","last_became_active":"2024-05-09T06:36:07.864109+0000","last_became_peered":"2024-05-09T06:36:07.864109+0000","last_unstale":"2024-05-09T06:36:07.874890+0000","last_undegraded":"2024-05-09T06:36:07.874890+0000","last_fullsized":"2024-05-09T06:36:07.874890+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:23:14.982108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T06:36:07.881369+0000","last_change":"2024-05-09T06:36:07.879922+0000","last_active":"2024-05-09T06:36:07.881369+0000","last_peered":"2024-05-09T06:36:07.881369+0000","last_clean":"2024-05-09T06:36:07.881369+0000","last_became_active":"2024-05-09T06:36:07.879559+0000","last_became_peered":"2024-05-09T06:36:07.879559+0000","last_unstale":"2024-05-09T06:36:07.881369+0000","last_undegraded":"2024-05-09T06:36:07.881369+0000","last_fullsized":"2024-05-09T06:36:07.881369+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:44:00.455366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T06:36:07.869262+0000","last_change":"2024-05-09T06:36:07.867057+0000","last_active":"2024-05-09T06:36:07.869262+0000","last_peered":"2024-05-09T06:36:07.869262+0000","last_clean":"2024-05-09T06:36:07.869262+0000","last_became_active":"2024-05-09T06:36:07.866657+0000","last_became_peered":"2024-05-09T06:36:07.866657+0000","last_unstale":"2024-05-09T06:36:07.869262+0000","last_undegraded":"2024-05-09T06:36:07.869262+0000","last_fullsized":"2024-05-09T06:36:07.869262+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:39:07.691670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T06:36:07.884464+0000","last_change":"2024-05-09T06:36:07.883141+0000","last_active":"2024-05-09T06:36:07.884464+0000","last_peered":"2024-05-09T06:36:07.884464+0000","last_clean":"2024-05-09T06:36:07.884464+0000","last_became_active":"2024-05-09T06:36:07.882826+0000","last_became_peered":"2024-05-09T06:36:07.882826+0000","last_unstale":"2024-05-09T06:36:07.884464+0000","last_undegraded":"2024-05-09T06:36:07.884464+0000","last_fullsized":"2024-05-09T06:36:07.884464+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:50:02.245694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T06:36:07.882132+0000","last_change":"2024-05-09T06:36:07.876998+0000","last_active":"2024-05-09T06:36:07.882132+0000","last_peered":"2024-05-09T06:36:07.882132+0000","last_clean":"2024-05-09T06:36:07.882132+0000","last_became_active":"2024-05-09T06:36:07.876672+0000","last_became_peered":"2024-05-09T06:36:07.876672+0000","last_unstale":"2024-05-09T06:36:07.882132+0000","last_undegraded":"2024-05-09T06:36:07.882132+0000","last_fullsized":"2024-05-09T06:36:07.882132+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:50:02.245694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T06:36:07.882731+0000","last_change":"2024-05-09T06:36:07.878020+0000","last_active":"2024-05-09T06:36:07.882731+0000","last_peered":"2024-05-09T06:36:07.882731+0000","last_clean":"2024-05-09T06:36:07.882731+0000","last_became_active":"2024-05-09T06:36:07.877767+0000","last_became_peered":"2024-05-09T06:36:07.877767+0000","last_unstale":"2024-05-09T06:36:07.882731+0000","last_undegraded":"2024-05-09T06:36:07.882731+0000","last_fullsized":"2024-05-09T06:36:07.882731+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:41:33.459622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T06:36:07.882156+0000","last_change":"2024-05-09T06:36:07.877214+0000","last_active":"2024-05-09T06:36:07.882156+0000","last_peered":"2024-05-09T06:36:07.882156+0000","last_clean":"2024-05-09T06:36:07.882156+0000","last_became_active":"2024-05-09T06:36:07.876924+0000","last_became_peered":"2024-05-09T06:36:07.876924+0000","last_unstale":"2024-05-09T06:36:07.882156+0000","last_undegraded":"2024-05-09T06:36:07.882156+0000","last_fullsized":"2024-05-09T06:36:07.882156+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:51:24.938206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T06:36:07.882483+0000","last_change":"2024-05-09T06:36:07.879229+0000","last_active":"2024-05-09T06:36:07.882483+0000","last_peered":"2024-05-09T06:36:07.882483+0000","last_clean":"2024-05-09T06:36:07.882483+0000","last_became_active":"2024-05-09T06:36:07.879000+0000","last_became_peered":"2024-05-09T06:36:07.879000+0000","last_unstale":"2024-05-09T06:36:07.882483+0000","last_undegraded":"2024-05-09T06:36:07.882483+0000","last_fullsized":"2024-05-09T06:36:07.882483+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:41:33.459622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T06:36:07.882187+0000","last_change":"2024-05-09T06:36:07.878671+0000","last_active":"2024-05-09T06:36:07.882187+0000","last_peered":"2024-05-09T06:36:07.882187+0000","last_clean":"2024-05-09T06:36:07.882187+0000","last_became_active":"2024-05-09T06:36:07.878389+0000","last_became_peered":"2024-05-09T06:36:07.878389+0000","last_unstale":"2024-05-09T06:36:07.882187+0000","last_undegraded":"2024-05-09T06:36:07.882187+0000","last_fullsized":"2024-05-09T06:36:07.882187+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:47:56.779394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T06:36:07.868118+0000","last_change":"2024-05-09T06:36:07.859955+0000","last_active":"2024-05-09T06:36:07.868118+0000","last_peered":"2024-05-09T06:36:07.868118+0000","last_clean":"2024-05-09T06:36:07.868118+0000","last_became_active":"2024-05-09T06:36:07.859649+0000","last_became_peered":"2024-05-09T06:36:07.859649+0000","last_unstale":"2024-05-09T06:36:07.868118+0000","last_undegraded":"2024-05-09T06:36:07.868118+0000","last_fullsized":"2024-05-09T06:36:07.868118+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:05:59.095770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T06:36:07.877186+0000","last_change":"2024-05-09T06:36:07.870408+0000","last_active":"2024-05-09T06:36:07.877186+0000","last_peered":"2024-05-09T06:36:07.877186+0000","last_clean":"2024-05-09T06:36:07.877186+0000","last_became_active":"2024-05-09T06:36:07.868679+0000","last_became_peered":"2024-05-09T06:36:07.868679+0000","last_unstale":"2024-05-09T06:36:07.877186+0000","last_undegraded":"2024-05-09T06:36:07.877186+0000","last_fullsized":"2024-05-09T06:36:07.877186+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:16:35.564277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T06:36:07.882449+0000","last_change":"2024-05-09T06:36:07.872146+0000","last_active":"2024-05-09T06:36:07.882449+0000","last_peered":"2024-05-09T06:36:07.882449+0000","last_clean":"2024-05-09T06:36:07.882449+0000","last_became_active":"2024-05-09T06:36:07.871858+0000","last_became_peered":"2024-05-09T06:36:07.871858+0000","last_unstale":"2024-05-09T06:36:07.882449+0000","last_undegraded":"2024-05-09T06:36:07.882449+0000","last_fullsized":"2024-05-09T06:36:07.882449+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:15:50.253604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T06:36:07.871548+0000","last_change":"2024-05-09T06:36:07.864771+0000","last_active":"2024-05-09T06:36:07.871548+0000","last_peered":"2024-05-09T06:36:07.871548+0000","last_clean":"2024-05-09T06:36:07.871548+0000","last_became_active":"2024-05-09T06:36:07.864352+0000","last_became_peered":"2024-05-09T06:36:07.864352+0000","last_unstale":"2024-05-09T06:36:07.871548+0000","last_undegraded":"2024-05-09T06:36:07.871548+0000","last_fullsized":"2024-05-09T06:36:07.871548+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:59:57.900963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T06:36:07.884449+0000","last_change":"2024-05-09T06:36:07.883189+0000","last_active":"2024-05-09T06:36:07.884449+0000","last_peered":"2024-05-09T06:36:07.884449+0000","last_clean":"2024-05-09T06:36:07.884449+0000","last_became_active":"2024-05-09T06:36:07.882906+0000","last_became_peered":"2024-05-09T06:36:07.882906+0000","last_unstale":"2024-05-09T06:36:07.884449+0000","last_undegraded":"2024-05-09T06:36:07.884449+0000","last_fullsized":"2024-05-09T06:36:07.884449+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:51:24.938206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T06:36:07.881323+0000","last_change":"2024-05-09T06:36:07.878966+0000","last_active":"2024-05-09T06:36:07.881323+0000","last_peered":"2024-05-09T06:36:07.881323+0000","last_clean":"2024-05-09T06:36:07.881323+0000","last_became_active":"2024-05-09T06:36:07.878678+0000","last_became_peered":"2024-05-09T06:36:07.878678+0000","last_unstale":"2024-05-09T06:36:07.881323+0000","last_undegraded":"2024-05-09T06:36:07.881323+0000","last_fullsized":"2024-05-09T06:36:07.881323+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:53:50.215704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T06:36:07.868140+0000","last_change":"2024-05-09T06:36:07.866493+0000","last_active":"2024-05-09T06:36:07.868140+0000","last_peered":"2024-05-09T06:36:07.868140+0000","last_clean":"2024-05-09T06:36:07.868140+0000","last_became_active":"2024-05-09T06:36:07.866157+0000","last_became_peered":"2024-05-09T06:36:07.866157+0000","last_unstale":"2024-05-09T06:36:07.868140+0000","last_undegraded":"2024-05-09T06:36:07.868140+0000","last_fullsized":"2024-05-09T06:36:07.868140+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:40:42.090918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T06:36:07.869466+0000","last_change":"2024-05-09T06:36:07.866352+0000","last_active":"2024-05-09T06:36:07.869466+0000","last_peered":"2024-05-09T06:36:07.869466+0000","last_clean":"2024-05-09T06:36:07.869466+0000","last_became_active":"2024-05-09T06:36:07.866066+0000","last_became_peered":"2024-05-09T06:36:07.866066+0000","last_unstale":"2024-05-09T06:36:07.869466+0000","last_undegraded":"2024-05-09T06:36:07.869466+0000","last_fullsized":"2024-05-09T06:36:07.869466+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:59:57.900963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T06:36:07.865622+0000","last_change":"2024-05-09T06:36:07.858555+0000","last_active":"2024-05-09T06:36:07.865622+0000","last_peered":"2024-05-09T06:36:07.865622+0000","last_clean":"2024-05-09T06:36:07.865622+0000","last_became_active":"2024-05-09T06:36:07.858259+0000","last_became_peered":"2024-05-09T06:36:07.858259+0000","last_unstale":"2024-05-09T06:36:07.865622+0000","last_undegraded":"2024-05-09T06:36:07.865622+0000","last_fullsized":"2024-05-09T06:36:07.865622+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:16:35.564277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T06:36:07.883670+0000","last_change":"2024-05-09T06:36:07.875125+0000","last_active":"2024-05-09T06:36:07.883670+0000","last_peered":"2024-05-09T06:36:07.883670+0000","last_clean":"2024-05-09T06:36:07.883670+0000","last_became_active":"2024-05-09T06:36:07.874432+0000","last_became_peered":"2024-05-09T06:36:07.874432+0000","last_unstale":"2024-05-09T06:36:07.883670+0000","last_undegraded":"2024-05-09T06:36:07.883670+0000","last_fullsized":"2024-05-09T06:36:07.883670+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:16:35.564277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T06:36:07.881068+0000","last_change":"2024-05-09T06:36:07.876701+0000","last_active":"2024-05-09T06:36:07.881068+0000","last_peered":"2024-05-09T06:36:07.881068+0000","last_clean":"2024-05-09T06:36:07.881068+0000","last_became_active":"2024-05-09T06:36:07.876416+0000","last_became_peered":"2024-05-09T06:36:07.876416+0000","last_unstale":"2024-05-09T06:36:07.881068+0000","last_undegraded":"2024-05-09T06:36:07.881068+0000","last_fullsized":"2024-05-09T06:36:07.881068+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:47:56.779394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T06:36:07.875346+0000","last_change":"2024-05-09T06:36:07.868404+0000","last_active":"2024-05-09T06:36:07.875346+0000","last_peered":"2024-05-09T06:36:07.875346+0000","last_clean":"2024-05-09T06:36:07.875346+0000","last_became_active":"2024-05-09T06:36:07.867909+0000","last_became_peered":"2024-05-09T06:36:07.867909+0000","last_unstale":"2024-05-09T06:36:07.875346+0000","last_undegraded":"2024-05-09T06:36:07.875346+0000","last_fullsized":"2024-05-09T06:36:07.875346+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:40:42.090918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T06:36:07.881396+0000","last_change":"2024-05-09T06:36:07.880411+0000","last_active":"2024-05-09T06:36:07.881396+0000","last_peered":"2024-05-09T06:36:07.881396+0000","last_clean":"2024-05-09T06:36:07.881396+0000","last_became_active":"2024-05-09T06:36:07.879887+0000","last_became_peered":"2024-05-09T06:36:07.879887+0000","last_unstale":"2024-05-09T06:36:07.881396+0000","last_undegraded":"2024-05-09T06:36:07.881396+0000","last_fullsized":"2024-05-09T06:36:07.881396+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:34:13.971998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T06:36:07.875405+0000","last_change":"2024-05-09T06:36:07.868320+0000","last_active":"2024-05-09T06:36:07.875405+0000","last_peered":"2024-05-09T06:36:07.875405+0000","last_clean":"2024-05-09T06:36:07.875405+0000","last_became_active":"2024-05-09T06:36:07.867969+0000","last_became_peered":"2024-05-09T06:36:07.867969+0000","last_unstale":"2024-05-09T06:36:07.875405+0000","last_undegraded":"2024-05-09T06:36:07.875405+0000","last_fullsized":"2024-05-09T06:36:07.875405+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:15:50.253604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T06:36:07.873521+0000","last_change":"2024-05-09T06:36:07.869956+0000","last_active":"2024-05-09T06:36:07.873521+0000","last_peered":"2024-05-09T06:36:07.873521+0000","last_clean":"2024-05-09T06:36:07.873521+0000","last_became_active":"2024-05-09T06:36:07.869695+0000","last_became_peered":"2024-05-09T06:36:07.869695+0000","last_unstale":"2024-05-09T06:36:07.873521+0000","last_undegraded":"2024-05-09T06:36:07.873521+0000","last_fullsized":"2024-05-09T06:36:07.873521+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:39:07.691670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T06:36:07.871772+0000","last_change":"2024-05-09T06:36:07.867102+0000","last_active":"2024-05-09T06:36:07.871772+0000","last_peered":"2024-05-09T06:36:07.871772+0000","last_clean":"2024-05-09T06:36:07.871772+0000","last_became_active":"2024-05-09T06:36:07.866785+0000","last_became_peered":"2024-05-09T06:36:07.866785+0000","last_unstale":"2024-05-09T06:36:07.871772+0000","last_undegraded":"2024-05-09T06:36:07.871772+0000","last_fullsized":"2024-05-09T06:36:07.871772+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:53:50.215704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T06:36:07.875611+0000","last_change":"2024-05-09T06:36:07.871788+0000","last_active":"2024-05-09T06:36:07.875611+0000","last_peered":"2024-05-09T06:36:07.875611+0000","last_clean":"2024-05-09T06:36:07.875611+0000","last_became_active":"2024-05-09T06:36:07.871500+0000","last_became_peered":"2024-05-09T06:36:07.871500+0000","last_unstale":"2024-05-09T06:36:07.875611+0000","last_undegraded":"2024-05-09T06:36:07.875611+0000","last_fullsized":"2024-05-09T06:36:07.875611+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:39:07.691670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T06:36:07.883742+0000","last_change":"2024-05-09T06:36:07.876623+0000","last_active":"2024-05-09T06:36:07.883742+0000","last_peered":"2024-05-09T06:36:07.883742+0000","last_clean":"2024-05-09T06:36:07.883742+0000","last_became_active":"2024-05-09T06:36:07.876358+0000","last_became_peered":"2024-05-09T06:36:07.876358+0000","last_unstale":"2024-05-09T06:36:07.883742+0000","last_undegraded":"2024-05-09T06:36:07.883742+0000","last_fullsized":"2024-05-09T06:36:07.883742+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:58:19.164284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T06:36:07.881325+0000","last_change":"2024-05-09T06:36:07.879149+0000","last_active":"2024-05-09T06:36:07.881325+0000","last_peered":"2024-05-09T06:36:07.881325+0000","last_clean":"2024-05-09T06:36:07.881325+0000","last_became_active":"2024-05-09T06:36:07.878868+0000","last_became_peered":"2024-05-09T06:36:07.878868+0000","last_unstale":"2024-05-09T06:36:07.881325+0000","last_undegraded":"2024-05-09T06:36:07.881325+0000","last_fullsized":"2024-05-09T06:36:07.881325+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:24:51.534066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T06:36:07.879373+0000","last_change":"2024-05-09T06:36:07.876443+0000","last_active":"2024-05-09T06:36:07.879373+0000","last_peered":"2024-05-09T06:36:07.879373+0000","last_clean":"2024-05-09T06:36:07.879373+0000","last_became_active":"2024-05-09T06:36:07.876122+0000","last_became_peered":"2024-05-09T06:36:07.876122+0000","last_unstale":"2024-05-09T06:36:07.879373+0000","last_undegraded":"2024-05-09T06:36:07.879373+0000","last_fullsized":"2024-05-09T06:36:07.879373+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:59:30.741519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T06:36:07.870539+0000","last_change":"2024-05-09T06:36:07.867899+0000","last_active":"2024-05-09T06:36:07.870539+0000","last_peered":"2024-05-09T06:36:07.870539+0000","last_clean":"2024-05-09T06:36:07.870539+0000","last_became_active":"2024-05-09T06:36:07.867371+0000","last_became_peered":"2024-05-09T06:36:07.867371+0000","last_unstale":"2024-05-09T06:36:07.870539+0000","last_undegraded":"2024-05-09T06:36:07.870539+0000","last_fullsized":"2024-05-09T06:36:07.870539+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:03:39.564328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T06:36:07.874468+0000","last_change":"2024-05-09T06:36:07.871355+0000","last_active":"2024-05-09T06:36:07.874468+0000","last_peered":"2024-05-09T06:36:07.874468+0000","last_clean":"2024-05-09T06:36:07.874468+0000","last_became_active":"2024-05-09T06:36:07.871025+0000","last_became_peered":"2024-05-09T06:36:07.871025+0000","last_unstale":"2024-05-09T06:36:07.874468+0000","last_undegraded":"2024-05-09T06:36:07.874468+0000","last_fullsized":"2024-05-09T06:36:07.874468+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:15:50.253604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T06:36:07.871608+0000","last_change":"2024-05-09T06:36:06.865248+0000","last_active":"2024-05-09T06:36:07.871608+0000","last_peered":"2024-05-09T06:36:07.871608+0000","last_clean":"2024-05-09T06:36:07.871608+0000","last_became_active":"2024-05-09T06:36:06.864940+0000","last_became_peered":"2024-05-09T06:36:06.864940+0000","last_unstale":"2024-05-09T06:36:07.871608+0000","last_undegraded":"2024-05-09T06:36:07.871608+0000","last_fullsized":"2024-05-09T06:36:07.871608+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_clean_scrub_stamp":"2024-05-09T06:36:05.833229+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:08:20.004456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T06:36:07.882647+0000","last_change":"2024-05-09T06:36:07.874223+0000","last_active":"2024-05-09T06:36:07.882647+0000","last_peered":"2024-05-09T06:36:07.882647+0000","last_clean":"2024-05-09T06:36:07.882647+0000","last_became_active":"2024-05-09T06:36:07.873787+0000","last_became_peered":"2024-05-09T06:36:07.873787+0000","last_unstale":"2024-05-09T06:36:07.882647+0000","last_undegraded":"2024-05-09T06:36:07.882647+0000","last_fullsized":"2024-05-09T06:36:07.882647+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:59:30.741519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T06:36:07.868643+0000","last_change":"2024-05-09T06:36:06.862586+0000","last_active":"2024-05-09T06:36:07.868643+0000","last_peered":"2024-05-09T06:36:07.868643+0000","last_clean":"2024-05-09T06:36:07.868643+0000","last_became_active":"2024-05-09T06:36:06.861545+0000","last_became_peered":"2024-05-09T06:36:06.861545+0000","last_unstale":"2024-05-09T06:36:07.868643+0000","last_undegraded":"2024-05-09T06:36:07.868643+0000","last_fullsized":"2024-05-09T06:36:07.868643+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_clean_scrub_stamp":"2024-05-09T06:36:05.833229+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:15:49.248390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T06:36:07.859545+0000","last_change":"2024-05-09T06:36:06.867498+0000","last_active":"2024-05-09T06:36:07.859545+0000","last_peered":"2024-05-09T06:36:07.859545+0000","last_clean":"2024-05-09T06:36:07.859545+0000","last_became_active":"2024-05-09T06:36:06.867177+0000","last_became_peered":"2024-05-09T06:36:06.867177+0000","last_unstale":"2024-05-09T06:36:07.859545+0000","last_undegraded":"2024-05-09T06:36:07.859545+0000","last_fullsized":"2024-05-09T06:36:07.859545+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_clean_scrub_stamp":"2024-05-09T06:36:05.833229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:23:13.976894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T06:36:07.875735+0000","last_change":"2024-05-09T06:36:07.871392+0000","last_active":"2024-05-09T06:36:07.875735+0000","last_peered":"2024-05-09T06:36:07.875735+0000","last_clean":"2024-05-09T06:36:07.875735+0000","last_became_active":"2024-05-09T06:36:07.871103+0000","last_became_peered":"2024-05-09T06:36:07.871103+0000","last_unstale":"2024-05-09T06:36:07.875735+0000","last_undegraded":"2024-05-09T06:36:07.875735+0000","last_fullsized":"2024-05-09T06:36:07.875735+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:59:57.900963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T06:36:07.871825+0000","last_change":"2024-05-09T06:36:06.865012+0000","last_active":"2024-05-09T06:36:07.871825+0000","last_peered":"2024-05-09T06:36:07.871825+0000","last_clean":"2024-05-09T06:36:07.871825+0000","last_became_active":"2024-05-09T06:36:06.864434+0000","last_became_peered":"2024-05-09T06:36:06.864434+0000","last_unstale":"2024-05-09T06:36:07.871825+0000","last_undegraded":"2024-05-09T06:36:07.871825+0000","last_fullsized":"2024-05-09T06:36:07.871825+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_clean_scrub_stamp":"2024-05-09T06:36:05.833229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:16:34.559063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T06:36:07.879368+0000","last_change":"2024-05-09T06:36:07.870720+0000","last_active":"2024-05-09T06:36:07.879368+0000","last_peered":"2024-05-09T06:36:07.879368+0000","last_clean":"2024-05-09T06:36:07.879368+0000","last_became_active":"2024-05-09T06:36:07.870431+0000","last_became_peered":"2024-05-09T06:36:07.870431+0000","last_unstale":"2024-05-09T06:36:07.879368+0000","last_undegraded":"2024-05-09T06:36:07.879368+0000","last_fullsized":"2024-05-09T06:36:07.879368+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:24:51.534066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T06:36:07.857540+0000","last_change":"2024-05-09T06:36:06.866523+0000","last_active":"2024-05-09T06:36:07.857540+0000","last_peered":"2024-05-09T06:36:07.857540+0000","last_clean":"2024-05-09T06:36:07.857540+0000","last_became_active":"2024-05-09T06:36:06.866220+0000","last_became_peered":"2024-05-09T06:36:06.866220+0000","last_unstale":"2024-05-09T06:36:07.857540+0000","last_undegraded":"2024-05-09T06:36:07.857540+0000","last_fullsized":"2024-05-09T06:36:07.857540+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_clean_scrub_stamp":"2024-05-09T06:36:05.833229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:40:41.085704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T06:36:07.881032+0000","last_change":"2024-05-09T06:36:07.877537+0000","last_active":"2024-05-09T06:36:07.881032+0000","last_peered":"2024-05-09T06:36:07.881032+0000","last_clean":"2024-05-09T06:36:07.881032+0000","last_became_active":"2024-05-09T06:36:07.877261+0000","last_became_peered":"2024-05-09T06:36:07.877261+0000","last_unstale":"2024-05-09T06:36:07.881032+0000","last_undegraded":"2024-05-09T06:36:07.881032+0000","last_fullsized":"2024-05-09T06:36:07.881032+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:41:33.459622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T06:36:07.866215+0000","last_change":"2024-05-09T06:36:06.864863+0000","last_active":"2024-05-09T06:36:07.866215+0000","last_peered":"2024-05-09T06:36:07.866215+0000","last_clean":"2024-05-09T06:36:07.866215+0000","last_became_active":"2024-05-09T06:36:06.864300+0000","last_became_peered":"2024-05-09T06:36:06.864300+0000","last_unstale":"2024-05-09T06:36:07.866215+0000","last_undegraded":"2024-05-09T06:36:07.866215+0000","last_fullsized":"2024-05-09T06:36:07.866215+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_clean_scrub_stamp":"2024-05-09T06:36:05.833229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:15:49.248390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T06:36:07.883809+0000","last_change":"2024-05-09T06:36:07.877642+0000","last_active":"2024-05-09T06:36:07.883809+0000","last_peered":"2024-05-09T06:36:07.883809+0000","last_clean":"2024-05-09T06:36:07.883809+0000","last_became_active":"2024-05-09T06:36:07.877358+0000","last_became_peered":"2024-05-09T06:36:07.877358+0000","last_unstale":"2024-05-09T06:36:07.883809+0000","last_undegraded":"2024-05-09T06:36:07.883809+0000","last_fullsized":"2024-05-09T06:36:07.883809+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:03:39.564328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":15,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T06:36:07.879614+0000","last_change":"2024-05-09T06:36:06.865905+0000","last_active":"2024-05-09T06:36:07.879614+0000","last_peered":"2024-05-09T06:36:07.879614+0000","last_clean":"2024-05-09T06:36:07.879614+0000","last_became_active":"2024-05-09T06:36:06.865528+0000","last_became_peered":"2024-05-09T06:36:06.865528+0000","last_unstale":"2024-05-09T06:36:07.879614+0000","last_undegraded":"2024-05-09T06:36:07.879614+0000","last_fullsized":"2024-05-09T06:36:07.879614+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_clean_scrub_stamp":"2024-05-09T06:36:05.833229+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:40:41.085704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T06:36:07.868158+0000","last_change":"2024-05-09T06:36:07.864517+0000","last_active":"2024-05-09T06:36:07.868158+0000","last_peered":"2024-05-09T06:36:07.868158+0000","last_clean":"2024-05-09T06:36:07.868158+0000","last_became_active":"2024-05-09T06:36:07.864076+0000","last_became_peered":"2024-05-09T06:36:07.864076+0000","last_unstale":"2024-05-09T06:36:07.868158+0000","last_undegraded":"2024-05-09T06:36:07.868158+0000","last_fullsized":"2024-05-09T06:36:07.868158+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:08:21.009670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":18,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T06:36:07.887025+0000","last_change":"2024-05-09T06:36:06.862854+0000","last_active":"2024-05-09T06:36:07.887025+0000","last_peered":"2024-05-09T06:36:07.887025+0000","last_clean":"2024-05-09T06:36:07.887025+0000","last_became_active":"2024-05-09T06:36:06.862538+0000","last_became_peered":"2024-05-09T06:36:06.862538+0000","last_unstale":"2024-05-09T06:36:07.887025+0000","last_undegraded":"2024-05-09T06:36:07.887025+0000","last_fullsized":"2024-05-09T06:36:07.887025+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_clean_scrub_stamp":"2024-05-09T06:36:05.833229+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:08:20.004456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T06:36:07.873153+0000","last_change":"2024-05-09T06:36:07.872477+0000","last_active":"2024-05-09T06:36:07.873153+0000","last_peered":"2024-05-09T06:36:07.873153+0000","last_clean":"2024-05-09T06:36:07.873153+0000","last_became_active":"2024-05-09T06:36:07.872181+0000","last_became_peered":"2024-05-09T06:36:07.872181+0000","last_unstale":"2024-05-09T06:36:07.873153+0000","last_undegraded":"2024-05-09T06:36:07.873153+0000","last_fullsized":"2024-05-09T06:36:07.873153+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:44:00.455366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T06:36:07.874818+0000","last_change":"2024-05-09T06:36:06.861757+0000","last_active":"2024-05-09T06:36:07.874818+0000","last_peered":"2024-05-09T06:36:07.874818+0000","last_clean":"2024-05-09T06:36:07.874818+0000","last_became_active":"2024-05-09T06:36:06.861434+0000","last_became_peered":"2024-05-09T06:36:06.861434+0000","last_unstale":"2024-05-09T06:36:07.874818+0000","last_undegraded":"2024-05-09T06:36:07.874818+0000","last_fullsized":"2024-05-09T06:36:07.874818+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_clean_scrub_stamp":"2024-05-09T06:36:05.833229+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:56:44.324093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T06:36:07.882619+0000","last_change":"2024-05-09T06:36:07.873646+0000","last_active":"2024-05-09T06:36:07.882619+0000","last_peered":"2024-05-09T06:36:07.882619+0000","last_clean":"2024-05-09T06:36:07.882619+0000","last_became_active":"2024-05-09T06:36:07.872272+0000","last_became_peered":"2024-05-09T06:36:07.872272+0000","last_unstale":"2024-05-09T06:36:07.882619+0000","last_undegraded":"2024-05-09T06:36:07.882619+0000","last_fullsized":"2024-05-09T06:36:07.882619+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:40:42.090918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T06:36:07.871242+0000","last_change":"2024-05-09T06:36:06.865709+0000","last_active":"2024-05-09T06:36:07.871242+0000","last_peered":"2024-05-09T06:36:07.871242+0000","last_clean":"2024-05-09T06:36:07.871242+0000","last_became_active":"2024-05-09T06:36:06.864907+0000","last_became_peered":"2024-05-09T06:36:06.864907+0000","last_unstale":"2024-05-09T06:36:07.871242+0000","last_undegraded":"2024-05-09T06:36:07.871242+0000","last_fullsized":"2024-05-09T06:36:07.871242+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_clean_scrub_stamp":"2024-05-09T06:36:05.833229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:40:41.085704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T06:36:07.875323+0000","last_change":"2024-05-09T06:36:07.873795+0000","last_active":"2024-05-09T06:36:07.875323+0000","last_peered":"2024-05-09T06:36:07.875323+0000","last_clean":"2024-05-09T06:36:07.875323+0000","last_became_active":"2024-05-09T06:36:07.873491+0000","last_became_peered":"2024-05-09T06:36:07.873491+0000","last_unstale":"2024-05-09T06:36:07.875323+0000","last_undegraded":"2024-05-09T06:36:07.875323+0000","last_fullsized":"2024-05-09T06:36:07.875323+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:34:13.971998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T06:36:07.855945+0000","last_change":"2024-05-09T06:36:06.862067+0000","last_active":"2024-05-09T06:36:07.855945+0000","last_peered":"2024-05-09T06:36:07.855945+0000","last_clean":"2024-05-09T06:36:07.855945+0000","last_became_active":"2024-05-09T06:36:06.861791+0000","last_became_peered":"2024-05-09T06:36:06.861791+0000","last_unstale":"2024-05-09T06:36:07.855945+0000","last_undegraded":"2024-05-09T06:36:07.855945+0000","last_fullsized":"2024-05-09T06:36:07.855945+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_clean_scrub_stamp":"2024-05-09T06:36:05.833229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:03:17.189398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T06:36:07.870516+0000","last_change":"2024-05-09T06:36:07.867637+0000","last_active":"2024-05-09T06:36:07.870516+0000","last_peered":"2024-05-09T06:36:07.870516+0000","last_clean":"2024-05-09T06:36:07.870516+0000","last_became_active":"2024-05-09T06:36:07.867028+0000","last_became_peered":"2024-05-09T06:36:07.867028+0000","last_unstale":"2024-05-09T06:36:07.870516+0000","last_undegraded":"2024-05-09T06:36:07.870516+0000","last_fullsized":"2024-05-09T06:36:07.870516+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:58:19.164284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":20,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T06:36:07.884338+0000","last_change":"2024-05-09T06:36:06.852528+0000","last_active":"2024-05-09T06:36:07.884338+0000","last_peered":"2024-05-09T06:36:07.884338+0000","last_clean":"2024-05-09T06:36:07.884338+0000","last_became_active":"2024-05-09T06:36:06.852202+0000","last_became_peered":"2024-05-09T06:36:06.852202+0000","last_unstale":"2024-05-09T06:36:07.884338+0000","last_undegraded":"2024-05-09T06:36:07.884338+0000","last_fullsized":"2024-05-09T06:36:07.884338+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_clean_scrub_stamp":"2024-05-09T06:36:05.833229+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:08:20.004456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T06:36:07.869370+0000","last_change":"2024-05-09T06:36:07.867532+0000","last_active":"2024-05-09T06:36:07.869370+0000","last_peered":"2024-05-09T06:36:07.869370+0000","last_clean":"2024-05-09T06:36:07.869370+0000","last_became_active":"2024-05-09T06:36:07.867262+0000","last_became_peered":"2024-05-09T06:36:07.867262+0000","last_unstale":"2024-05-09T06:36:07.869370+0000","last_undegraded":"2024-05-09T06:36:07.869370+0000","last_fullsized":"2024-05-09T06:36:07.869370+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:47:56.779394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T06:36:07.866872+0000","last_change":"2024-05-09T06:36:06.863522+0000","last_active":"2024-05-09T06:36:07.866872+0000","last_peered":"2024-05-09T06:36:07.866872+0000","last_clean":"2024-05-09T06:36:07.866872+0000","last_became_active":"2024-05-09T06:36:06.863099+0000","last_became_peered":"2024-05-09T06:36:06.863099+0000","last_unstale":"2024-05-09T06:36:07.866872+0000","last_undegraded":"2024-05-09T06:36:07.866872+0000","last_fullsized":"2024-05-09T06:36:07.866872+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_clean_scrub_stamp":"2024-05-09T06:36:05.833229+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:56:44.324093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T06:36:07.869347+0000","last_change":"2024-05-09T06:36:07.868102+0000","last_active":"2024-05-09T06:36:07.869347+0000","last_peered":"2024-05-09T06:36:07.869347+0000","last_clean":"2024-05-09T06:36:07.869347+0000","last_became_active":"2024-05-09T06:36:07.867800+0000","last_became_peered":"2024-05-09T06:36:07.867800+0000","last_unstale":"2024-05-09T06:36:07.869347+0000","last_undegraded":"2024-05-09T06:36:07.869347+0000","last_fullsized":"2024-05-09T06:36:07.869347+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:41:33.459622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T06:36:07.883152+0000","last_change":"2024-05-09T06:36:06.861458+0000","last_active":"2024-05-09T06:36:07.883152+0000","last_peered":"2024-05-09T06:36:07.883152+0000","last_clean":"2024-05-09T06:36:07.883152+0000","last_became_active":"2024-05-09T06:36:06.861098+0000","last_became_peered":"2024-05-09T06:36:06.861098+0000","last_unstale":"2024-05-09T06:36:07.883152+0000","last_undegraded":"2024-05-09T06:36:07.883152+0000","last_fullsized":"2024-05-09T06:36:07.883152+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_clean_scrub_stamp":"2024-05-09T06:36:05.833229+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:15:49.248390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T06:36:07.874654+0000","last_change":"2024-05-09T06:36:07.868487+0000","last_active":"2024-05-09T06:36:07.874654+0000","last_peered":"2024-05-09T06:36:07.874654+0000","last_clean":"2024-05-09T06:36:07.874654+0000","last_became_active":"2024-05-09T06:36:07.867705+0000","last_became_peered":"2024-05-09T06:36:07.867705+0000","last_unstale":"2024-05-09T06:36:07.874654+0000","last_undegraded":"2024-05-09T06:36:07.874654+0000","last_fullsized":"2024-05-09T06:36:07.874654+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:53:50.215704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T06:36:07.879521+0000","last_change":"2024-05-09T06:36:06.858041+0000","last_active":"2024-05-09T06:36:07.879521+0000","last_peered":"2024-05-09T06:36:07.879521+0000","last_clean":"2024-05-09T06:36:07.879521+0000","last_became_active":"2024-05-09T06:36:06.857745+0000","last_became_peered":"2024-05-09T06:36:06.857745+0000","last_unstale":"2024-05-09T06:36:07.879521+0000","last_undegraded":"2024-05-09T06:36:07.879521+0000","last_fullsized":"2024-05-09T06:36:07.879521+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_clean_scrub_stamp":"2024-05-09T06:36:05.833229+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:58:18.159070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T06:36:07.880969+0000","last_change":"2024-05-09T06:36:07.876459+0000","last_active":"2024-05-09T06:36:07.880969+0000","last_peered":"2024-05-09T06:36:07.880969+0000","last_clean":"2024-05-09T06:36:07.880969+0000","last_became_active":"2024-05-09T06:36:07.876151+0000","last_became_peered":"2024-05-09T06:36:07.876151+0000","last_unstale":"2024-05-09T06:36:07.880969+0000","last_undegraded":"2024-05-09T06:36:07.880969+0000","last_fullsized":"2024-05-09T06:36:07.880969+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:39:07.691670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T06:36:07.856918+0000","last_change":"2024-05-09T06:36:06.866239+0000","last_active":"2024-05-09T06:36:07.856918+0000","last_peered":"2024-05-09T06:36:07.856918+0000","last_clean":"2024-05-09T06:36:07.856918+0000","last_became_active":"2024-05-09T06:36:06.865929+0000","last_became_peered":"2024-05-09T06:36:06.865929+0000","last_unstale":"2024-05-09T06:36:07.856918+0000","last_undegraded":"2024-05-09T06:36:07.856918+0000","last_fullsized":"2024-05-09T06:36:07.856918+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_clean_scrub_stamp":"2024-05-09T06:36:05.833229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:03:17.189398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T06:36:07.885088+0000","last_change":"2024-05-09T06:36:07.877578+0000","last_active":"2024-05-09T06:36:07.885088+0000","last_peered":"2024-05-09T06:36:07.885088+0000","last_clean":"2024-05-09T06:36:07.885088+0000","last_became_active":"2024-05-09T06:36:07.877319+0000","last_became_peered":"2024-05-09T06:36:07.877319+0000","last_unstale":"2024-05-09T06:36:07.885088+0000","last_undegraded":"2024-05-09T06:36:07.885088+0000","last_fullsized":"2024-05-09T06:36:07.885088+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:19:58.370458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":168,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T06:36:07.858464+0000","last_change":"2024-05-09T06:35:55.895214+0000","last_active":"2024-05-09T06:36:07.858464+0000","last_peered":"2024-05-09T06:36:07.858464+0000","last_clean":"2024-05-09T06:36:07.858464+0000","last_became_active":"2024-05-09T06:35:55.894087+0000","last_became_peered":"2024-05-09T06:35:55.894087+0000","last_unstale":"2024-05-09T06:36:07.858464+0000","last_undegraded":"2024-05-09T06:36:07.858464+0000","last_fullsized":"2024-05-09T06:36:07.858464+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:35:54.820102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:35:54.820102+0000","last_clean_scrub_stamp":"2024-05-09T06:35:54.820102+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:29:20.408099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T06:36:07.859959+0000","last_change":"2024-05-09T06:36:06.864900+0000","last_active":"2024-05-09T06:36:07.859959+0000","last_peered":"2024-05-09T06:36:07.859959+0000","last_clean":"2024-05-09T06:36:07.859959+0000","last_became_active":"2024-05-09T06:36:06.864169+0000","last_became_peered":"2024-05-09T06:36:06.864169+0000","last_unstale":"2024-05-09T06:36:07.859959+0000","last_undegraded":"2024-05-09T06:36:07.859959+0000","last_fullsized":"2024-05-09T06:36:07.859959+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_clean_scrub_stamp":"2024-05-09T06:36:05.833229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:03:17.189398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T06:36:07.886527+0000","last_change":"2024-05-09T06:36:07.880436+0000","last_active":"2024-05-09T06:36:07.886527+0000","last_peered":"2024-05-09T06:36:07.886527+0000","last_clean":"2024-05-09T06:36:07.886527+0000","last_became_active":"2024-05-09T06:36:07.879931+0000","last_became_peered":"2024-05-09T06:36:07.879931+0000","last_unstale":"2024-05-09T06:36:07.886527+0000","last_undegraded":"2024-05-09T06:36:07.886527+0000","last_fullsized":"2024-05-09T06:36:07.886527+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:53:50.215704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T06:36:07.858392+0000","last_change":"2024-05-09T06:36:06.860567+0000","last_active":"2024-05-09T06:36:07.858392+0000","last_peered":"2024-05-09T06:36:07.858392+0000","last_clean":"2024-05-09T06:36:07.858392+0000","last_became_active":"2024-05-09T06:36:06.860284+0000","last_became_peered":"2024-05-09T06:36:06.860284+0000","last_unstale":"2024-05-09T06:36:07.858392+0000","last_undegraded":"2024-05-09T06:36:07.858392+0000","last_fullsized":"2024-05-09T06:36:07.858392+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_clean_scrub_stamp":"2024-05-09T06:36:05.833229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:03:38.559114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T06:36:07.874795+0000","last_change":"2024-05-09T06:36:07.865957+0000","last_active":"2024-05-09T06:36:07.874795+0000","last_peered":"2024-05-09T06:36:07.874795+0000","last_clean":"2024-05-09T06:36:07.874795+0000","last_became_active":"2024-05-09T06:36:07.865683+0000","last_became_peered":"2024-05-09T06:36:07.865683+0000","last_unstale":"2024-05-09T06:36:07.874795+0000","last_undegraded":"2024-05-09T06:36:07.874795+0000","last_fullsized":"2024-05-09T06:36:07.874795+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:08:21.009670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T06:36:07.856303+0000","last_change":"2024-05-09T06:36:06.862232+0000","last_active":"2024-05-09T06:36:07.856303+0000","last_peered":"2024-05-09T06:36:07.856303+0000","last_clean":"2024-05-09T06:36:07.856303+0000","last_became_active":"2024-05-09T06:36:06.861918+0000","last_became_peered":"2024-05-09T06:36:06.861918+0000","last_unstale":"2024-05-09T06:36:07.856303+0000","last_undegraded":"2024-05-09T06:36:07.856303+0000","last_fullsized":"2024-05-09T06:36:07.856303+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_clean_scrub_stamp":"2024-05-09T06:36:05.833229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:23:13.976894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T06:36:07.871659+0000","last_change":"2024-05-09T06:36:07.865850+0000","last_active":"2024-05-09T06:36:07.871659+0000","last_peered":"2024-05-09T06:36:07.871659+0000","last_clean":"2024-05-09T06:36:07.871659+0000","last_became_active":"2024-05-09T06:36:07.865520+0000","last_became_peered":"2024-05-09T06:36:07.865520+0000","last_unstale":"2024-05-09T06:36:07.871659+0000","last_undegraded":"2024-05-09T06:36:07.871659+0000","last_fullsized":"2024-05-09T06:36:07.871659+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:41:33.459622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T06:36:07.871402+0000","last_change":"2024-05-09T06:36:06.865545+0000","last_active":"2024-05-09T06:36:07.871402+0000","last_peered":"2024-05-09T06:36:07.871402+0000","last_clean":"2024-05-09T06:36:07.871402+0000","last_became_active":"2024-05-09T06:36:06.865255+0000","last_became_peered":"2024-05-09T06:36:06.865255+0000","last_unstale":"2024-05-09T06:36:07.871402+0000","last_undegraded":"2024-05-09T06:36:07.871402+0000","last_fullsized":"2024-05-09T06:36:07.871402+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_clean_scrub_stamp":"2024-05-09T06:36:05.833229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:59:29.736305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T06:36:07.870022+0000","last_change":"2024-05-09T06:36:07.867264+0000","last_active":"2024-05-09T06:36:07.870022+0000","last_peered":"2024-05-09T06:36:07.870022+0000","last_clean":"2024-05-09T06:36:07.870022+0000","last_became_active":"2024-05-09T06:36:07.866981+0000","last_became_peered":"2024-05-09T06:36:07.866981+0000","last_unstale":"2024-05-09T06:36:07.870022+0000","last_undegraded":"2024-05-09T06:36:07.870022+0000","last_fullsized":"2024-05-09T06:36:07.870022+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:03:18.194613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T06:36:07.852112+0000","last_change":"2024-05-09T06:36:06.853399+0000","last_active":"2024-05-09T06:36:07.852112+0000","last_peered":"2024-05-09T06:36:07.852112+0000","last_clean":"2024-05-09T06:36:07.852112+0000","last_became_active":"2024-05-09T06:36:06.852409+0000","last_became_peered":"2024-05-09T06:36:06.852409+0000","last_unstale":"2024-05-09T06:36:07.852112+0000","last_undegraded":"2024-05-09T06:36:07.852112+0000","last_fullsized":"2024-05-09T06:36:07.852112+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_clean_scrub_stamp":"2024-05-09T06:36:05.833229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:15:49.248390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T06:36:07.880913+0000","last_change":"2024-05-09T06:36:07.875455+0000","last_active":"2024-05-09T06:36:07.880913+0000","last_peered":"2024-05-09T06:36:07.880913+0000","last_clean":"2024-05-09T06:36:07.880913+0000","last_became_active":"2024-05-09T06:36:07.875153+0000","last_became_peered":"2024-05-09T06:36:07.875153+0000","last_unstale":"2024-05-09T06:36:07.880913+0000","last_undegraded":"2024-05-09T06:36:07.880913+0000","last_fullsized":"2024-05-09T06:36:07.880913+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:03:39.564328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T06:36:07.856427+0000","last_change":"2024-05-09T06:36:06.861152+0000","last_active":"2024-05-09T06:36:07.856427+0000","last_peered":"2024-05-09T06:36:07.856427+0000","last_clean":"2024-05-09T06:36:07.856427+0000","last_became_active":"2024-05-09T06:36:06.860905+0000","last_became_peered":"2024-05-09T06:36:06.860905+0000","last_unstale":"2024-05-09T06:36:07.856427+0000","last_undegraded":"2024-05-09T06:36:07.856427+0000","last_fullsized":"2024-05-09T06:36:07.856427+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_clean_scrub_stamp":"2024-05-09T06:36:05.833229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:59:29.736305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T06:36:07.871597+0000","last_change":"2024-05-09T06:36:07.865602+0000","last_active":"2024-05-09T06:36:07.871597+0000","last_peered":"2024-05-09T06:36:07.871597+0000","last_clean":"2024-05-09T06:36:07.871597+0000","last_became_active":"2024-05-09T06:36:07.865083+0000","last_became_peered":"2024-05-09T06:36:07.865083+0000","last_unstale":"2024-05-09T06:36:07.871597+0000","last_undegraded":"2024-05-09T06:36:07.871597+0000","last_fullsized":"2024-05-09T06:36:07.871597+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:47:56.779394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T06:36:07.852890+0000","last_change":"2024-05-09T06:36:06.855928+0000","last_active":"2024-05-09T06:36:07.852890+0000","last_peered":"2024-05-09T06:36:07.852890+0000","last_clean":"2024-05-09T06:36:07.852890+0000","last_became_active":"2024-05-09T06:36:06.855322+0000","last_became_peered":"2024-05-09T06:36:06.855322+0000","last_unstale":"2024-05-09T06:36:07.852890+0000","last_undegraded":"2024-05-09T06:36:07.852890+0000","last_fullsized":"2024-05-09T06:36:07.852890+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_clean_scrub_stamp":"2024-05-09T06:36:05.833229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:03:38.559114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T06:36:07.875474+0000","last_change":"2024-05-09T06:36:07.870976+0000","last_active":"2024-05-09T06:36:07.875474+0000","last_peered":"2024-05-09T06:36:07.875474+0000","last_clean":"2024-05-09T06:36:07.875474+0000","last_became_active":"2024-05-09T06:36:07.870713+0000","last_became_peered":"2024-05-09T06:36:07.870713+0000","last_unstale":"2024-05-09T06:36:07.875474+0000","last_undegraded":"2024-05-09T06:36:07.875474+0000","last_fullsized":"2024-05-09T06:36:07.875474+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:03:39.564328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T06:36:07.874941+0000","last_change":"2024-05-09T06:36:06.863328+0000","last_active":"2024-05-09T06:36:07.874941+0000","last_peered":"2024-05-09T06:36:07.874941+0000","last_clean":"2024-05-09T06:36:07.874941+0000","last_became_active":"2024-05-09T06:36:06.862686+0000","last_became_peered":"2024-05-09T06:36:06.862686+0000","last_unstale":"2024-05-09T06:36:07.874941+0000","last_undegraded":"2024-05-09T06:36:07.874941+0000","last_fullsized":"2024-05-09T06:36:07.874941+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_clean_scrub_stamp":"2024-05-09T06:36:05.833229+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:59:29.736305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T06:36:07.882438+0000","last_change":"2024-05-09T06:36:07.868926+0000","last_active":"2024-05-09T06:36:07.882438+0000","last_peered":"2024-05-09T06:36:07.882438+0000","last_clean":"2024-05-09T06:36:07.882438+0000","last_became_active":"2024-05-09T06:36:07.868613+0000","last_became_peered":"2024-05-09T06:36:07.868613+0000","last_unstale":"2024-05-09T06:36:07.882438+0000","last_undegraded":"2024-05-09T06:36:07.882438+0000","last_fullsized":"2024-05-09T06:36:07.882438+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:03:18.194613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T06:36:07.858225+0000","last_change":"2024-05-09T06:36:06.861104+0000","last_active":"2024-05-09T06:36:07.858225+0000","last_peered":"2024-05-09T06:36:07.858225+0000","last_clean":"2024-05-09T06:36:07.858225+0000","last_became_active":"2024-05-09T06:36:06.860799+0000","last_became_peered":"2024-05-09T06:36:06.860799+0000","last_unstale":"2024-05-09T06:36:07.858225+0000","last_undegraded":"2024-05-09T06:36:07.858225+0000","last_fullsized":"2024-05-09T06:36:07.858225+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_clean_scrub_stamp":"2024-05-09T06:36:05.833229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:58:18.159070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T06:36:07.866537+0000","last_change":"2024-05-09T06:36:07.863846+0000","last_active":"2024-05-09T06:36:07.866537+0000","last_peered":"2024-05-09T06:36:07.866537+0000","last_clean":"2024-05-09T06:36:07.866537+0000","last_became_active":"2024-05-09T06:36:07.863161+0000","last_became_peered":"2024-05-09T06:36:07.863161+0000","last_unstale":"2024-05-09T06:36:07.866537+0000","last_undegraded":"2024-05-09T06:36:07.866537+0000","last_fullsized":"2024-05-09T06:36:07.866537+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:23:14.982108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T06:36:07.852530+0000","last_change":"2024-05-09T06:36:06.861276+0000","last_active":"2024-05-09T06:36:07.852530+0000","last_peered":"2024-05-09T06:36:07.852530+0000","last_clean":"2024-05-09T06:36:07.852530+0000","last_became_active":"2024-05-09T06:36:06.860977+0000","last_became_peered":"2024-05-09T06:36:06.860977+0000","last_unstale":"2024-05-09T06:36:07.852530+0000","last_undegraded":"2024-05-09T06:36:07.852530+0000","last_fullsized":"2024-05-09T06:36:07.852530+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_clean_scrub_stamp":"2024-05-09T06:36:05.833229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:16:34.559063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T06:36:07.875756+0000","last_change":"2024-05-09T06:36:07.870964+0000","last_active":"2024-05-09T06:36:07.875756+0000","last_peered":"2024-05-09T06:36:07.875756+0000","last_clean":"2024-05-09T06:36:07.875756+0000","last_became_active":"2024-05-09T06:36:07.870669+0000","last_became_peered":"2024-05-09T06:36:07.870669+0000","last_unstale":"2024-05-09T06:36:07.875756+0000","last_undegraded":"2024-05-09T06:36:07.875756+0000","last_fullsized":"2024-05-09T06:36:07.875756+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:58:19.164284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T06:36:07.852209+0000","last_change":"2024-05-09T06:36:06.861162+0000","last_active":"2024-05-09T06:36:07.852209+0000","last_peered":"2024-05-09T06:36:07.852209+0000","last_clean":"2024-05-09T06:36:07.852209+0000","last_became_active":"2024-05-09T06:36:06.860880+0000","last_became_peered":"2024-05-09T06:36:06.860880+0000","last_unstale":"2024-05-09T06:36:07.852209+0000","last_undegraded":"2024-05-09T06:36:07.852209+0000","last_fullsized":"2024-05-09T06:36:07.852209+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_clean_scrub_stamp":"2024-05-09T06:36:05.833229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:59:29.736305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T06:36:07.877210+0000","last_change":"2024-05-09T06:36:07.872872+0000","last_active":"2024-05-09T06:36:07.877210+0000","last_peered":"2024-05-09T06:36:07.877210+0000","last_clean":"2024-05-09T06:36:07.877210+0000","last_became_active":"2024-05-09T06:36:07.872556+0000","last_became_peered":"2024-05-09T06:36:07.872556+0000","last_unstale":"2024-05-09T06:36:07.877210+0000","last_undegraded":"2024-05-09T06:36:07.877210+0000","last_fullsized":"2024-05-09T06:36:07.877210+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:47:56.779394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T06:36:07.877875+0000","last_change":"2024-05-09T06:36:06.862160+0000","last_active":"2024-05-09T06:36:07.877875+0000","last_peered":"2024-05-09T06:36:07.877875+0000","last_clean":"2024-05-09T06:36:07.877875+0000","last_became_active":"2024-05-09T06:36:06.861881+0000","last_became_peered":"2024-05-09T06:36:06.861881+0000","last_unstale":"2024-05-09T06:36:07.877875+0000","last_undegraded":"2024-05-09T06:36:07.877875+0000","last_fullsized":"2024-05-09T06:36:07.877875+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_clean_scrub_stamp":"2024-05-09T06:36:05.833229+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:16:34.559063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T06:36:07.873367+0000","last_change":"2024-05-09T06:36:07.869000+0000","last_active":"2024-05-09T06:36:07.873367+0000","last_peered":"2024-05-09T06:36:07.873367+0000","last_clean":"2024-05-09T06:36:07.873367+0000","last_became_active":"2024-05-09T06:36:07.868522+0000","last_became_peered":"2024-05-09T06:36:07.868522+0000","last_unstale":"2024-05-09T06:36:07.873367+0000","last_undegraded":"2024-05-09T06:36:07.873367+0000","last_fullsized":"2024-05-09T06:36:07.873367+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:59:57.900963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T06:36:07.880954+0000","last_change":"2024-05-09T06:36:07.875476+0000","last_active":"2024-05-09T06:36:07.880954+0000","last_peered":"2024-05-09T06:36:07.880954+0000","last_clean":"2024-05-09T06:36:07.880954+0000","last_became_active":"2024-05-09T06:36:07.875170+0000","last_became_peered":"2024-05-09T06:36:07.875170+0000","last_unstale":"2024-05-09T06:36:07.880954+0000","last_undegraded":"2024-05-09T06:36:07.880954+0000","last_fullsized":"2024-05-09T06:36:07.880954+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:58:19.164284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T06:36:07.880546+0000","last_change":"2024-05-09T06:36:06.866061+0000","last_active":"2024-05-09T06:36:07.880546+0000","last_peered":"2024-05-09T06:36:07.880546+0000","last_clean":"2024-05-09T06:36:07.880546+0000","last_became_active":"2024-05-09T06:36:06.865783+0000","last_became_peered":"2024-05-09T06:36:06.865783+0000","last_unstale":"2024-05-09T06:36:07.880546+0000","last_undegraded":"2024-05-09T06:36:07.880546+0000","last_fullsized":"2024-05-09T06:36:07.880546+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_clean_scrub_stamp":"2024-05-09T06:36:05.833229+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:15:49.248390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T06:36:07.880933+0000","last_change":"2024-05-09T06:36:07.876132+0000","last_active":"2024-05-09T06:36:07.880933+0000","last_peered":"2024-05-09T06:36:07.880933+0000","last_clean":"2024-05-09T06:36:07.880933+0000","last_became_active":"2024-05-09T06:36:07.874679+0000","last_became_peered":"2024-05-09T06:36:07.874679+0000","last_unstale":"2024-05-09T06:36:07.880933+0000","last_undegraded":"2024-05-09T06:36:07.880933+0000","last_fullsized":"2024-05-09T06:36:07.880933+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:59:57.900963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T06:36:07.874964+0000","last_change":"2024-05-09T06:36:06.860619+0000","last_active":"2024-05-09T06:36:07.874964+0000","last_peered":"2024-05-09T06:36:07.874964+0000","last_clean":"2024-05-09T06:36:07.874964+0000","last_became_active":"2024-05-09T06:36:06.860323+0000","last_became_peered":"2024-05-09T06:36:06.860323+0000","last_unstale":"2024-05-09T06:36:07.874964+0000","last_undegraded":"2024-05-09T06:36:07.874964+0000","last_fullsized":"2024-05-09T06:36:07.874964+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_clean_scrub_stamp":"2024-05-09T06:36:05.833229+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:40:41.085704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T06:36:07.883896+0000","last_change":"2024-05-09T06:36:07.880245+0000","last_active":"2024-05-09T06:36:07.883896+0000","last_peered":"2024-05-09T06:36:07.883896+0000","last_clean":"2024-05-09T06:36:07.883896+0000","last_became_active":"2024-05-09T06:36:07.879926+0000","last_became_peered":"2024-05-09T06:36:07.879926+0000","last_unstale":"2024-05-09T06:36:07.883896+0000","last_undegraded":"2024-05-09T06:36:07.883896+0000","last_fullsized":"2024-05-09T06:36:07.883896+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:59:57.900963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T06:36:07.878622+0000","last_change":"2024-05-09T06:36:06.864934+0000","last_active":"2024-05-09T06:36:07.878622+0000","last_peered":"2024-05-09T06:36:07.878622+0000","last_clean":"2024-05-09T06:36:07.878622+0000","last_became_active":"2024-05-09T06:36:06.864680+0000","last_became_peered":"2024-05-09T06:36:06.864680+0000","last_unstale":"2024-05-09T06:36:07.878622+0000","last_undegraded":"2024-05-09T06:36:07.878622+0000","last_fullsized":"2024-05-09T06:36:07.878622+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_clean_scrub_stamp":"2024-05-09T06:36:05.833229+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:03:17.189398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T06:36:07.877563+0000","last_change":"2024-05-09T06:36:07.870194+0000","last_active":"2024-05-09T06:36:07.877563+0000","last_peered":"2024-05-09T06:36:07.877563+0000","last_clean":"2024-05-09T06:36:07.877563+0000","last_became_active":"2024-05-09T06:36:07.869895+0000","last_became_peered":"2024-05-09T06:36:07.869895+0000","last_unstale":"2024-05-09T06:36:07.877563+0000","last_undegraded":"2024-05-09T06:36:07.877563+0000","last_fullsized":"2024-05-09T06:36:07.877563+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:59:30.741519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":16,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T06:36:07.875880+0000","last_change":"2024-05-09T06:36:06.861090+0000","last_active":"2024-05-09T06:36:07.875880+0000","last_peered":"2024-05-09T06:36:07.875880+0000","last_clean":"2024-05-09T06:36:07.875880+0000","last_became_active":"2024-05-09T06:36:06.860774+0000","last_became_peered":"2024-05-09T06:36:06.860774+0000","last_unstale":"2024-05-09T06:36:07.875880+0000","last_undegraded":"2024-05-09T06:36:07.875880+0000","last_fullsized":"2024-05-09T06:36:07.875880+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_clean_scrub_stamp":"2024-05-09T06:36:05.833229+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:05:58.090556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27792,"kb_used_data":772,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344048,"statfs":{"total":96636764160,"available":96608305152,"internally_reserved":0,"allocated":790528,"data_stored":655584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574854,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":204800,"data_stored":66246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574853,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27212,"kb_used_data":192,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344628,"statfs":{"total":96636764160,"available":96608899072,"internally_reserved":0,"allocated":196608,"data_stored":64724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574853,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":204800,"data_stored":64760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":204800,"data_stored":66246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574853,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27208,"kb_used_data":188,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344632,"statfs":{"total":96636764160,"available":96608903168,"internally_reserved":0,"allocated":192512,"data_stored":64634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574853,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27216,"kb_used_data":196,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344624,"statfs":{"total":96636764160,"available":96608894976,"internally_reserved":0,"allocated":200704,"data_stored":64794,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27804,"kb_used_data":780,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344036,"statfs":{"total":96636764160,"available":96608292864,"internally_reserved":0,"allocated":798720,"data_stored":655640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-09T06:36:12.219 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-05-09T06:36:12.219 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-05-09T06:36:12.219 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-05-09T06:36:12.219 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-05-09T06:36:12.219 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-05-09T06:36:12.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.379+0000 7f1709200640 1 Processor -- start 2024-05-09T06:36:12.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.379+0000 7f1709200640 1 -- start start 2024-05-09T06:36:12.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.380+0000 7f1709200640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f170415e3c0 0x7f17040d47c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:12.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.380+0000 7f1709200640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f17040cf660 0x7f17040d09a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:12.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.380+0000 7f1709200640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f17040cfaf0 con 0x7f17040d4d00 2024-05-09T06:36:12.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.380+0000 7f1709200640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f170415e7a0 con 0x7f170415e3c0 2024-05-09T06:36:12.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.380+0000 7f1709200640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f17040d00b0 con 0x7f17040cf660 2024-05-09T06:36:12.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.380+0000 7f170359e640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f17040d4d00 0x7f17040d1c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:49196/0 (socket says 172.21.15.3:49196) 2024-05-09T06:36:12.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.380+0000 7f170359e640 1 -- 172.21.15.3:0/2422326497 learned_addr learned my addr 172.21.15.3:0/2422326497 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-05-09T06:36:12.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.380+0000 7f170259c640 1 --2- 172.21.15.3:0/2422326497 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f17040cf660 0x7f17040d09a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:12.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.380+0000 7f1702d9d640 1 --2- 172.21.15.3:0/2422326497 >> v2:172.21.15.190:3300/0 conn(0x7f170415e3c0 0x7f17040d47c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:12.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.380+0000 7f1701d9b640 1 -- 172.21.15.3:0/2422326497 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2956836161 0 0) 0x7f17040cfaf0 con 0x7f17040d4d00 2024-05-09T06:36:12.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.380+0000 7f1701d9b640 1 -- 172.21.15.3:0/2422326497 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f16ec003520 con 0x7f17040d4d00 2024-05-09T06:36:12.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.380+0000 7f1701d9b640 1 -- 172.21.15.3:0/2422326497 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2675886783 0 0) 0x7f16ec003520 con 0x7f17040d4d00 2024-05-09T06:36:12.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.380+0000 7f1701d9b640 1 -- 172.21.15.3:0/2422326497 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f17040cf660 msgr2=0x7f17040d09a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:12.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.380+0000 7f1701d9b640 1 --2- 172.21.15.3:0/2422326497 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f17040cf660 0x7f17040d09a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:12.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.380+0000 7f1701d9b640 1 -- 172.21.15.3:0/2422326497 >> v2:172.21.15.190:3300/0 conn(0x7f170415e3c0 msgr2=0x7f17040d47c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:12.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.380+0000 7f1701d9b640 1 --2- 172.21.15.3:0/2422326497 >> v2:172.21.15.190:3300/0 conn(0x7f170415e3c0 0x7f17040d47c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:12.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.380+0000 7f1701d9b640 1 -- 172.21.15.3:0/2422326497 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17040cfdd0 con 0x7f17040d4d00 2024-05-09T06:36:12.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.380+0000 7f1701d9b640 1 -- 172.21.15.3:0/2422326497 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 563312031 0 0) 0x7f16fc0019f0 con 0x7f17040d4d00 2024-05-09T06:36:12.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.380+0000 7f1701d9b640 1 -- 172.21.15.3:0/2422326497 >> v1:172.21.15.3:6789/0 conn(0x7f17040d4d00 legacy=0x7f17040d1c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:12.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.380+0000 7f1701d9b640 1 --2- 172.21.15.3:0/2422326497 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f16ec0039b0 0x7f16ec003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:12.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.380+0000 7f1701d9b640 1 --2- 172.21.15.3:0/2422326497 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f16ec004310 0x7f16ec004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:12.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.380+0000 7f1701d9b640 1 --2- 172.21.15.3:0/2422326497 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f16ec004e50 0x7f16ec0052d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:12.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.380+0000 7f1701d9b640 1 -- 172.21.15.3:0/2422326497 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f170415e7a0 con 0x7f16ec004e50 2024-05-09T06:36:12.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.380+0000 7f1701d9b640 1 -- 172.21.15.3:0/2422326497 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f16ec00bcc0 con 0x7f16ec004310 2024-05-09T06:36:12.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.380+0000 7f1701d9b640 1 -- 172.21.15.3:0/2422326497 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f16ec00bec0 con 0x7f16ec0039b0 2024-05-09T06:36:12.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.381+0000 7f170359e640 1 --2- 172.21.15.3:0/2422326497 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f16ec004e50 0x7f16ec0052d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:12.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.381+0000 7f170259c640 1 --2- 172.21.15.3:0/2422326497 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f16ec004310 0x7f16ec004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:12.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.381+0000 7f1702d9d640 1 --2- 172.21.15.3:0/2422326497 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f16ec0039b0 0x7f16ec003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:12.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.381+0000 7f170359e640 1 -- 172.21.15.3:0/2422326497 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f16ec0039b0 msgr2=0x7f16ec003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:12.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.381+0000 7f170359e640 1 --2- 172.21.15.3:0/2422326497 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f16ec0039b0 0x7f16ec003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:12.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.381+0000 7f170359e640 1 -- 172.21.15.3:0/2422326497 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f16ec004310 msgr2=0x7f16ec004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:12.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.381+0000 7f170359e640 1 --2- 172.21.15.3:0/2422326497 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f16ec004310 0x7f16ec004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:12.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.381+0000 7f170359e640 1 -- 172.21.15.3:0/2422326497 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16ec003520 con 0x7f16ec004e50 2024-05-09T06:36:12.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.381+0000 7f1702d9d640 1 --2- 172.21.15.3:0/2422326497 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f16ec0039b0 0x7f16ec003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:36:12.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.382+0000 7f170359e640 1 --2- 172.21.15.3:0/2422326497 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f16ec004e50 0x7f16ec0052d0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f170406bc80 tx=0x7f16ec005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d7a7bb6b2b6c7ca server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:12.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.382+0000 7f1701d9b640 1 -- 172.21.15.3:0/2422326497 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16fc00a500 con 0x7f16ec004e50 2024-05-09T06:36:12.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.382+0000 7f1701d9b640 1 -- 172.21.15.3:0/2422326497 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f16fc02f070 con 0x7f16ec004e50 2024-05-09T06:36:12.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.382+0000 7f1701d9b640 1 -- 172.21.15.3:0/2422326497 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16fc02bc70 con 0x7f16ec004e50 2024-05-09T06:36:12.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.382+0000 7f1709200640 1 -- 172.21.15.3:0/2422326497 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f16ec004e50 msgr2=0x7f16ec0052d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:12.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.382+0000 7f1709200640 1 --2- 172.21.15.3:0/2422326497 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f16ec004e50 0x7f16ec0052d0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f170406bc80 tx=0x7f16ec005830 comp rx=0 tx=0).stop 2024-05-09T06:36:12.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.382+0000 7f170359e640 1 -- 172.21.15.3:0/2422326497 reap_dead start 2024-05-09T06:36:12.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.382+0000 7f1709200640 1 -- 172.21.15.3:0/2422326497 shutdown_connections 2024-05-09T06:36:12.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.382+0000 7f1709200640 1 -- 172.21.15.3:0/2422326497 >> 172.21.15.3:0/2422326497 conn(0x7f170409d320 msgr2=0x7f17040d4040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:12.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.382+0000 7f1709200640 1 -- 172.21.15.3:0/2422326497 shutdown_connections 2024-05-09T06:36:12.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.382+0000 7f1709200640 1 -- 172.21.15.3:0/2422326497 wait complete. 2024-05-09T06:36:12.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.384+0000 7f1709200640 1 Processor -- start 2024-05-09T06:36:12.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.384+0000 7f1709200640 1 -- start start 2024-05-09T06:36:12.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.385+0000 7f1709200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f16ec004e50 0x7f17041f48d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:12.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.385+0000 7f1709200640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f170415c430 0x7f17040b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:12.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.385+0000 7f1709200640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f17041f41b0 0x7f17040d1d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:12.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.385+0000 7f1709200640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f17040c9540 con 0x7f16ec004e50 2024-05-09T06:36:12.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.385+0000 7f1709200640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f17040c8010 con 0x7f170415c430 2024-05-09T06:36:12.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.385+0000 7f1709200640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f17040c96c0 con 0x7f17041f41b0 2024-05-09T06:36:12.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.385+0000 7f1702d9d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f16ec004e50 0x7f17041f48d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:12.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.385+0000 7f1702d9d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f16ec004e50 0x7f17041f48d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49420/0 (socket says 172.21.15.3:49420) 2024-05-09T06:36:12.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.385+0000 7f1702d9d640 1 -- 172.21.15.3:0/1918070885 learned_addr learned my addr 172.21.15.3:0/1918070885 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:36:12.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.385+0000 7f170259c640 1 --2- 172.21.15.3:0/1918070885 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f170415c430 0x7f17040b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:12.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.385+0000 7f170359e640 1 --2- 172.21.15.3:0/1918070885 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f17041f41b0 0x7f17040d1d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:12.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.385+0000 7f1702d9d640 1 -- 172.21.15.3:0/1918070885 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f17041f41b0 msgr2=0x7f17040d1d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:12.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.385+0000 7f1702d9d640 1 --2- 172.21.15.3:0/1918070885 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f17041f41b0 0x7f17040d1d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:12.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.385+0000 7f1702d9d640 1 -- 172.21.15.3:0/1918070885 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f170415c430 msgr2=0x7f17040b8a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:12.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.385+0000 7f1702d9d640 1 --2- 172.21.15.3:0/1918070885 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f170415c430 0x7f17040b8a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:12.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.385+0000 7f1702d9d640 1 -- 172.21.15.3:0/1918070885 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17040b6c70 con 0x7f16ec004e50 2024-05-09T06:36:12.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.385+0000 7f170359e640 1 --2- 172.21.15.3:0/1918070885 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f17041f41b0 0x7f17040d1d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:36:12.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.385+0000 7f1702d9d640 1 --2- 172.21.15.3:0/1918070885 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f16ec004e50 0x7f17041f48d0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f16f4011630 tx=0x7f16f4011af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:12.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.385+0000 7f16eb7fe640 1 -- 172.21.15.3:0/1918070885 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16f400f8c0 con 0x7f16ec004e50 2024-05-09T06:36:12.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.385+0000 7f16eb7fe640 1 -- 172.21.15.3:0/1918070885 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f16f400fa60 con 0x7f16ec004e50 2024-05-09T06:36:12.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.385+0000 7f1709200640 1 -- 172.21.15.3:0/1918070885 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17040b9100 con 0x7f16ec004e50 2024-05-09T06:36:12.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.385+0000 7f16eb7fe640 1 -- 172.21.15.3:0/1918070885 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16f400fe30 con 0x7f16ec004e50 2024-05-09T06:36:12.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.385+0000 7f1709200640 1 -- 172.21.15.3:0/1918070885 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17040b96c0 con 0x7f16ec004e50 2024-05-09T06:36:12.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.386+0000 7f1709200640 1 -- 172.21.15.3:0/1918070885 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f170404f5f0 con 0x7f16ec004e50 2024-05-09T06:36:12.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.387+0000 7f16eb7fe640 1 -- 172.21.15.3:0/1918070885 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f16f4015460 con 0x7f16ec004e50 2024-05-09T06:36:12.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.387+0000 7f16eb7fe640 1 --2- 172.21.15.3:0/1918070885 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f16f006be30 0x7f16f006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:12.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.387+0000 7f16eb7fe640 1 -- 172.21.15.3:0/1918070885 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f16f408b860 con 0x7f16ec004e50 2024-05-09T06:36:12.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.388+0000 7f170259c640 1 --2- 172.21.15.3:0/1918070885 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f16f006be30 0x7f16f006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:12.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.388+0000 7f170259c640 1 --2- 172.21.15.3:0/1918070885 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f16f006be30 0x7f16f006e2f0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f170415cd50 tx=0x7f16f8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:12.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.392+0000 7f16eb7fe640 1 -- 172.21.15.3:0/1918070885 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f16f4019030 con 0x7f16ec004e50 2024-05-09T06:36:12.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.629+0000 7f1709200640 1 -- 172.21.15.3:0/1918070885 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f17040d0670 con 0x7f16ec004e50 2024-05-09T06:36:12.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.632+0000 7f16eb7fe640 1 -- 172.21.15.3:0/1918070885 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) ==== 74+0+172942 (secure 0 0 0) 0x7f16f405a170 con 0x7f16ec004e50 2024-05-09T06:36:12.658 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-09T06:36:12.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.635+0000 7f1709200640 1 -- 172.21.15.3:0/1918070885 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f16f006be30 msgr2=0x7f16f006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:12.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.635+0000 7f1709200640 1 --2- 172.21.15.3:0/1918070885 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f16f006be30 0x7f16f006e2f0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f170415cd50 tx=0x7f16f8008040 comp rx=0 tx=0).stop 2024-05-09T06:36:12.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.635+0000 7f1709200640 1 -- 172.21.15.3:0/1918070885 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f16ec004e50 msgr2=0x7f17041f48d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:12.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.635+0000 7f1709200640 1 --2- 172.21.15.3:0/1918070885 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f16ec004e50 0x7f17041f48d0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f16f4011630 tx=0x7f16f4011af0 comp rx=0 tx=0).stop 2024-05-09T06:36:12.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.635+0000 7f1709200640 1 -- 172.21.15.3:0/1918070885 shutdown_connections 2024-05-09T06:36:12.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.635+0000 7f1709200640 1 --2- 172.21.15.3:0/1918070885 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f16f006be30 0x7f16f006e2f0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:12.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.635+0000 7f1709200640 1 --2- 172.21.15.3:0/1918070885 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f17041f41b0 0x7f17040d1d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:12.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.635+0000 7f1709200640 1 --2- 172.21.15.3:0/1918070885 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f170415c430 0x7f17040b8a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:12.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.635+0000 7f1709200640 1 --2- 172.21.15.3:0/1918070885 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f16ec004e50 0x7f17041f48d0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:12.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.635+0000 7f1709200640 1 -- 172.21.15.3:0/1918070885 >> 172.21.15.3:0/1918070885 conn(0x7f170409d320 msgr2=0x7f17040ca320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:12.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.635+0000 7f1709200640 1 -- 172.21.15.3:0/1918070885 shutdown_connections 2024-05-09T06:36:12.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.635+0000 7f1709200640 1 -- 172.21.15.3:0/1918070885 wait complete. 2024-05-09T06:36:12.661 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":5,"flags":0,"active_gid":4105,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":975893321},{"type":"v1","addr":"172.21.15.190:6833","nonce":975893321}]},"active_addr":"172.21.15.190:6833/975893321","active_change":"2024-05-09T06:35:46.764079+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":4107,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.190:0","nonce":658212118}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.190:0","nonce":2567150848}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.190:0","nonce":2582713269}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.190:0","nonce":3766699314}]}]} 2024-05-09T06:36:12.665 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-05-09T06:36:12.665 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-05-09T06:36:12.666 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T06:36:12.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.789+0000 7f283346b640 1 Processor -- start 2024-05-09T06:36:12.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.790+0000 7f283346b640 1 -- start start 2024-05-09T06:36:12.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.790+0000 7f283346b640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f282c0d2c20 0x7f282c169ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:12.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.790+0000 7f283346b640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f282c0d4e50 con 0x7f282c0d27c0 2024-05-09T06:36:12.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.790+0000 7f283346b640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f282c0d0270 con 0x7f282c15e480 2024-05-09T06:36:12.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.790+0000 7f283346b640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f282c0d30d0 con 0x7f282c0d2c20 2024-05-09T06:36:12.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.791+0000 7f2830a06640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f282c0d27c0 0x7f282c166390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:49202/0 (socket says 172.21.15.3:49202) 2024-05-09T06:36:12.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.791+0000 7f2830a06640 1 -- 172.21.15.3:0/2681424646 learned_addr learned my addr 172.21.15.3:0/2681424646 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-05-09T06:36:12.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.791+0000 7f2831a08640 1 --2- 172.21.15.3:0/2681424646 >> v2:172.21.15.190:3300/0 conn(0x7f282c0d2c20 0x7f282c169ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:12.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.791+0000 7f282bfff640 1 -- 172.21.15.3:0/2681424646 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3694348721 0 0) 0x7f2820001110 con 0x7f282c0d27c0 2024-05-09T06:36:12.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.791+0000 7f282bfff640 1 -- 172.21.15.3:0/2681424646 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2814003520 con 0x7f282c0d27c0 2024-05-09T06:36:12.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.791+0000 7f282bfff640 1 -- 172.21.15.3:0/2681424646 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1335723272 0 0) 0x7f282c0d0270 con 0x7f282c15e480 2024-05-09T06:36:12.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.791+0000 7f282bfff640 1 -- 172.21.15.3:0/2681424646 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2820001110 con 0x7f282c15e480 2024-05-09T06:36:12.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.791+0000 7f282bfff640 1 -- 172.21.15.3:0/2681424646 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2598170379 0 0) 0x7f2814003520 con 0x7f282c0d27c0 2024-05-09T06:36:12.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.791+0000 7f2831a08640 1 -- 172.21.15.3:0/2681424646 >> v1:172.21.15.190:6789/0 conn(0x7f282c15e480 legacy=0x7f282c0d1dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:12.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.791+0000 7f2831a08640 1 -- 172.21.15.3:0/2681424646 >> v1:172.21.15.3:6789/0 conn(0x7f282c0d27c0 legacy=0x7f282c166390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:12.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.791+0000 7f2831a08640 1 -- 172.21.15.3:0/2681424646 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f282c0cf1d0 con 0x7f282c0d2c20 2024-05-09T06:36:12.774 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.792+0000 7f2831a08640 1 --2- 172.21.15.3:0/2681424646 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f282c0d2c20 0x7f282c169ac0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f281c009dc0 tx=0x7f281c02a6c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=10ef861476d99273 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:12.774 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.792+0000 7f282bfff640 1 -- 172.21.15.3:0/2681424646 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2820001840 con 0x7f282c0d2c20 2024-05-09T06:36:12.774 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.792+0000 7f282bfff640 1 -- 172.21.15.3:0/2681424646 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f281c007540 con 0x7f282c0d2c20 2024-05-09T06:36:12.774 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.792+0000 7f282bfff640 1 -- 172.21.15.3:0/2681424646 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f281c007910 con 0x7f282c0d2c20 2024-05-09T06:36:12.774 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.792+0000 7f283346b640 1 -- 172.21.15.3:0/2681424646 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f282c0d2c20 msgr2=0x7f282c169ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:12.774 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.792+0000 7f283346b640 1 --2- 172.21.15.3:0/2681424646 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f282c0d2c20 0x7f282c169ac0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f281c009dc0 tx=0x7f281c02a6c0 comp rx=0 tx=0).stop 2024-05-09T06:36:12.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.793+0000 7f283346b640 1 -- 172.21.15.3:0/2681424646 shutdown_connections 2024-05-09T06:36:12.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.793+0000 7f283346b640 1 --2- 172.21.15.3:0/2681424646 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f282c0d2c20 0x7f282c169ac0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:12.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.793+0000 7f283346b640 1 -- 172.21.15.3:0/2681424646 >> 172.21.15.3:0/2681424646 conn(0x7f282c09d4d0 msgr2=0x7f282c0d4200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:12.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.793+0000 7f283346b640 1 -- 172.21.15.3:0/2681424646 shutdown_connections 2024-05-09T06:36:12.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.793+0000 7f283346b640 1 -- 172.21.15.3:0/2681424646 wait complete. 2024-05-09T06:36:12.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.794+0000 7f283346b640 1 Processor -- start 2024-05-09T06:36:12.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.794+0000 7f283346b640 1 -- start start 2024-05-09T06:36:12.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.795+0000 7f283346b640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f282c0d27c0 0x7f282c14c590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:12.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.795+0000 7f283346b640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f282c0d2c20 0x7f282c0ca860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:12.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.795+0000 7f283346b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f282c15e480 0x7f282c0b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:12.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.795+0000 7f283346b640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f282c0cfa70 con 0x7f282c15e480 2024-05-09T06:36:12.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.795+0000 7f283346b640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f282c0cf790 con 0x7f282c0d27c0 2024-05-09T06:36:12.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.795+0000 7f283346b640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f282c0cfd50 con 0x7f282c0d2c20 2024-05-09T06:36:12.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.795+0000 7f2831a08640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f282c15e480 0x7f282c0b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:12.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.795+0000 7f2831a08640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f282c15e480 0x7f282c0b4e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49424/0 (socket says 172.21.15.3:49424) 2024-05-09T06:36:12.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.795+0000 7f2831a08640 1 -- 172.21.15.3:0/1058382504 learned_addr learned my addr 172.21.15.3:0/1058382504 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:36:12.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.795+0000 7f2830a06640 1 --2- 172.21.15.3:0/1058382504 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f282c0d2c20 0x7f282c0ca860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:12.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.795+0000 7f2831207640 1 --2- 172.21.15.3:0/1058382504 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f282c0d27c0 0x7f282c14c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:12.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.795+0000 7f2831a08640 1 -- 172.21.15.3:0/1058382504 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f282c0d2c20 msgr2=0x7f282c0ca860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:12.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.795+0000 7f2831a08640 1 --2- 172.21.15.3:0/1058382504 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f282c0d2c20 0x7f282c0ca860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:12.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.795+0000 7f2831a08640 1 -- 172.21.15.3:0/1058382504 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f282c0d27c0 msgr2=0x7f282c14c590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:12.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.795+0000 7f2831a08640 1 --2- 172.21.15.3:0/1058382504 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f282c0d27c0 0x7f282c14c590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:12.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.795+0000 7f2831a08640 1 -- 172.21.15.3:0/1058382504 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f282c1f77e0 con 0x7f282c15e480 2024-05-09T06:36:12.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.795+0000 7f2831207640 1 --2- 172.21.15.3:0/1058382504 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f282c0d27c0 0x7f282c14c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:36:12.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.795+0000 7f2831a08640 1 --2- 172.21.15.3:0/1058382504 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f282c15e480 0x7f282c0b4e20 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f281c009ee0 tx=0x7f281c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:12.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.795+0000 7f2829ffb640 1 -- 172.21.15.3:0/1058382504 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f281c040070 con 0x7f282c15e480 2024-05-09T06:36:12.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.795+0000 7f2829ffb640 1 -- 172.21.15.3:0/1058382504 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f281c031bc0 con 0x7f282c15e480 2024-05-09T06:36:12.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.795+0000 7f2829ffb640 1 -- 172.21.15.3:0/1058382504 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f281c009450 con 0x7f282c15e480 2024-05-09T06:36:12.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.795+0000 7f283346b640 1 -- 172.21.15.3:0/1058382504 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f282c1f7ac0 con 0x7f282c15e480 2024-05-09T06:36:12.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.796+0000 7f283346b640 1 -- 172.21.15.3:0/1058382504 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f282c1f8080 con 0x7f282c15e480 2024-05-09T06:36:12.780 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.797+0000 7f283346b640 1 -- 172.21.15.3:0/1058382504 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f282c1f8460 con 0x7f282c15e480 2024-05-09T06:36:12.780 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.797+0000 7f2829ffb640 1 -- 172.21.15.3:0/1058382504 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f281c039040 con 0x7f282c15e480 2024-05-09T06:36:12.780 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.798+0000 7f2829ffb640 1 --2- 172.21.15.3:0/1058382504 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f281806be30 0x7f281806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:12.780 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.798+0000 7f2829ffb640 1 -- 172.21.15.3:0/1058382504 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f281c0ac430 con 0x7f282c15e480 2024-05-09T06:36:12.780 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.798+0000 7f2831207640 1 --2- 172.21.15.3:0/1058382504 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f281806be30 0x7f281806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:12.780 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.798+0000 7f2831207640 1 --2- 172.21.15.3:0/1058382504 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f281806be30 0x7f281806e2f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f28240011e0 tx=0x7f2824009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:12.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.802+0000 7f2829ffb640 1 -- 172.21.15.3:0/1058382504 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f282c1f8460 con 0x7f282c15e480 2024-05-09T06:36:12.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.987+0000 7f283346b640 1 -- 172.21.15.3:0/1058382504 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f282c04f660 con 0x7f282c15e480 2024-05-09T06:36:12.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.988+0000 7f2829ffb640 1 -- 172.21.15.3:0/1058382504 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16521 (secure 0 0 0) 0x7f281c02ae20 con 0x7f282c15e480 2024-05-09T06:36:12.969 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-09T06:36:12.969 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":20,"fsid":"16ed737d-2665-431a-8ac6-ef242be3975a","created":"2024-05-09T06:35:42.541368+0000","modified":"2024-05-09T06:36:08.856391+0000","last_up_change":"2024-05-09T06:35:53.778274+0000","last_in_change":"2024-05-09T06:35:44.780586+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T06:35:54.819999+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T06:36:04.931314+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T06:36:06.200433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fae3a8db-4826-4f3c-b33a-12c09bb5df6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":1897849827},{"type":"v1","addr":"172.21.15.3:6817","nonce":1897849827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":1897849827},{"type":"v1","addr":"172.21.15.3:6819","nonce":1897849827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":1897849827},{"type":"v1","addr":"172.21.15.3:6823","nonce":1897849827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":1897849827},{"type":"v1","addr":"172.21.15.3:6821","nonce":1897849827}]},"public_addr":"172.21.15.3:6817/1897849827","cluster_addr":"172.21.15.3:6819/1897849827","heartbeat_back_addr":"172.21.15.3:6823/1897849827","heartbeat_front_addr":"172.21.15.3:6821/1897849827","state":["exists","up"]},{"osd":1,"uuid":"646e882d-4ce4-4720-8c84-f71df563a35e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":926805498},{"type":"v1","addr":"172.21.15.3:6801","nonce":926805498}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":926805498},{"type":"v1","addr":"172.21.15.3:6803","nonce":926805498}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":926805498},{"type":"v1","addr":"172.21.15.3:6807","nonce":926805498}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":926805498},{"type":"v1","addr":"172.21.15.3:6805","nonce":926805498}]},"public_addr":"172.21.15.3:6801/926805498","cluster_addr":"172.21.15.3:6803/926805498","heartbeat_back_addr":"172.21.15.3:6807/926805498","heartbeat_front_addr":"172.21.15.3:6805/926805498","state":["exists","up"]},{"osd":2,"uuid":"1abbadb5-a570-4359-9643-acff3ede35a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":801295451},{"type":"v1","addr":"172.21.15.3:6809","nonce":801295451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":801295451},{"type":"v1","addr":"172.21.15.3:6811","nonce":801295451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":801295451},{"type":"v1","addr":"172.21.15.3:6815","nonce":801295451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":801295451},{"type":"v1","addr":"172.21.15.3:6813","nonce":801295451}]},"public_addr":"172.21.15.3:6809/801295451","cluster_addr":"172.21.15.3:6811/801295451","heartbeat_back_addr":"172.21.15.3:6815/801295451","heartbeat_front_addr":"172.21.15.3:6813/801295451","state":["exists","up"]},{"osd":3,"uuid":"62ac4dc1-2371-4dd2-acd3-e2dbb11aedf5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":1903389857},{"type":"v1","addr":"172.21.15.3:6825","nonce":1903389857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":1903389857},{"type":"v1","addr":"172.21.15.3:6827","nonce":1903389857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":1903389857},{"type":"v1","addr":"172.21.15.3:6831","nonce":1903389857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":1903389857},{"type":"v1","addr":"172.21.15.3:6829","nonce":1903389857}]},"public_addr":"172.21.15.3:6825/1903389857","cluster_addr":"172.21.15.3:6827/1903389857","heartbeat_back_addr":"172.21.15.3:6831/1903389857","heartbeat_front_addr":"172.21.15.3:6829/1903389857","state":["exists","up"]},{"osd":4,"uuid":"ae808398-8326-4d7c-9319-a2a0eafffb59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":2143624576},{"type":"v1","addr":"172.21.15.190:6801","nonce":2143624576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":2143624576},{"type":"v1","addr":"172.21.15.190:6803","nonce":2143624576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":2143624576},{"type":"v1","addr":"172.21.15.190:6807","nonce":2143624576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":2143624576},{"type":"v1","addr":"172.21.15.190:6805","nonce":2143624576}]},"public_addr":"172.21.15.190:6801/2143624576","cluster_addr":"172.21.15.190:6803/2143624576","heartbeat_back_addr":"172.21.15.190:6807/2143624576","heartbeat_front_addr":"172.21.15.190:6805/2143624576","state":["exists","up"]},{"osd":5,"uuid":"918adf50-c440-48ac-9a8d-4837562d7275","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":1766923126},{"type":"v1","addr":"172.21.15.190:6814","nonce":1766923126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":1766923126},{"type":"v1","addr":"172.21.15.190:6818","nonce":1766923126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":1766923126},{"type":"v1","addr":"172.21.15.190:6823","nonce":1766923126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":1766923126},{"type":"v1","addr":"172.21.15.190:6821","nonce":1766923126}]},"public_addr":"172.21.15.190:6814/1766923126","cluster_addr":"172.21.15.190:6818/1766923126","heartbeat_back_addr":"172.21.15.190:6823/1766923126","heartbeat_front_addr":"172.21.15.190:6821/1766923126","state":["exists","up"]},{"osd":6,"uuid":"ac2882bc-2a7c-463c-8f6c-76ff49828adc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":342414248},{"type":"v1","addr":"172.21.15.190:6825","nonce":342414248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":342414248},{"type":"v1","addr":"172.21.15.190:6827","nonce":342414248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":342414248},{"type":"v1","addr":"172.21.15.190:6831","nonce":342414248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":342414248},{"type":"v1","addr":"172.21.15.190:6829","nonce":342414248}]},"public_addr":"172.21.15.190:6825/342414248","cluster_addr":"172.21.15.190:6827/342414248","heartbeat_back_addr":"172.21.15.190:6831/342414248","heartbeat_front_addr":"172.21.15.190:6829/342414248","state":["exists","up"]},{"osd":7,"uuid":"dd4e767a-4730-4c70-8bd5-3f088140a5a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":537849891},{"type":"v1","addr":"172.21.15.190:6809","nonce":537849891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":537849891},{"type":"v1","addr":"172.21.15.190:6811","nonce":537849891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6817","nonce":537849891},{"type":"v1","addr":"172.21.15.190:6819","nonce":537849891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6813","nonce":537849891},{"type":"v1","addr":"172.21.15.190:6815","nonce":537849891}]},"public_addr":"172.21.15.190:6809/537849891","cluster_addr":"172.21.15.190:6811/537849891","heartbeat_back_addr":"172.21.15.190:6819/537849891","heartbeat_front_addr":"172.21.15.190:6815/537849891","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.889784+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.974002+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.973549+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.955396+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.928471+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.902418+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.958696+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.926474+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T06:36:12.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.990+0000 7f283346b640 1 -- 172.21.15.3:0/1058382504 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f281806be30 msgr2=0x7f281806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:12.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.990+0000 7f283346b640 1 --2- 172.21.15.3:0/1058382504 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f281806be30 0x7f281806e2f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f28240011e0 tx=0x7f2824009040 comp rx=0 tx=0).stop 2024-05-09T06:36:12.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.990+0000 7f283346b640 1 -- 172.21.15.3:0/1058382504 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f282c15e480 msgr2=0x7f282c0b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:12.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.990+0000 7f283346b640 1 --2- 172.21.15.3:0/1058382504 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f282c15e480 0x7f282c0b4e20 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f281c009ee0 tx=0x7f281c004a40 comp rx=0 tx=0).stop 2024-05-09T06:36:12.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.990+0000 7f283346b640 1 -- 172.21.15.3:0/1058382504 shutdown_connections 2024-05-09T06:36:12.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.990+0000 7f283346b640 1 --2- 172.21.15.3:0/1058382504 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f281806be30 0x7f281806e2f0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:12.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.990+0000 7f283346b640 1 --2- 172.21.15.3:0/1058382504 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f282c15e480 0x7f282c0b4e20 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:12.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.990+0000 7f283346b640 1 --2- 172.21.15.3:0/1058382504 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f282c0d2c20 0x7f282c0ca860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:12.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.990+0000 7f283346b640 1 --2- 172.21.15.3:0/1058382504 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f282c0d27c0 0x7f282c14c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:12.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.990+0000 7f283346b640 1 -- 172.21.15.3:0/1058382504 >> 172.21.15.3:0/1058382504 conn(0x7f282c09d4d0 msgr2=0x7f282c15d5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:12.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.990+0000 7f283346b640 1 -- 172.21.15.3:0/1058382504 shutdown_connections 2024-05-09T06:36:12.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:12.990+0000 7f283346b640 1 -- 172.21.15.3:0/1058382504 wait complete. 2024-05-09T06:36:12.982 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-05-09T06:36:12.982 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T06:36:13.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.105+0000 7f3285734640 1 Processor -- start 2024-05-09T06:36:13.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.106+0000 7f3285734640 1 -- start start 2024-05-09T06:36:13.091 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.106+0000 7f3285734640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f32800cee00 0x7f32800d41f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:13.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.106+0000 7f3285734640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f32800d4730 0x7f32800b42d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:13.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.106+0000 7f3285734640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f32800cf400 con 0x7f3280161b80 2024-05-09T06:36:13.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.106+0000 7f3285734640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f32800d4ba0 con 0x7f32800cee00 2024-05-09T06:36:13.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.106+0000 7f3285734640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f32800cf6e0 con 0x7f32800d4730 2024-05-09T06:36:13.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.107+0000 7f327e7fc640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f32800cee00 0x7f32800d41f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:13.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.107+0000 7f327f7fe640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f32800d4730 0x7f32800b42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:13.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.107+0000 7f327e7fc640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f32800cee00 0x7f32800d41f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.3:47622/0 (socket says 172.21.15.3:47622) 2024-05-09T06:36:13.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.107+0000 7f327e7fc640 1 -- 172.21.15.3:0/3953066394 learned_addr learned my addr 172.21.15.3:0/3953066394 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:36:13.093 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.107+0000 7f327e7fc640 1 -- 172.21.15.3:0/3953066394 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f32800d4730 msgr2=0x7f32800b42d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:13.093 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.107+0000 7f327e7fc640 1 --2- 172.21.15.3:0/3953066394 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f32800d4730 0x7f32800b42d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:13.093 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.107+0000 7f327e7fc640 1 -- 172.21.15.3:0/3953066394 >> v1:172.21.15.190:6789/0 conn(0x7f3280161b80 legacy=0x7f32800b37e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:13.093 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.107+0000 7f327e7fc640 1 -- 172.21.15.3:0/3953066394 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32800cf9c0 con 0x7f32800cee00 2024-05-09T06:36:13.093 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.107+0000 7f327dffb640 1 -- 172.21.15.3:0/3953066394 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3851316763 0 0) 0x7f32800cf400 con 0x7f3280161b80 2024-05-09T06:36:13.093 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.107+0000 7f327f7fe640 1 --2- 172.21.15.3:0/3953066394 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f32800d4730 0x7f32800b42d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T06:36:13.093 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.108+0000 7f327e7fc640 1 --2- 172.21.15.3:0/3953066394 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f32800cee00 0x7f32800d41f0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f3268009870 tx=0x7f326802a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bd111d2968e1799f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:13.093 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.108+0000 7f327dffb640 1 -- 172.21.15.3:0/3953066394 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3268037070 con 0x7f32800cee00 2024-05-09T06:36:13.093 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.108+0000 7f327dffb640 1 -- 172.21.15.3:0/3953066394 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3268032440 con 0x7f32800cee00 2024-05-09T06:36:13.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.108+0000 7f327dffb640 1 -- 172.21.15.3:0/3953066394 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3268032810 con 0x7f32800cee00 2024-05-09T06:36:13.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.109+0000 7f3285734640 1 -- 172.21.15.3:0/3953066394 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f32800cee00 msgr2=0x7f32800d41f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:13.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.109+0000 7f3285734640 1 --2- 172.21.15.3:0/3953066394 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f32800cee00 0x7f32800d41f0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f3268009870 tx=0x7f326802a270 comp rx=0 tx=0).stop 2024-05-09T06:36:13.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.109+0000 7f3285734640 1 -- 172.21.15.3:0/3953066394 shutdown_connections 2024-05-09T06:36:13.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.109+0000 7f3285734640 1 --2- 172.21.15.3:0/3953066394 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f32800d4730 0x7f32800b42d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:13.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.109+0000 7f3285734640 1 --2- 172.21.15.3:0/3953066394 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f32800cee00 0x7f32800d41f0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:13.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.109+0000 7f3285734640 1 -- 172.21.15.3:0/3953066394 >> 172.21.15.3:0/3953066394 conn(0x7f328008dba0 msgr2=0x7f328009d4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:13.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.109+0000 7f3285734640 1 -- 172.21.15.3:0/3953066394 shutdown_connections 2024-05-09T06:36:13.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.109+0000 7f3285734640 1 -- 172.21.15.3:0/3953066394 wait complete. 2024-05-09T06:36:13.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.110+0000 7f3285734640 1 Processor -- start 2024-05-09T06:36:13.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.110+0000 7f3285734640 1 -- start start 2024-05-09T06:36:13.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.111+0000 7f3285734640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f32800cee00 0x7f32800b22c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:13.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.111+0000 7f3285734640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f32800d4730 0x7f32800b9680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:13.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.111+0000 7f3285734640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3280161b80 0x7f32801f71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:13.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.111+0000 7f3285734640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f32800b4810 con 0x7f3280161b80 2024-05-09T06:36:13.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.111+0000 7f3285734640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f3280161f60 con 0x7f32800cee00 2024-05-09T06:36:13.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.111+0000 7f3285734640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f32800b4990 con 0x7f32800d4730 2024-05-09T06:36:13.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.111+0000 7f327f7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3280161b80 0x7f32801f71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:13.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.111+0000 7f327f7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3280161b80 0x7f32801f71b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49430/0 (socket says 172.21.15.3:49430) 2024-05-09T06:36:13.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.111+0000 7f327f7fe640 1 -- 172.21.15.3:0/444621489 learned_addr learned my addr 172.21.15.3:0/444621489 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:36:13.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.111+0000 7f327e7fc640 1 --2- 172.21.15.3:0/444621489 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f32800d4730 0x7f32800b9680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:13.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.111+0000 7f327effd640 1 --2- 172.21.15.3:0/444621489 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f32800cee00 0x7f32800b22c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:13.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.111+0000 7f327f7fe640 1 -- 172.21.15.3:0/444621489 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f32800d4730 msgr2=0x7f32800b9680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:13.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.111+0000 7f327f7fe640 1 --2- 172.21.15.3:0/444621489 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f32800d4730 0x7f32800b9680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:13.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.111+0000 7f327f7fe640 1 -- 172.21.15.3:0/444621489 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f32800cee00 msgr2=0x7f32800b22c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:13.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.111+0000 7f327f7fe640 1 --2- 172.21.15.3:0/444621489 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f32800cee00 0x7f32800b22c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:13.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.111+0000 7f327f7fe640 1 -- 172.21.15.3:0/444621489 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32800b2800 con 0x7f3280161b80 2024-05-09T06:36:13.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.111+0000 7f327e7fc640 1 --2- 172.21.15.3:0/444621489 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f32800d4730 0x7f32800b9680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:36:13.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.111+0000 7f327effd640 1 --2- 172.21.15.3:0/444621489 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f32800cee00 0x7f32800b22c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:36:13.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.112+0000 7f327f7fe640 1 --2- 172.21.15.3:0/444621489 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3280161b80 0x7f32801f71b0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f327000efc0 tx=0x7f3270009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:13.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.112+0000 7f32777fe640 1 -- 172.21.15.3:0/444621489 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3270018030 con 0x7f3280161b80 2024-05-09T06:36:13.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.112+0000 7f32777fe640 1 -- 172.21.15.3:0/444621489 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f327000cde0 con 0x7f3280161b80 2024-05-09T06:36:13.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.112+0000 7f3285734640 1 -- 172.21.15.3:0/444621489 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32801f4d80 con 0x7f3280161b80 2024-05-09T06:36:13.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.112+0000 7f32777fe640 1 -- 172.21.15.3:0/444621489 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3270003bb0 con 0x7f3280161b80 2024-05-09T06:36:13.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.112+0000 7f3285734640 1 -- 172.21.15.3:0/444621489 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32801f5340 con 0x7f3280161b80 2024-05-09T06:36:13.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.113+0000 7f3285734640 1 -- 172.21.15.3:0/444621489 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f328004f660 con 0x7f3280161b80 2024-05-09T06:36:13.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.113+0000 7f32777fe640 1 -- 172.21.15.3:0/444621489 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3270005d10 con 0x7f3280161b80 2024-05-09T06:36:13.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.114+0000 7f32777fe640 1 --2- 172.21.15.3:0/444621489 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f326006be30 0x7f326006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:13.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.114+0000 7f32777fe640 1 -- 172.21.15.3:0/444621489 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f327008b870 con 0x7f3280161b80 2024-05-09T06:36:13.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.114+0000 7f327effd640 1 --2- 172.21.15.3:0/444621489 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f326006be30 0x7f326006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:13.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.114+0000 7f327effd640 1 --2- 172.21.15.3:0/444621489 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f326006be30 0x7f326006e2f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f3278000f60 tx=0x7f3278009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:13.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.118+0000 7f32777fe640 1 -- 172.21.15.3:0/444621489 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f3270022070 con 0x7f3280161b80 2024-05-09T06:36:13.280 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.300+0000 7f3285734640 1 -- 172.21.15.3:0/444621489 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f32800d1510 con 0x7f3280161b80 2024-05-09T06:36:13.281 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.301+0000 7f32777fe640 1 -- 172.21.15.3:0/444621489 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16521 (secure 0 0 0) 0x7f32800b2800 con 0x7f3280161b80 2024-05-09T06:36:13.282 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-09T06:36:13.282 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":20,"fsid":"16ed737d-2665-431a-8ac6-ef242be3975a","created":"2024-05-09T06:35:42.541368+0000","modified":"2024-05-09T06:36:08.856391+0000","last_up_change":"2024-05-09T06:35:53.778274+0000","last_in_change":"2024-05-09T06:35:44.780586+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T06:35:54.819999+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T06:36:04.931314+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T06:36:06.200433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fae3a8db-4826-4f3c-b33a-12c09bb5df6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":1897849827},{"type":"v1","addr":"172.21.15.3:6817","nonce":1897849827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":1897849827},{"type":"v1","addr":"172.21.15.3:6819","nonce":1897849827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":1897849827},{"type":"v1","addr":"172.21.15.3:6823","nonce":1897849827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":1897849827},{"type":"v1","addr":"172.21.15.3:6821","nonce":1897849827}]},"public_addr":"172.21.15.3:6817/1897849827","cluster_addr":"172.21.15.3:6819/1897849827","heartbeat_back_addr":"172.21.15.3:6823/1897849827","heartbeat_front_addr":"172.21.15.3:6821/1897849827","state":["exists","up"]},{"osd":1,"uuid":"646e882d-4ce4-4720-8c84-f71df563a35e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":926805498},{"type":"v1","addr":"172.21.15.3:6801","nonce":926805498}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":926805498},{"type":"v1","addr":"172.21.15.3:6803","nonce":926805498}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":926805498},{"type":"v1","addr":"172.21.15.3:6807","nonce":926805498}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":926805498},{"type":"v1","addr":"172.21.15.3:6805","nonce":926805498}]},"public_addr":"172.21.15.3:6801/926805498","cluster_addr":"172.21.15.3:6803/926805498","heartbeat_back_addr":"172.21.15.3:6807/926805498","heartbeat_front_addr":"172.21.15.3:6805/926805498","state":["exists","up"]},{"osd":2,"uuid":"1abbadb5-a570-4359-9643-acff3ede35a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":801295451},{"type":"v1","addr":"172.21.15.3:6809","nonce":801295451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":801295451},{"type":"v1","addr":"172.21.15.3:6811","nonce":801295451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":801295451},{"type":"v1","addr":"172.21.15.3:6815","nonce":801295451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":801295451},{"type":"v1","addr":"172.21.15.3:6813","nonce":801295451}]},"public_addr":"172.21.15.3:6809/801295451","cluster_addr":"172.21.15.3:6811/801295451","heartbeat_back_addr":"172.21.15.3:6815/801295451","heartbeat_front_addr":"172.21.15.3:6813/801295451","state":["exists","up"]},{"osd":3,"uuid":"62ac4dc1-2371-4dd2-acd3-e2dbb11aedf5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":1903389857},{"type":"v1","addr":"172.21.15.3:6825","nonce":1903389857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":1903389857},{"type":"v1","addr":"172.21.15.3:6827","nonce":1903389857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":1903389857},{"type":"v1","addr":"172.21.15.3:6831","nonce":1903389857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":1903389857},{"type":"v1","addr":"172.21.15.3:6829","nonce":1903389857}]},"public_addr":"172.21.15.3:6825/1903389857","cluster_addr":"172.21.15.3:6827/1903389857","heartbeat_back_addr":"172.21.15.3:6831/1903389857","heartbeat_front_addr":"172.21.15.3:6829/1903389857","state":["exists","up"]},{"osd":4,"uuid":"ae808398-8326-4d7c-9319-a2a0eafffb59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":2143624576},{"type":"v1","addr":"172.21.15.190:6801","nonce":2143624576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":2143624576},{"type":"v1","addr":"172.21.15.190:6803","nonce":2143624576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":2143624576},{"type":"v1","addr":"172.21.15.190:6807","nonce":2143624576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":2143624576},{"type":"v1","addr":"172.21.15.190:6805","nonce":2143624576}]},"public_addr":"172.21.15.190:6801/2143624576","cluster_addr":"172.21.15.190:6803/2143624576","heartbeat_back_addr":"172.21.15.190:6807/2143624576","heartbeat_front_addr":"172.21.15.190:6805/2143624576","state":["exists","up"]},{"osd":5,"uuid":"918adf50-c440-48ac-9a8d-4837562d7275","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":1766923126},{"type":"v1","addr":"172.21.15.190:6814","nonce":1766923126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":1766923126},{"type":"v1","addr":"172.21.15.190:6818","nonce":1766923126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":1766923126},{"type":"v1","addr":"172.21.15.190:6823","nonce":1766923126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":1766923126},{"type":"v1","addr":"172.21.15.190:6821","nonce":1766923126}]},"public_addr":"172.21.15.190:6814/1766923126","cluster_addr":"172.21.15.190:6818/1766923126","heartbeat_back_addr":"172.21.15.190:6823/1766923126","heartbeat_front_addr":"172.21.15.190:6821/1766923126","state":["exists","up"]},{"osd":6,"uuid":"ac2882bc-2a7c-463c-8f6c-76ff49828adc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":342414248},{"type":"v1","addr":"172.21.15.190:6825","nonce":342414248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":342414248},{"type":"v1","addr":"172.21.15.190:6827","nonce":342414248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":342414248},{"type":"v1","addr":"172.21.15.190:6831","nonce":342414248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":342414248},{"type":"v1","addr":"172.21.15.190:6829","nonce":342414248}]},"public_addr":"172.21.15.190:6825/342414248","cluster_addr":"172.21.15.190:6827/342414248","heartbeat_back_addr":"172.21.15.190:6831/342414248","heartbeat_front_addr":"172.21.15.190:6829/342414248","state":["exists","up"]},{"osd":7,"uuid":"dd4e767a-4730-4c70-8bd5-3f088140a5a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":537849891},{"type":"v1","addr":"172.21.15.190:6809","nonce":537849891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":537849891},{"type":"v1","addr":"172.21.15.190:6811","nonce":537849891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6817","nonce":537849891},{"type":"v1","addr":"172.21.15.190:6819","nonce":537849891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6813","nonce":537849891},{"type":"v1","addr":"172.21.15.190:6815","nonce":537849891}]},"public_addr":"172.21.15.190:6809/537849891","cluster_addr":"172.21.15.190:6811/537849891","heartbeat_back_addr":"172.21.15.190:6819/537849891","heartbeat_front_addr":"172.21.15.190:6815/537849891","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.889784+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.974002+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.973549+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.955396+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.928471+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.902418+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.958696+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.926474+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T06:36:13.284 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.303+0000 7f3285734640 1 -- 172.21.15.3:0/444621489 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f326006be30 msgr2=0x7f326006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:13.284 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.303+0000 7f3285734640 1 --2- 172.21.15.3:0/444621489 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f326006be30 0x7f326006e2f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f3278000f60 tx=0x7f3278009040 comp rx=0 tx=0).stop 2024-05-09T06:36:13.284 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.303+0000 7f3285734640 1 -- 172.21.15.3:0/444621489 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3280161b80 msgr2=0x7f32801f71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:13.284 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.303+0000 7f3285734640 1 --2- 172.21.15.3:0/444621489 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3280161b80 0x7f32801f71b0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f327000efc0 tx=0x7f3270009f90 comp rx=0 tx=0).stop 2024-05-09T06:36:13.284 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.303+0000 7f3285734640 1 -- 172.21.15.3:0/444621489 shutdown_connections 2024-05-09T06:36:13.284 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.303+0000 7f3285734640 1 --2- 172.21.15.3:0/444621489 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f326006be30 0x7f326006e2f0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:13.284 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.303+0000 7f3285734640 1 --2- 172.21.15.3:0/444621489 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3280161b80 0x7f32801f71b0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:13.284 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.303+0000 7f3285734640 1 --2- 172.21.15.3:0/444621489 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f32800d4730 0x7f32800b9680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:13.284 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.303+0000 7f3285734640 1 --2- 172.21.15.3:0/444621489 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f32800cee00 0x7f32800b22c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:13.284 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.303+0000 7f3285734640 1 -- 172.21.15.3:0/444621489 >> 172.21.15.3:0/444621489 conn(0x7f328008dba0 msgr2=0x7f328006d4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:13.284 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.303+0000 7f3285734640 1 -- 172.21.15.3:0/444621489 shutdown_connections 2024-05-09T06:36:13.284 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.303+0000 7f3285734640 1 -- 172.21.15.3:0/444621489 wait complete. 2024-05-09T06:36:13.295 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-05-09T06:36:13.295 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-05-09T06:36:13.296 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-05-09T06:36:13.296 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-05-09T06:36:13.296 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-05-09T06:36:13.297 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-05-09T06:36:13.297 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-05-09T06:36:13.298 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-05-09T06:36:13.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.498+0000 7ff57ba38640 1 Processor -- start 2024-05-09T06:36:13.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.498+0000 7ff57ba38640 1 -- start start 2024-05-09T06:36:13.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.497+0000 7f0ba3ebd640 1 Processor -- start 2024-05-09T06:36:13.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.498+0000 7f0ba3ebd640 1 -- start start 2024-05-09T06:36:13.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.498+0000 7fe2fc8e7640 1 Processor -- start 2024-05-09T06:36:13.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.499+0000 7fe2fc8e7640 1 -- start start 2024-05-09T06:36:13.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.499+0000 7f28ac570640 1 Processor -- start 2024-05-09T06:36:13.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.498+0000 7f40053d6640 1 Processor -- start 2024-05-09T06:36:13.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.499+0000 7f40053d6640 1 -- start start 2024-05-09T06:36:13.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.499+0000 7f28ac570640 1 -- start start 2024-05-09T06:36:13.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.499+0000 7f83d28f8640 1 Processor -- start 2024-05-09T06:36:13.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.500+0000 7ff57ba38640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7ff574154900 0x7ff574154ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:13.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.500+0000 7f83d28f8640 1 -- start start 2024-05-09T06:36:13.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.500+0000 7ff57ba38640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff57415adf0 0x7ff57406b250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:13.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.500+0000 7f0ba3ebd640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f0b940504f0 0x7f0b940e9db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:13.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.500+0000 7ff57ba38640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff57406e720 con 0x7ff574066f90 2024-05-09T06:36:13.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.500+0000 7f0ba3ebd640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0b94054d30 con 0x7f0b940df3d0 2024-05-09T06:36:13.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.500+0000 7ff57ba38640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7ff57406cbf0 con 0x7ff57415adf0 2024-05-09T06:36:13.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.500+0000 7ff57ba38640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7ff57406c760 con 0x7ff574154900 2024-05-09T06:36:13.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.500+0000 7f0ba3ebd640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0b94054630 con 0x7f0b940deff0 2024-05-09T06:36:13.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.500+0000 7f0ba245a640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f0b940504f0 0x7f0b940e9db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:13.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.500+0000 7ff578fd3640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff57415adf0 0x7ff57406b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:13.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.500+0000 7ff578fd3640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff57415adf0 0x7ff57406b250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49446/0 (socket says 172.21.15.3:49446) 2024-05-09T06:36:13.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.500+0000 7ff578fd3640 1 -- 172.21.15.3:0/584871213 learned_addr learned my addr 172.21.15.3:0/584871213 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:36:13.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.500+0000 7f0ba245a640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f0b940504f0 0x7f0b940e9db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.3:47628/0 (socket says 172.21.15.3:47628) 2024-05-09T06:36:13.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.500+0000 7f0ba245a640 1 -- 172.21.15.3:0/218686405 learned_addr learned my addr 172.21.15.3:0/218686405 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:36:13.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.500+0000 7fe2fc8e7640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fe2f815c0d0 0x7fe2f80cca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:13.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.500+0000 7fe2fc8e7640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe2f815c4b0 0x7fe2f80ccf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:13.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.500+0000 7fe2fc8e7640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe2f80cfad0 con 0x7fe2f80cf600 2024-05-09T06:36:13.486 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.500+0000 7fe2fc8e7640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7fe2f80cfdb0 con 0x7fe2f815c0d0 2024-05-09T06:36:13.486 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.500+0000 7f0ba3ebd640 1 -- 172.21.15.3:0/218686405 --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f0b94006dd0 con 0x7f0b940504f0 2024-05-09T06:36:13.486 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.500+0000 7fe2fc8e7640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fe2f806b250 con 0x7fe2f815c4b0 2024-05-09T06:36:13.486 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.501+0000 7ff5797d4640 1 --2- 172.21.15.3:0/584871213 >> v2:172.21.15.190:3300/0 conn(0x7ff574154900 0x7ff574154ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:13.487 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.500+0000 7fe2f6d9d640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe2f815c4b0 0x7fe2f80ccf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:13.487 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.501+0000 7f1903ad4640 1 Processor -- start 2024-05-09T06:36:13.487 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.499+0000 7f6358b73640 1 Processor -- start 2024-05-09T06:36:13.487 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.499+0000 7f6358b73640 1 -- start start 2024-05-09T06:36:13.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.501+0000 7f6358b73640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f634c0073f0 0x7f634c105050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:13.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.501+0000 7f6358b73640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f634c105590 0x7f634c052c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:13.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.501+0000 7f6358b73640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f634c054240 con 0x7f634c054520 2024-05-09T06:36:13.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.501+0000 7f6358b73640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f634c03a3a0 con 0x7f634c0073f0 2024-05-09T06:36:13.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.501+0000 7f6358b73640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f634c03a680 con 0x7f634c105590 2024-05-09T06:36:13.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.501+0000 7f28ac570640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f28a40b35e0 0x7f28a406d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:13.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.500+0000 7f40053d6640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f3ff80eacf0 0x7f3ff804fa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:13.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.501+0000 7f3fff7fe640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f3ff804ffc0 0x7f3ff8037bc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:49218/0 (socket says 172.21.15.3:49218) 2024-05-09T06:36:13.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.501+0000 7f3ffffff640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f3ff80eacf0 0x7f3ff804fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:13.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.501+0000 7f3ffffff640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f3ff80eacf0 0x7f3ff804fa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.3:47652/0 (socket says 172.21.15.3:47652) 2024-05-09T06:36:13.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.501+0000 7f3fff7fe640 1 -- 172.21.15.3:0/3008078279 learned_addr learned my addr 172.21.15.3:0/3008078279 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-05-09T06:36:13.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.501+0000 7f40053d6640 1 -- 172.21.15.3:0/3008078279 --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3ff8053710 con 0x7f3ff804ffc0 2024-05-09T06:36:13.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.501+0000 7f40053d6640 1 -- 172.21.15.3:0/3008078279 --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3ff80539f0 con 0x7f3ff8006ee0 2024-05-09T06:36:13.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.501+0000 7f40053d6640 1 -- 172.21.15.3:0/3008078279 --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f3ff8054320 con 0x7f3ff80eacf0 2024-05-09T06:36:13.490 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.501+0000 7f28ac570640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f28a4072f10 0x7f28a406dae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:13.490 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.501+0000 7f28ac570640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f28a40711d0 con 0x7f28a4159ed0 2024-05-09T06:36:13.490 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.501+0000 7f28ac570640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f28a4072470 con 0x7f28a4072f10 2024-05-09T06:36:13.490 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.501+0000 7f28ac570640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f28a4071470 con 0x7f28a40b35e0 2024-05-09T06:36:13.490 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.502+0000 7f83d28f8640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f83c40501a0 0x7f83c4038080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:13.490 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.502+0000 7f28a9b0b640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f28a40b35e0 0x7f28a406d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:13.490 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.502+0000 7f28a9b0b640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f28a40b35e0 0x7f28a406d5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.3:44986/0 (socket says 172.21.15.3:44986) 2024-05-09T06:36:13.490 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.502+0000 7f28a9b0b640 1 -- 172.21.15.3:0/777743219 learned_addr learned my addr 172.21.15.3:0/777743219 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:36:13.491 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.501+0000 7f28aa30c640 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7f28a4159ed0 0x7f28a40b2ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.3:58376/0 (socket says 172.21.15.3:58376) 2024-05-09T06:36:13.491 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.502+0000 7f83d28f8640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f83c4053c70 0x7f83c4006a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:13.491 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.502+0000 7f83d28f8640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f83c4007830 con 0x7f83c40df220 2024-05-09T06:36:13.491 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.502+0000 7f83d28f8640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f83c40544c0 con 0x7f83c40501a0 2024-05-09T06:36:13.491 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.502+0000 7f83d28f8640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f83c40541e0 con 0x7f83c4053c70 2024-05-09T06:36:13.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.501+0000 7f0ba0c57640 1 -- 172.21.15.3:0/218686405 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1535147069 0 0) 0x7f0b94054630 con 0x7f0b940deff0 2024-05-09T06:36:13.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.501+0000 7f0ba0c57640 1 -- 172.21.15.3:0/218686405 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0b88003520 con 0x7f0b940deff0 2024-05-09T06:36:13.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.502+0000 7f0ba245a640 1 -- 172.21.15.3:0/218686405 >> v1:172.21.15.190:6789/0 conn(0x7f0b940deff0 legacy=0x7f0b9404f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:13.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.502+0000 7f0ba245a640 1 -- 172.21.15.3:0/218686405 >> v1:172.21.15.3:6789/0 conn(0x7f0b940df3d0 legacy=0x7f0b9404fde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:13.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.502+0000 7f0ba245a640 1 -- 172.21.15.3:0/218686405 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b940070b0 con 0x7f0b940504f0 2024-05-09T06:36:13.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.502+0000 7f83d20f7640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f83c4053c70 0x7f83c4006a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:13.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.502+0000 7f83d20f7640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f83c4053c70 0x7f83c4006a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.3:47664/0 (socket says 172.21.15.3:47664) 2024-05-09T06:36:13.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.502+0000 7f83d20f7640 1 -- 172.21.15.3:0/3993722418 learned_addr learned my addr 172.21.15.3:0/3993722418 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:36:13.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.502+0000 7f28a930a640 1 -- 172.21.15.3:0/777743219 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3074814549 0 0) 0x7f28a40711d0 con 0x7f28a4159ed0 2024-05-09T06:36:13.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.501+0000 7ff578fd3640 1 -- 172.21.15.3:0/584871213 >> v2:172.21.15.190:3300/0 conn(0x7ff574154900 msgr2=0x7ff574154ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:13.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.501+0000 7ff578fd3640 1 --2- 172.21.15.3:0/584871213 >> v2:172.21.15.190:3300/0 conn(0x7ff574154900 0x7ff574154ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:13.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.501+0000 7ff578fd3640 1 -- 172.21.15.3:0/584871213 >> v1:172.21.15.190:6789/0 conn(0x7ff574066f90 legacy=0x7ff5740b4f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:13.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.501+0000 7ff578fd3640 1 -- 172.21.15.3:0/584871213 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff57406bb50 con 0x7ff57415adf0 2024-05-09T06:36:13.494 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.502+0000 7f28a9b0b640 1 -- 172.21.15.3:0/777743219 >> v2:172.21.15.3:3300/0 conn(0x7f28a4072f10 msgr2=0x7f28a406dae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:13.494 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.502+0000 7f28aab0d640 1 --2- 172.21.15.3:0/777743219 >> v2:172.21.15.3:3300/0 conn(0x7f28a4072f10 0x7f28a406dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:13.494 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.501+0000 7f6353fff640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f634c105590 0x7f634c052c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:13.494 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.501+0000 7f6353fff640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f634c105590 0x7f634c052c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.3:44978/0 (socket says 172.21.15.3:44978) 2024-05-09T06:36:13.494 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.501+0000 7f6353fff640 1 -- 172.21.15.3:0/3096428996 learned_addr learned my addr 172.21.15.3:0/3096428996 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:36:13.494 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.502+0000 7f6352ffd640 1 --2- 172.21.15.3:0/3096428996 >> v2:172.21.15.3:3300/0 conn(0x7f634c0073f0 0x7f634c105050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:13.494 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.502+0000 7f6353fff640 1 -- 172.21.15.3:0/3096428996 >> v2:172.21.15.3:3300/0 conn(0x7f634c0073f0 msgr2=0x7f634c105050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:13.494 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.502+0000 7f6353fff640 1 --2- 172.21.15.3:0/3096428996 >> v2:172.21.15.3:3300/0 conn(0x7f634c0073f0 0x7f634c105050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:13.494 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.502+0000 7f63527fc640 1 -- 172.21.15.3:0/3096428996 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2569891011 0 0) 0x7f634c054240 con 0x7f634c054520 2024-05-09T06:36:13.494 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.502+0000 7f6353fff640 1 -- 172.21.15.3:0/3096428996 >> v1:172.21.15.190:6789/0 conn(0x7f634c054520 legacy=0x7f634c104940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:13.494 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.502+0000 7f6353fff640 1 -- 172.21.15.3:0/3096428996 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f634c03a960 con 0x7f634c105590 2024-05-09T06:36:13.495 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.502+0000 7f28a9b0b640 1 --2- 172.21.15.3:0/777743219 >> v2:172.21.15.3:3300/0 conn(0x7f28a4072f10 0x7f28a406dae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:13.495 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.502+0000 7f28a9b0b640 1 -- 172.21.15.3:0/777743219 >> v1:172.21.15.190:6789/0 conn(0x7f28a4159ed0 legacy=0x7f28a40b2ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:13.495 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.502+0000 7f3ffeffd640 1 -- 172.21.15.3:0/3008078279 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 761427343 0 0) 0x7f3ff8053710 con 0x7f3ff804ffc0 2024-05-09T06:36:13.495 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.502+0000 7f3ffeffd640 1 -- 172.21.15.3:0/3008078279 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3ff0003520 con 0x7f3ff804ffc0 2024-05-09T06:36:13.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.501+0000 7fe2f6d9d640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe2f815c4b0 0x7fe2f80ccf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.3:44968/0 (socket says 172.21.15.3:44968) 2024-05-09T06:36:13.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.501+0000 7fe2f6d9d640 1 -- 172.21.15.3:0/1925562112 learned_addr learned my addr 172.21.15.3:0/1925562112 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:36:13.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.502+0000 7f28a9b0b640 1 -- 172.21.15.3:0/777743219 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28a4072190 con 0x7f28a40b35e0 2024-05-09T06:36:13.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.501+0000 7fe2f659c640 1 --2- 172.21.15.3:0/1925562112 >> v2:172.21.15.190:3300/0 conn(0x7fe2f815c0d0 0x7fe2f80cca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:13.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.501+0000 7fe2f6d9d640 1 -- 172.21.15.3:0/1925562112 >> v2:172.21.15.190:3300/0 conn(0x7fe2f815c0d0 msgr2=0x7fe2f80cca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:13.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.501+0000 7fe2f6d9d640 1 --2- 172.21.15.3:0/1925562112 >> v2:172.21.15.190:3300/0 conn(0x7fe2f815c0d0 0x7fe2f80cca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:13.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.501+0000 7fe2f6d9d640 1 -- 172.21.15.3:0/1925562112 >> v1:172.21.15.190:6789/0 conn(0x7fe2f80cf600 legacy=0x7fe2f806ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:13.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.501+0000 7fe2f6d9d640 1 -- 172.21.15.3:0/1925562112 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2f806b530 con 0x7fe2f815c4b0 2024-05-09T06:36:13.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.501+0000 7f1903ad4640 1 -- start start 2024-05-09T06:36:13.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.502+0000 7f1903ad4640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f18fc15e420 0x7f18fc0d1750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:13.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.503+0000 7ff578fd3640 1 --2- 172.21.15.3:0/584871213 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff57415adf0 0x7ff57406b250 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7ff574070440 tx=0x7ff56402c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23449ff1cfaf77c8 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:13.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.503+0000 7f3ffffff640 1 -- 172.21.15.3:0/3008078279 >> v1:172.21.15.190:6789/0 conn(0x7f3ff8006ee0 legacy=0x7f3ff80382d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:13.498 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.503+0000 7f3ffffff640 1 -- 172.21.15.3:0/3008078279 >> v1:172.21.15.3:6789/0 conn(0x7f3ff804ffc0 legacy=0x7f3ff8037bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:13.498 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.503+0000 7f3ffffff640 1 -- 172.21.15.3:0/3008078279 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ff8053cd0 con 0x7f3ff80eacf0 2024-05-09T06:36:13.498 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.503+0000 7f6353fff640 1 --2- 172.21.15.3:0/3096428996 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f634c105590 0x7f634c052c50 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f6354062df0 tx=0x7f6354094fa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=29d51a43240a8e93 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:13.498 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.502+0000 7f83d20f7640 1 -- 172.21.15.3:0/3993722418 >> v2:172.21.15.3:3300/0 conn(0x7f83c40501a0 msgr2=0x7f83c4038080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:13.498 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.502+0000 7f83d20f7640 1 --2- 172.21.15.3:0/3993722418 >> v2:172.21.15.3:3300/0 conn(0x7f83c40501a0 0x7f83c4038080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:13.498 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.502+0000 7f83d20f7640 1 -- 172.21.15.3:0/3993722418 >> v1:172.21.15.3:6789/0 conn(0x7f83c40df220 legacy=0x7f83c404fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:13.498 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.503+0000 7f83d20f7640 1 -- 172.21.15.3:0/3993722418 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83c4005f80 con 0x7f83c4053c70 2024-05-09T06:36:13.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.503+0000 7f1901870640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f18fc15e420 0x7f18fc0d1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:13.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.503+0000 7f1901870640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f18fc15e420 0x7f18fc0d1750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49488/0 (socket says 172.21.15.3:49488) 2024-05-09T06:36:13.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.503+0000 7f1901870640 1 -- 172.21.15.3:0/535861900 learned_addr learned my addr 172.21.15.3:0/535861900 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:36:13.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.503+0000 7f63527fc640 1 -- 172.21.15.3:0/3096428996 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63540a2070 con 0x7f634c105590 2024-05-09T06:36:13.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.503+0000 7f1903ad4640 1 -- 172.21.15.3:0/535861900 --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f18fc0cf010 con 0x7f18fc0cfdd0 2024-05-09T06:36:13.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.503+0000 7f63527fc640 1 -- 172.21.15.3:0/3096428996 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6354095df0 con 0x7f634c105590 2024-05-09T06:36:13.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.503+0000 7f1903ad4640 1 -- 172.21.15.3:0/535861900 --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f18fc0cf2f0 con 0x7f18fc0d3f40 2024-05-09T06:36:13.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.503+0000 7f63527fc640 1 -- 172.21.15.3:0/3096428996 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f635409d640 con 0x7f634c105590 2024-05-09T06:36:13.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.503+0000 7f1903ad4640 1 -- 172.21.15.3:0/535861900 --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f18fc0cede0 con 0x7f18fc15e420 2024-05-09T06:36:13.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.503+0000 7ff56bfff640 1 -- 172.21.15.3:0/584871213 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff564038070 con 0x7ff57415adf0 2024-05-09T06:36:13.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.503+0000 7ff56bfff640 1 -- 172.21.15.3:0/584871213 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff564031710 con 0x7ff57415adf0 2024-05-09T06:36:13.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.503+0000 7f0ba245a640 1 --2- 172.21.15.3:0/218686405 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0b940504f0 0x7f0b940e9db0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f0b8c009870 tx=0x7f0b8c02e700 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f8658cd3446ec091 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:13.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.503+0000 7f190086e640 1 -- 172.21.15.3:0/535861900 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3357614783 0 0) 0x7f18fc0cf010 con 0x7f18fc0cfdd0 2024-05-09T06:36:13.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.503+0000 7f190086e640 1 -- 172.21.15.3:0/535861900 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f18e8003520 con 0x7f18fc0cfdd0 2024-05-09T06:36:13.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.503+0000 7f28aab0d640 1 --2- 172.21.15.3:0/777743219 >> v2:172.21.15.3:3300/0 conn(0x7f28a4072f10 0x7f28a406dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:36:13.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.503+0000 7ff56bfff640 1 -- 172.21.15.3:0/584871213 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff564031ae0 con 0x7ff57415adf0 2024-05-09T06:36:13.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.504+0000 7f190086e640 1 -- 172.21.15.3:0/535861900 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3021243049 0 0) 0x7f18fc0cf2f0 con 0x7f18fc0d3f40 2024-05-09T06:36:13.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.504+0000 7f190086e640 1 -- 172.21.15.3:0/535861900 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f18fc0cf010 con 0x7f18fc0d3f40 2024-05-09T06:36:13.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.504+0000 7f1901870640 1 -- 172.21.15.3:0/535861900 >> v1:172.21.15.190:6789/0 conn(0x7f18fc0d3f40 legacy=0x7f18fc0d4390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:13.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.504+0000 7f1901870640 1 -- 172.21.15.3:0/535861900 >> v1:172.21.15.3:6789/0 conn(0x7f18fc0cfdd0 legacy=0x7f18fc0d2110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:13.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.504+0000 7f1901870640 1 -- 172.21.15.3:0/535861900 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18fc0cf5d0 con 0x7f18fc15e420 2024-05-09T06:36:13.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.504+0000 7f0ba0c57640 1 -- 172.21.15.3:0/218686405 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b8c03a070 con 0x7f0b940504f0 2024-05-09T06:36:13.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.504+0000 7fe2f6d9d640 1 --2- 172.21.15.3:0/1925562112 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe2f815c4b0 0x7fe2f80ccf50 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fe2ec009870 tx=0x7fe2ec02a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6725bcbfa0803fe7 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:13.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.504+0000 7f28a9b0b640 1 --2- 172.21.15.3:0/777743219 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f28a40b35e0 0x7f28a406d5a0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f28a0008870 tx=0x7f28a002c6f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d49e9df020aee1f5 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:13.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.504+0000 7f0ba0c57640 1 -- 172.21.15.3:0/218686405 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b8c033710 con 0x7f0b940504f0 2024-05-09T06:36:13.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.504+0000 7f0ba0c57640 1 -- 172.21.15.3:0/218686405 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b8c033ae0 con 0x7f0b940504f0 2024-05-09T06:36:13.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.504+0000 7fe2f559a640 1 -- 172.21.15.3:0/1925562112 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2ec037070 con 0x7fe2f815c4b0 2024-05-09T06:36:13.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.504+0000 7f83d20f7640 1 --2- 172.21.15.3:0/3993722418 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f83c4053c70 0x7f83c4006a60 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f83cc062df0 tx=0x7f83cc0a5e20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4dcde722dacd0f86 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:13.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.504+0000 7fe2f559a640 1 -- 172.21.15.3:0/1925562112 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe2ec032440 con 0x7fe2f815c4b0 2024-05-09T06:36:13.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.504+0000 7fe2f559a640 1 -- 172.21.15.3:0/1925562112 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2ec037070 con 0x7fe2f815c4b0 2024-05-09T06:36:13.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.504+0000 7f83d08f4640 1 -- 172.21.15.3:0/3993722418 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83cc0b3070 con 0x7f83c4053c70 2024-05-09T06:36:13.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.504+0000 7f83d08f4640 1 -- 172.21.15.3:0/3993722418 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83cc0a6c30 con 0x7f83c4053c70 2024-05-09T06:36:13.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.504+0000 7f83d08f4640 1 -- 172.21.15.3:0/3993722418 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83cc0ae640 con 0x7f83c4053c70 2024-05-09T06:36:13.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.504+0000 7f28a930a640 1 -- 172.21.15.3:0/777743219 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28a003a070 con 0x7f28a40b35e0 2024-05-09T06:36:13.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.504+0000 7f6358b73640 1 -- 172.21.15.3:0/3096428996 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f634c105590 msgr2=0x7f634c052c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:13.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.504+0000 7f6358b73640 1 --2- 172.21.15.3:0/3096428996 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f634c105590 0x7f634c052c50 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f6354062df0 tx=0x7f6354094fa0 comp rx=0 tx=0).stop 2024-05-09T06:36:13.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.504+0000 7f28a930a640 1 -- 172.21.15.3:0/777743219 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f28a0032480 con 0x7f28a40b35e0 2024-05-09T06:36:13.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.504+0000 7f6358b73640 1 -- 172.21.15.3:0/3096428996 shutdown_connections 2024-05-09T06:36:13.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.504+0000 7f6358b73640 1 --2- 172.21.15.3:0/3096428996 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f634c105590 0x7f634c052c50 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:13.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.504+0000 7f6358b73640 1 --2- 172.21.15.3:0/3096428996 >> v2:172.21.15.3:3300/0 conn(0x7f634c0073f0 0x7f634c105050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:13.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.504+0000 7f6358b73640 1 -- 172.21.15.3:0/3096428996 >> 172.21.15.3:0/3096428996 conn(0x7f634c009fb0 msgr2=0x7f634c00a3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:13.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.504+0000 7f28a930a640 1 -- 172.21.15.3:0/777743219 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28a0032890 con 0x7f28a40b35e0 2024-05-09T06:36:13.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.505+0000 7ff57ba38640 1 -- 172.21.15.3:0/584871213 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff57415adf0 msgr2=0x7ff57406b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:13.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.505+0000 7ff57ba38640 1 --2- 172.21.15.3:0/584871213 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff57415adf0 0x7ff57406b250 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7ff574070440 tx=0x7ff56402c440 comp rx=0 tx=0).stop 2024-05-09T06:36:13.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.505+0000 7f0ba3ebd640 1 -- 172.21.15.3:0/218686405 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0b940504f0 msgr2=0x7f0b940e9db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:13.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.505+0000 7f0ba3ebd640 1 --2- 172.21.15.3:0/218686405 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0b940504f0 0x7f0b940e9db0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f0b8c009870 tx=0x7f0b8c02e700 comp rx=0 tx=0).stop 2024-05-09T06:36:13.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.505+0000 7ff57ba38640 1 -- 172.21.15.3:0/584871213 shutdown_connections 2024-05-09T06:36:13.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.505+0000 7ff57ba38640 1 --2- 172.21.15.3:0/584871213 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff57415adf0 0x7ff57406b250 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:13.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.505+0000 7ff57ba38640 1 --2- 172.21.15.3:0/584871213 >> v2:172.21.15.190:3300/0 conn(0x7ff574154900 0x7ff574154ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:13.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.505+0000 7ff57ba38640 1 -- 172.21.15.3:0/584871213 >> 172.21.15.3:0/584871213 conn(0x7ff5740d0a70 msgr2=0x7ff5740d0e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:13.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.505+0000 7f0ba3ebd640 1 -- 172.21.15.3:0/218686405 shutdown_connections 2024-05-09T06:36:13.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.504+0000 7f6358b73640 1 -- 172.21.15.3:0/3096428996 shutdown_connections 2024-05-09T06:36:13.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.505+0000 7f6358b73640 1 -- 172.21.15.3:0/3096428996 wait complete. 2024-05-09T06:36:13.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.505+0000 7f0ba3ebd640 1 --2- 172.21.15.3:0/218686405 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0b940504f0 0x7f0b940e9db0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:13.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.505+0000 7f0ba3ebd640 1 -- 172.21.15.3:0/218686405 >> 172.21.15.3:0/218686405 conn(0x7f0b9400a3f0 msgr2=0x7f0b9400a800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:13.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.505+0000 7f1901870640 1 --2- 172.21.15.3:0/535861900 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f18fc15e420 0x7f18fc0d1750 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f18f8009870 tx=0x7f18f802a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58951b82682f1caf server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:13.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.505+0000 7fe2fc8e7640 1 -- 172.21.15.3:0/1925562112 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe2f815c4b0 msgr2=0x7fe2f80ccf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:13.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.505+0000 7f3ffffff640 1 --2- 172.21.15.3:0/3008078279 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3ff80eacf0 0x7f3ff804fa80 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f3ff40097a0 tx=0x7f3ff402a1e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cd156d2abf11b0a3 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:13.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.505+0000 7ff57ba38640 1 -- 172.21.15.3:0/584871213 shutdown_connections 2024-05-09T06:36:13.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.505+0000 7ff57ba38640 1 -- 172.21.15.3:0/584871213 wait complete. 2024-05-09T06:36:13.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.505+0000 7fe2fc8e7640 1 --2- 172.21.15.3:0/1925562112 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe2f815c4b0 0x7fe2f80ccf50 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fe2ec009870 tx=0x7fe2ec02a210 comp rx=0 tx=0).stop 2024-05-09T06:36:13.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.505+0000 7f0ba3ebd640 1 -- 172.21.15.3:0/218686405 shutdown_connections 2024-05-09T06:36:13.512 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.505+0000 7fe2fc8e7640 1 -- 172.21.15.3:0/1925562112 shutdown_connections 2024-05-09T06:36:13.512 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.505+0000 7fe2fc8e7640 1 --2- 172.21.15.3:0/1925562112 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe2f815c4b0 0x7fe2f80ccf50 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:13.512 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.505+0000 7fe2fc8e7640 1 --2- 172.21.15.3:0/1925562112 >> v2:172.21.15.190:3300/0 conn(0x7fe2f815c0d0 0x7fe2f80cca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:13.512 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.505+0000 7fe2fc8e7640 1 -- 172.21.15.3:0/1925562112 >> 172.21.15.3:0/1925562112 conn(0x7fe2f80d04a0 msgr2=0x7fe2f80d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:13.512 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.505+0000 7fe2fc8e7640 1 -- 172.21.15.3:0/1925562112 shutdown_connections 2024-05-09T06:36:13.512 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.505+0000 7fe2fc8e7640 1 -- 172.21.15.3:0/1925562112 wait complete. 2024-05-09T06:36:13.512 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.505+0000 7f83d28f8640 1 -- 172.21.15.3:0/3993722418 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f83c4053c70 msgr2=0x7f83c4006a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:13.513 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.505+0000 7f83d28f8640 1 --2- 172.21.15.3:0/3993722418 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f83c4053c70 0x7f83c4006a60 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f83cc062df0 tx=0x7f83cc0a5e20 comp rx=0 tx=0).stop 2024-05-09T06:36:13.513 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.505+0000 7f83d28f8640 1 -- 172.21.15.3:0/3993722418 shutdown_connections 2024-05-09T06:36:13.513 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.505+0000 7f83d28f8640 1 --2- 172.21.15.3:0/3993722418 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f83c4053c70 0x7f83c4006a60 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:13.513 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.505+0000 7f83d28f8640 1 --2- 172.21.15.3:0/3993722418 >> v2:172.21.15.3:3300/0 conn(0x7f83c40501a0 0x7f83c4038080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:13.513 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.505+0000 7f3ffeffd640 1 -- 172.21.15.3:0/3008078279 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ff4037070 con 0x7f3ff80eacf0 2024-05-09T06:36:13.513 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.505+0000 7f83d28f8640 1 -- 172.21.15.3:0/3993722418 >> 172.21.15.3:0/3993722418 conn(0x7f83c4009fb0 msgr2=0x7f83c400a390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:13.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.505+0000 7f3ffeffd640 1 -- 172.21.15.3:0/3008078279 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ff4032440 con 0x7f3ff80eacf0 2024-05-09T06:36:13.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.505+0000 7f3ffeffd640 1 -- 172.21.15.3:0/3008078279 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ff4032810 con 0x7f3ff80eacf0 2024-05-09T06:36:13.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.506+0000 7f28ac570640 1 -- 172.21.15.3:0/777743219 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f28a40b35e0 msgr2=0x7f28a406d5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:13.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.506+0000 7f28ac570640 1 --2- 172.21.15.3:0/777743219 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f28a40b35e0 0x7f28a406d5a0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f28a0008870 tx=0x7f28a002c6f0 comp rx=0 tx=0).stop 2024-05-09T06:36:13.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.506+0000 7f28ac570640 1 -- 172.21.15.3:0/777743219 shutdown_connections 2024-05-09T06:36:13.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.506+0000 7f28ac570640 1 --2- 172.21.15.3:0/777743219 >> v2:172.21.15.3:3300/0 conn(0x7f28a4072f10 0x7f28a406dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:13.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.506+0000 7f28ac570640 1 --2- 172.21.15.3:0/777743219 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f28a40b35e0 0x7f28a406d5a0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:13.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.506+0000 7f28ac570640 1 -- 172.21.15.3:0/777743219 >> 172.21.15.3:0/777743219 conn(0x7f28a406b250 msgr2=0x7f28a406b660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:13.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.506+0000 7f83d28f8640 1 -- 172.21.15.3:0/3993722418 shutdown_connections 2024-05-09T06:36:13.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.506+0000 7f0ba3ebd640 1 -- 172.21.15.3:0/218686405 wait complete. 2024-05-09T06:36:13.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.506+0000 7f190086e640 1 -- 172.21.15.3:0/535861900 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18f8037070 con 0x7f18fc15e420 2024-05-09T06:36:13.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.506+0000 7f190086e640 1 -- 172.21.15.3:0/535861900 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f18f802ae50 con 0x7f18fc15e420 2024-05-09T06:36:13.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.506+0000 7f190086e640 1 -- 172.21.15.3:0/535861900 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18f8032860 con 0x7f18fc15e420 2024-05-09T06:36:13.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.506+0000 7f40053d6640 1 -- 172.21.15.3:0/3008078279 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3ff80eacf0 msgr2=0x7f3ff804fa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:13.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.506+0000 7f83d28f8640 1 -- 172.21.15.3:0/3993722418 wait complete. 2024-05-09T06:36:13.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.507+0000 7f1903ad4640 1 -- 172.21.15.3:0/535861900 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f18fc15e420 msgr2=0x7f18fc0d1750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:13.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.507+0000 7f1903ad4640 1 --2- 172.21.15.3:0/535861900 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f18fc15e420 0x7f18fc0d1750 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f18f8009870 tx=0x7f18f802a300 comp rx=0 tx=0).stop 2024-05-09T06:36:13.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.506+0000 7f40053d6640 1 --2- 172.21.15.3:0/3008078279 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3ff80eacf0 0x7f3ff804fa80 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f3ff40097a0 tx=0x7f3ff402a1e0 comp rx=0 tx=0).stop 2024-05-09T06:36:13.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.507+0000 7f1903ad4640 1 -- 172.21.15.3:0/535861900 shutdown_connections 2024-05-09T06:36:13.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.507+0000 7f1903ad4640 1 --2- 172.21.15.3:0/535861900 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f18fc15e420 0x7f18fc0d1750 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:13.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.507+0000 7f1903ad4640 1 -- 172.21.15.3:0/535861900 >> 172.21.15.3:0/535861900 conn(0x7f18fc08dbc0 msgr2=0x7f18fc09d4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:13.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.507+0000 7ff57ba38640 1 Processor -- start 2024-05-09T06:36:13.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.507+0000 7f40053d6640 1 -- 172.21.15.3:0/3008078279 shutdown_connections 2024-05-09T06:36:13.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.507+0000 7f40053d6640 1 --2- 172.21.15.3:0/3008078279 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3ff80eacf0 0x7f3ff804fa80 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:13.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.507+0000 7f1903ad4640 1 -- 172.21.15.3:0/535861900 shutdown_connections 2024-05-09T06:36:13.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.507+0000 7f40053d6640 1 -- 172.21.15.3:0/3008078279 >> 172.21.15.3:0/3008078279 conn(0x7f3ff8009fa0 msgr2=0x7f3ff800a380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:13.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.507+0000 7f1903ad4640 1 -- 172.21.15.3:0/535861900 wait complete. 2024-05-09T06:36:13.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.507+0000 7f28ac570640 1 -- 172.21.15.3:0/777743219 shutdown_connections 2024-05-09T06:36:13.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.507+0000 7f28ac570640 1 -- 172.21.15.3:0/777743219 wait complete. 2024-05-09T06:36:13.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.507+0000 7ff57ba38640 1 -- start start 2024-05-09T06:36:13.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.508+0000 7fe2fc8e7640 1 Processor -- start 2024-05-09T06:36:13.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.508+0000 7f83d28f8640 1 Processor -- start 2024-05-09T06:36:13.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.508+0000 7fe2fc8e7640 1 -- start start 2024-05-09T06:36:13.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.508+0000 7f0ba3ebd640 1 Processor -- start 2024-05-09T06:36:13.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.508+0000 7f83d28f8640 1 -- start start 2024-05-09T06:36:13.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.508+0000 7f0ba3ebd640 1 -- start start 2024-05-09T06:36:13.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.509+0000 7ff57ba38640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff574066f90 0x7ff574146c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:13.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.509+0000 7ff57ba38640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff574154900 0x7ff5740ce6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:13.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.507+0000 7f40053d6640 1 -- 172.21.15.3:0/3008078279 shutdown_connections 2024-05-09T06:36:13.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.507+0000 7f40053d6640 1 -- 172.21.15.3:0/3008078279 wait complete. 2024-05-09T06:36:13.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.509+0000 7ff57ba38640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff57415adf0 0x7ff5740cec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:13.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.509+0000 7f40053d6640 1 Processor -- start 2024-05-09T06:36:13.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.506+0000 7f6358b73640 1 Processor -- start 2024-05-09T06:36:13.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.507+0000 7f6358b73640 1 -- start start 2024-05-09T06:36:13.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.508+0000 7f6358b73640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f634c0073f0 0x7f634c0f66a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:13.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.509+0000 7ff578fd3640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff574154900 0x7ff5740ce6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:13.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.509+0000 7ff578fd3640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff574154900 0x7ff5740ce6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.3:47692/0 (socket says 172.21.15.3:47692) 2024-05-09T06:36:13.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.509+0000 7ff578fd3640 1 -- 172.21.15.3:0/1686658755 learned_addr learned my addr 172.21.15.3:0/1686658755 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:36:13.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.509+0000 7fe2fc8e7640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe2f80cf600 0x7fe2f80ca4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:13.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.509+0000 7f83d28f8640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f83c40501a0 0x7f83c40c6b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:13.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.509+0000 7f83d28f8640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f83c4053c70 0x7f83c40c2b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:13.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.508+0000 7f6358b73640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f634c054520 0x7f634c0f7030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:13.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.508+0000 7f6358b73640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f634c105590 0x7f634c0f7570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:13.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.508+0000 7f6358b73640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f634c1012a0 con 0x7f634c105590 2024-05-09T06:36:13.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.508+0000 7f6353fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f634c105590 0x7f634c0f7570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:13.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.508+0000 7f6352ffd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f634c054520 0x7f634c0f7030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:13.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.509+0000 7f83d28f8640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f83c40df220 0x7f83c40eced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:13.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.509+0000 7f83d28f8640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f83c4037910 con 0x7f83c4053c70 2024-05-09T06:36:13.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.509+0000 7f83d28f8640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f83c4006540 con 0x7f83c40501a0 2024-05-09T06:36:13.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.509+0000 7f83d28f8640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f83c4037a90 con 0x7f83c40df220 2024-05-09T06:36:13.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.508+0000 7f6352ffd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f634c054520 0x7f634c0f7030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.3:47676/0 (socket says 172.21.15.3:47676) 2024-05-09T06:36:13.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.508+0000 7f6352ffd640 1 -- 172.21.15.3:0/1857497576 learned_addr learned my addr 172.21.15.3:0/1857497576 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:36:13.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.508+0000 7f6358b73640 1 -- 172.21.15.3:0/1857497576 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f634c101120 con 0x7f634c054520 2024-05-09T06:36:13.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.508+0000 7f6358b73640 1 -- 172.21.15.3:0/1857497576 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f634c053190 con 0x7f634c0073f0 2024-05-09T06:36:13.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.509+0000 7f6352ffd640 1 -- 172.21.15.3:0/1857497576 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f634c0073f0 msgr2=0x7f634c0f66a0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-09T06:36:13.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.509+0000 7f6352ffd640 1 --2- 172.21.15.3:0/1857497576 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f634c0073f0 0x7f634c0f66a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:13.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.509+0000 7ff579fd5640 1 --2- 172.21.15.3:0/1686658755 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff57415adf0 0x7ff5740cec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:13.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.509+0000 7f6352ffd640 1 -- 172.21.15.3:0/1857497576 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f634c105590 msgr2=0x7f634c0f7570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:13.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.509+0000 7f6352ffd640 1 --2- 172.21.15.3:0/1857497576 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f634c105590 0x7f634c0f7570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:13.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.509+0000 7f6352ffd640 1 -- 172.21.15.3:0/1857497576 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f634c1a7c30 con 0x7f634c054520 2024-05-09T06:36:13.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.509+0000 7f6353fff640 1 --2- 172.21.15.3:0/1857497576 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f634c105590 0x7f634c0f7570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T06:36:13.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.509+0000 7f28ac570640 1 Processor -- start 2024-05-09T06:36:13.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.509+0000 7ff57ba38640 1 -- 172.21.15.3:0/1686658755 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7ff57406c110 con 0x7ff57415adf0 2024-05-09T06:36:13.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.509+0000 7f6352ffd640 1 --2- 172.21.15.3:0/1857497576 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f634c054520 0x7f634c0f7030 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f634000d650 tx=0x7f634000db10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:13.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.509+0000 7fe2fc8e7640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe2f815c0d0 0x7fe2f814aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:13.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.509+0000 7ff57ba38640 1 -- 172.21.15.3:0/1686658755 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7ff574155220 con 0x7ff574154900 2024-05-09T06:36:13.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.509+0000 7ff57ba38640 1 -- 172.21.15.3:0/1686658755 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7ff57406c3f0 con 0x7ff574066f90 2024-05-09T06:36:13.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.510+0000 7f28ac570640 1 -- start start 2024-05-09T06:36:13.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.509+0000 7fe2fc8e7640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe2f815c4b0 0x7fe2f814af80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:13.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.510+0000 7fe2f659c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe2f80cf600 0x7fe2f80ca4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:13.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.510+0000 7fe2f659c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe2f80cf600 0x7fe2f80ca4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.3:47704/0 (socket says 172.21.15.3:47704) 2024-05-09T06:36:13.529 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.509+0000 7f83d18f6640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f83c40501a0 0x7f83c40c6b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:13.529 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.510+0000 7f83d18f6640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f83c40501a0 0x7f83c40c6b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.3:47694/0 (socket says 172.21.15.3:47694) 2024-05-09T06:36:13.529 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.510+0000 7f83d18f6640 1 -- 172.21.15.3:0/2359566183 learned_addr learned my addr 172.21.15.3:0/2359566183 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:36:13.529 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.510+0000 7f1903ad4640 1 Processor -- start 2024-05-09T06:36:13.529 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.510+0000 7fe2f659c640 1 -- 172.21.15.3:0/475537639 learned_addr learned my addr 172.21.15.3:0/475537639 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:36:13.529 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.510+0000 7f83d20f7640 1 --2- 172.21.15.3:0/2359566183 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f83c40df220 0x7f83c40eced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:13.530 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.510+0000 7f83d10f5640 1 --2- 172.21.15.3:0/2359566183 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f83c4053c70 0x7f83c40c2b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:13.530 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.510+0000 7fe2fc8e7640 1 -- 172.21.15.3:0/475537639 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fe2f806fde0 con 0x7fe2f815c4b0 2024-05-09T06:36:13.530 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.510+0000 7fe2fc8e7640 1 -- 172.21.15.3:0/475537639 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fe2f806fc60 con 0x7fe2f80cf600 2024-05-09T06:36:13.530 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.510+0000 7fe2fc8e7640 1 -- 172.21.15.3:0/475537639 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fe2f806ff60 con 0x7fe2f815c0d0 2024-05-09T06:36:13.530 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.510+0000 7ff5797d4640 1 --2- 172.21.15.3:0/1686658755 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff574066f90 0x7ff574146c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:13.531 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.510+0000 7f0ba3ebd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0b940504f0 0x7f0b940dbb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:13.531 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.510+0000 7f0ba3ebd640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0b940deff0 0x7f0b940ccc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:13.531 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.510+0000 7fe2f6d9d640 1 --2- 172.21.15.3:0/475537639 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe2f815c4b0 0x7fe2f814af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:13.531 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.510+0000 7f0ba3ebd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0b940df3d0 0x7f0b940d8c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:13.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.510+0000 7f633ffff640 1 -- 172.21.15.3:0/1857497576 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f634000b6c0 con 0x7f634c054520 2024-05-09T06:36:13.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.510+0000 7f633ffff640 1 -- 172.21.15.3:0/1857497576 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6340019070 con 0x7f634c054520 2024-05-09T06:36:13.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.510+0000 7f633ffff640 1 -- 172.21.15.3:0/1857497576 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f634000baa0 con 0x7f634c054520 2024-05-09T06:36:13.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.510+0000 7f6358b73640 1 -- 172.21.15.3:0/1857497576 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f634c04a810 con 0x7f634c054520 2024-05-09T06:36:13.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.510+0000 7ff579fd5640 1 -- 172.21.15.3:0/1686658755 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff574066f90 msgr2=0x7ff574146c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:13.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.510+0000 7ff579fd5640 1 --2- 172.21.15.3:0/1686658755 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff574066f90 0x7ff574146c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:13.533 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.510+0000 7f0ba3ebd640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f0b94007b90 con 0x7f0b940df3d0 2024-05-09T06:36:13.533 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.510+0000 7f0ba3ebd640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f0b940076b0 con 0x7f0b940504f0 2024-05-09T06:36:13.533 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.510+0000 7ff579fd5640 1 -- 172.21.15.3:0/1686658755 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff574154900 msgr2=0x7ff5740ce6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:13.533 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.510+0000 7ff579fd5640 1 --2- 172.21.15.3:0/1686658755 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff574154900 0x7ff5740ce6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:13.533 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.510+0000 7ff579fd5640 1 -- 172.21.15.3:0/1686658755 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff57414a350 con 0x7ff57415adf0 2024-05-09T06:36:13.533 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.510+0000 7f0ba3ebd640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f0b94039070 con 0x7f0b940deff0 2024-05-09T06:36:13.534 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.510+0000 7fe2f659c640 1 -- 172.21.15.3:0/475537639 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe2f815c0d0 msgr2=0x7fe2f814aa40 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-09T06:36:13.534 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.510+0000 7fe2f659c640 1 --2- 172.21.15.3:0/475537639 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe2f815c0d0 0x7fe2f814aa40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:13.534 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.510+0000 7fe2f659c640 1 -- 172.21.15.3:0/475537639 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe2f815c4b0 msgr2=0x7fe2f814af80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:13.534 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.511+0000 7fe2f659c640 1 --2- 172.21.15.3:0/475537639 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe2f815c4b0 0x7fe2f814af80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:13.534 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.511+0000 7f83d18f6640 1 -- 172.21.15.3:0/2359566183 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f83c40df220 msgr2=0x7f83c40eced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:13.534 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.511+0000 7f83d18f6640 1 --2- 172.21.15.3:0/2359566183 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f83c40df220 0x7f83c40eced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:13.535 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.511+0000 7f83d18f6640 1 -- 172.21.15.3:0/2359566183 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f83c4053c70 msgr2=0x7f83c40c2b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:13.535 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.511+0000 7f83d18f6640 1 --2- 172.21.15.3:0/2359566183 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f83c4053c70 0x7f83c40c2b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:13.535 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.511+0000 7f0ba1458640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0b940deff0 0x7f0b940ccc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:13.535 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.511+0000 7f0ba245a640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0b940df3d0 0x7f0b940d8c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:13.535 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.511+0000 7f0ba245a640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0b940df3d0 0x7f0b940d8c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49522/0 (socket says 172.21.15.3:49522) 2024-05-09T06:36:13.535 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.511+0000 7f0ba245a640 1 -- 172.21.15.3:0/4243570696 learned_addr learned my addr 172.21.15.3:0/4243570696 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:36:13.535 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.511+0000 7f0ba1458640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0b940deff0 0x7f0b940ccc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.3:45016/0 (socket says 172.21.15.3:45016) 2024-05-09T06:36:13.535 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.511+0000 7fe2f659c640 1 -- 172.21.15.3:0/475537639 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2f814b4c0 con 0x7fe2f80cf600 2024-05-09T06:36:13.535 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.511+0000 7fe2f6d9d640 1 --2- 172.21.15.3:0/475537639 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe2f815c4b0 0x7fe2f814af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:36:13.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.511+0000 7f83d18f6640 1 -- 172.21.15.3:0/2359566183 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83c40c7160 con 0x7f83c40501a0 2024-05-09T06:36:13.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.511+0000 7f28ac570640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f28a4072f10 0x7f28a4146730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:13.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.511+0000 7f83d10f5640 1 --2- 172.21.15.3:0/2359566183 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f83c4053c70 0x7f83c40c2b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:36:13.537 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.510+0000 7f1903ad4640 1 -- start start 2024-05-09T06:36:13.537 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.511+0000 7ff579fd5640 1 --2- 172.21.15.3:0/1686658755 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff57415adf0 0x7ff5740cec10 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7ff56c00c780 tx=0x7ff56c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:13.537 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.511+0000 7f28ac570640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f28a40b35e0 0x7f28a4158990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:13.537 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.510+0000 7f40053d6640 1 -- start start 2024-05-09T06:36:13.537 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.511+0000 7f40053d6640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3ff8006ee0 0x7f3ff804e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:13.538 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.511+0000 7f83d18f6640 1 --2- 172.21.15.3:0/2359566183 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f83c40501a0 0x7f83c40c6b60 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f83c800b420 tx=0x7f83c800b8e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:13.538 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.511+0000 7f28ac570640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f28a4159ed0 0x7f28a4158ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:13.538 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.511+0000 7f3ffffff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3ff8006ee0 0x7f3ff804e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:13.538 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.511+0000 7f3ffffff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3ff8006ee0 0x7f3ff804e3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49524/0 (socket says 172.21.15.3:49524) 2024-05-09T06:36:13.539 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.511+0000 7f6358b73640 1 -- 172.21.15.3:0/1857497576 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f634c0369e0 con 0x7f634c054520 2024-05-09T06:36:13.539 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.511+0000 7f0ba245a640 1 -- 172.21.15.3:0/4243570696 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0b940deff0 msgr2=0x7f0b940ccc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:13.539 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.511+0000 7f0ba245a640 1 --2- 172.21.15.3:0/4243570696 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0b940deff0 0x7f0b940ccc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:13.539 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.511+0000 7f40053d6640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3ff804ffc0 0x7f3ff80da7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:13.539 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.511+0000 7f0ba245a640 1 -- 172.21.15.3:0/4243570696 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0b940504f0 msgr2=0x7f0b940dbb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:13.539 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.511+0000 7f0ba245a640 1 --2- 172.21.15.3:0/4243570696 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0b940504f0 0x7f0b940dbb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:13.540 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.511+0000 7f0ba245a640 1 -- 172.21.15.3:0/4243570696 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b940d94b0 con 0x7f0b940df3d0 2024-05-09T06:36:13.540 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.511+0000 7fe2f659c640 1 --2- 172.21.15.3:0/475537639 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe2f80cf600 0x7fe2f80ca4c0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fe2f000ccd0 tx=0x7fe2f000a5c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:13.540 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.511+0000 7f3ffffff640 1 -- 172.21.15.3:0/3464323938 learned_addr learned my addr 172.21.15.3:0/3464323938 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:36:13.540 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.511+0000 7f83c27fc640 1 -- 172.21.15.3:0/2359566183 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83c8011830 con 0x7f83c40501a0 2024-05-09T06:36:13.541 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.511+0000 7f28aa30c640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f28a4072f10 0x7f28a4146730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:13.541 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.512+0000 7f28aa30c640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f28a4072f10 0x7f28a4146730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.3:45038/0 (socket says 172.21.15.3:45038) 2024-05-09T06:36:13.541 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.512+0000 7f40053d6640 1 --2- 172.21.15.3:0/3464323938 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3ff80eacf0 0x7f3ff80dad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:13.541 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.512+0000 7f40053d6640 1 -- 172.21.15.3:0/3464323938 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f3ff8054930 con 0x7f3ff8006ee0 2024-05-09T06:36:13.541 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.512+0000 7f40053d6640 1 -- 172.21.15.3:0/3464323938 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f3ff80547b0 con 0x7f3ff804ffc0 2024-05-09T06:36:13.541 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.512+0000 7f40053d6640 1 -- 172.21.15.3:0/3464323938 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f3ff804f540 con 0x7f3ff80eacf0 2024-05-09T06:36:13.541 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.512+0000 7f3fff7fe640 1 --2- 172.21.15.3:0/3464323938 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3ff804ffc0 0x7f3ff80da7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:13.542 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.512+0000 7f0ba245a640 1 --2- 172.21.15.3:0/4243570696 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0b940df3d0 0x7f0b940d8c80 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f0b8c009870 tx=0x7f0b8c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:13.542 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.512+0000 7f83d28f8640 1 -- 172.21.15.3:0/2359566183 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83c40ec3f0 con 0x7f83c40501a0 2024-05-09T06:36:13.542 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.512+0000 7f28a9b0b640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f28a40b35e0 0x7f28a4158990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:13.542 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.512+0000 7f28aa30c640 1 -- 172.21.15.3:0/2160528106 learned_addr learned my addr 172.21.15.3:0/2160528106 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:36:13.543 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.512+0000 7f83d28f8640 1 -- 172.21.15.3:0/2359566183 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83c40ec0a0 con 0x7f83c40501a0 2024-05-09T06:36:13.543 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.512+0000 7f83c27fc640 1 -- 172.21.15.3:0/2359566183 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83c8016070 con 0x7f83c40501a0 2024-05-09T06:36:13.543 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.512+0000 7f83c27fc640 1 -- 172.21.15.3:0/2359566183 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83c8011c10 con 0x7f83c40501a0 2024-05-09T06:36:13.543 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.512+0000 7ff569ffb640 1 -- 172.21.15.3:0/1686658755 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff56c016020 con 0x7ff57415adf0 2024-05-09T06:36:13.543 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.512+0000 7ff569ffb640 1 -- 172.21.15.3:0/1686658755 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff56c00aa50 con 0x7ff57415adf0 2024-05-09T06:36:13.543 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.512+0000 7ff569ffb640 1 -- 172.21.15.3:0/1686658755 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff56c00ce30 con 0x7ff57415adf0 2024-05-09T06:36:13.544 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.512+0000 7f28ac570640 1 -- 172.21.15.3:0/2160528106 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f28a40acd60 con 0x7f28a4159ed0 2024-05-09T06:36:13.544 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.512+0000 7f28ac570640 1 -- 172.21.15.3:0/2160528106 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f28a406d080 con 0x7f28a40b35e0 2024-05-09T06:36:13.544 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.512+0000 7f28ac570640 1 -- 172.21.15.3:0/2160528106 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f28a406bd70 con 0x7f28a4072f10 2024-05-09T06:36:13.544 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.512+0000 7f28aab0d640 1 --2- 172.21.15.3:0/2160528106 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f28a4159ed0 0x7f28a4158ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:13.544 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.512+0000 7f3fff7fe640 1 -- 172.21.15.3:0/3464323938 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3ff80eacf0 msgr2=0x7f3ff80dad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:13.545 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.512+0000 7f3fff7fe640 1 --2- 172.21.15.3:0/3464323938 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3ff80eacf0 0x7f3ff80dad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:13.545 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.512+0000 7f3fff7fe640 1 -- 172.21.15.3:0/3464323938 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3ff8006ee0 msgr2=0x7f3ff804e3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:13.545 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.512+0000 7f3fff7fe640 1 --2- 172.21.15.3:0/3464323938 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3ff8006ee0 0x7f3ff804e3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:13.545 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.512+0000 7f3fff7fe640 1 -- 172.21.15.3:0/3464323938 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ff80db270 con 0x7f3ff804ffc0 2024-05-09T06:36:13.545 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.512+0000 7f6358b73640 1 -- 172.21.15.3:0/1857497576 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f634c036be0 con 0x7f634c054520 2024-05-09T06:36:13.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.513+0000 7f28aa30c640 1 -- 172.21.15.3:0/2160528106 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f28a40b35e0 msgr2=0x7f28a4158990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:13.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.513+0000 7f28aa30c640 1 --2- 172.21.15.3:0/2160528106 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f28a40b35e0 0x7f28a4158990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:13.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.513+0000 7f28aa30c640 1 -- 172.21.15.3:0/2160528106 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f28a4159ed0 msgr2=0x7f28a4158ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:13.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.513+0000 7f28aa30c640 1 --2- 172.21.15.3:0/2160528106 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f28a4159ed0 0x7f28a4158ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:13.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.513+0000 7f28aa30c640 1 -- 172.21.15.3:0/2160528106 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28a41f34d0 con 0x7f28a4072f10 2024-05-09T06:36:13.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.513+0000 7f28aab0d640 1 --2- 172.21.15.3:0/2160528106 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f28a4159ed0 0x7f28a4158ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:36:13.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.513+0000 7f28aa30c640 1 --2- 172.21.15.3:0/2160528106 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f28a4072f10 0x7f28a4146730 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f289c00c7b0 tx=0x7f289c00cc70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:13.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.513+0000 7f83d28f8640 1 -- 172.21.15.3:0/2359566183 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f83b4001030 con 0x7f83c40501a0 2024-05-09T06:36:13.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.513+0000 7f0b927fc640 1 -- 172.21.15.3:0/4243570696 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b8c045070 con 0x7f0b940df3d0 2024-05-09T06:36:13.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.512+0000 7fe2e6ffd640 1 -- 172.21.15.3:0/475537639 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2f00039c0 con 0x7fe2f80cf600 2024-05-09T06:36:13.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.512+0000 7fe2e6ffd640 1 -- 172.21.15.3:0/475537639 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe2f0003b60 con 0x7fe2f80cf600 2024-05-09T06:36:13.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.512+0000 7fe2fc8e7640 1 -- 172.21.15.3:0/475537639 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2f80c7bd0 con 0x7fe2f80cf600 2024-05-09T06:36:13.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.512+0000 7fe2fc8e7640 1 -- 172.21.15.3:0/475537639 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2f80c8190 con 0x7fe2f80cf600 2024-05-09T06:36:13.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.513+0000 7fe2e6ffd640 1 -- 172.21.15.3:0/475537639 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2f00039c0 con 0x7fe2f80cf600 2024-05-09T06:36:13.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.513+0000 7f1903ad4640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f18fc0cfdd0 0x7f18fc15cea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:13.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.513+0000 7f1903ad4640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f18fc0d3f40 0x7f18fc0b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:13.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.513+0000 7f1903ad4640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f18fc15e420 0x7f18fc160380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:13.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.513+0000 7f1903ad4640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f18fc15e800 con 0x7f18fc15e420 2024-05-09T06:36:13.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.513+0000 7f1903ad4640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f18fc0d28f0 con 0x7f18fc0cfdd0 2024-05-09T06:36:13.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.513+0000 7f1903ad4640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f18fc15e980 con 0x7f18fc0d3f40 2024-05-09T06:36:13.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.513+0000 7f289affd640 1 -- 172.21.15.3:0/2160528106 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f289c00a640 con 0x7f28a4072f10 2024-05-09T06:36:13.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.513+0000 7f28ac570640 1 -- 172.21.15.3:0/2160528106 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28a409ac40 con 0x7f28a4072f10 2024-05-09T06:36:13.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.513+0000 7f3fff7fe640 1 --2- 172.21.15.3:0/3464323938 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3ff804ffc0 0x7f3ff80da7f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f4000000b60 tx=0x7f4000068dc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:13.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.513+0000 7f3ffcff9640 1 -- 172.21.15.3:0/3464323938 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4000073860 con 0x7f3ff804ffc0 2024-05-09T06:36:13.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.513+0000 7f3ffcff9640 1 -- 172.21.15.3:0/3464323938 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4000071920 con 0x7f3ff804ffc0 2024-05-09T06:36:13.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.513+0000 7f3ffcff9640 1 -- 172.21.15.3:0/3464323938 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4000071d50 con 0x7f3ff804ffc0 2024-05-09T06:36:13.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.513+0000 7f40053d6640 1 -- 172.21.15.3:0/3464323938 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ff8049f50 con 0x7f3ff804ffc0 2024-05-09T06:36:13.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.513+0000 7f40053d6640 1 -- 172.21.15.3:0/3464323938 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ff804a510 con 0x7f3ff804ffc0 2024-05-09T06:36:13.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.514+0000 7ff57ba38640 1 -- 172.21.15.3:0/1686658755 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5740aa3c0 con 0x7ff57415adf0 2024-05-09T06:36:13.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.513+0000 7f289affd640 1 -- 172.21.15.3:0/2160528106 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f289c00a7e0 con 0x7f28a4072f10 2024-05-09T06:36:13.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.513+0000 7f28ac570640 1 -- 172.21.15.3:0/2160528106 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f28a409b200 con 0x7f28a4072f10 2024-05-09T06:36:13.550 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.513+0000 7f0ba3ebd640 1 -- 172.21.15.3:0/4243570696 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b940d9790 con 0x7f0b940df3d0 2024-05-09T06:36:13.550 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.513+0000 7f0ba3ebd640 1 -- 172.21.15.3:0/4243570696 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b940d9d50 con 0x7f0b940df3d0 2024-05-09T06:36:13.550 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.514+0000 7f0b927fc640 1 -- 172.21.15.3:0/4243570696 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b8c041870 con 0x7f0b940df3d0 2024-05-09T06:36:13.550 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.514+0000 7f0b927fc640 1 -- 172.21.15.3:0/4243570696 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b8c03a040 con 0x7f0b940df3d0 2024-05-09T06:36:13.550 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.514+0000 7ff57ba38640 1 -- 172.21.15.3:0/1686658755 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff574147860 con 0x7ff57415adf0 2024-05-09T06:36:13.550 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.514+0000 7f1901870640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f18fc0cfdd0 0x7f18fc15cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:13.551 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.514+0000 7f633ffff640 1 -- 172.21.15.3:0/1857497576 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f634000bc40 con 0x7f634c054520 2024-05-09T06:36:13.551 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.515+0000 7f289affd640 1 -- 172.21.15.3:0/2160528106 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f289c00abb0 con 0x7f28a4072f10 2024-05-09T06:36:13.551 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.515+0000 7f40053d6640 1 -- 172.21.15.3:0/3464323938 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f3ff80052b0 con 0x7f3ff804ffc0 2024-05-09T06:36:13.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.514+0000 7f1901870640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f18fc0cfdd0 0x7f18fc15cea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.3:47726/0 (socket says 172.21.15.3:47726) 2024-05-09T06:36:13.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.514+0000 7f1901870640 1 -- 172.21.15.3:0/1720337932 learned_addr learned my addr 172.21.15.3:0/1720337932 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:36:13.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.514+0000 7f1902071640 1 --2- 172.21.15.3:0/1720337932 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f18fc15e420 0x7f18fc160380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:13.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.515+0000 7ff5577fe640 1 -- 172.21.15.3:0/1686658755 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ff5740682e0 con 0x7ff57415adf0 2024-05-09T06:36:13.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.515+0000 7f83c27fc640 1 -- 172.21.15.3:0/2359566183 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f83c8004020 con 0x7f83c40501a0 2024-05-09T06:36:13.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.515+0000 7f83c27fc640 1 --2- 172.21.15.3:0/2359566183 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f83b006be30 0x7f83b006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:13.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.515+0000 7f83c27fc640 1 -- 172.21.15.3:0/2359566183 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f83c808ae00 con 0x7f83c40501a0 2024-05-09T06:36:13.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.516+0000 7f83c27fc640 1 --2- 172.21.15.3:0/2359566183 >> [v2:172.21.15.190:6800/2143624576,v1:172.21.15.190:6801/2143624576] conn(0x7f83b0074050 0x7f83b0076470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:13.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.516+0000 7f83c27fc640 1 -- 172.21.15.3:0/2359566183 --> [v2:172.21.15.190:6800/2143624576,v1:172.21.15.190:6801/2143624576] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f83c801fec0 con 0x7f83b0074050 2024-05-09T06:36:13.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.516+0000 7f83c27fc640 1 -- 172.21.15.3:0/2359566183 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f83c808b1f0 con 0x7f83c40501a0 2024-05-09T06:36:13.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.514+0000 7f0ba3ebd640 1 -- 172.21.15.3:0/4243570696 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f0b94005700 con 0x7f0b940df3d0 2024-05-09T06:36:13.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.516+0000 7fe2fc8e7640 1 -- 172.21.15.3:0/475537639 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe2f804f5f0 con 0x7fe2f80cf600 2024-05-09T06:36:13.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.515+0000 7f633ffff640 1 --2- 172.21.15.3:0/1857497576 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f633806bf00 0x7f633806e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:13.554 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.515+0000 7f633ffff640 1 -- 172.21.15.3:0/1857497576 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f634008b5c0 con 0x7f634c054520 2024-05-09T06:36:13.554 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.515+0000 7f633ffff640 1 --2- 172.21.15.3:0/1857497576 >> [v2:172.21.15.3:6808/801295451,v1:172.21.15.3:6809/801295451] conn(0x7f6338074100 0x7f6338076520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:13.554 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.515+0000 7f633ffff640 1 -- 172.21.15.3:0/1857497576 --> [v2:172.21.15.3:6808/801295451,v1:172.21.15.3:6809/801295451] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f634000ee50 con 0x7f6338074100 2024-05-09T06:36:13.554 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.516+0000 7f6353fff640 1 --2- 172.21.15.3:0/1857497576 >> [v2:172.21.15.3:6808/801295451,v1:172.21.15.3:6809/801295451] conn(0x7f6338074100 0x7f6338076520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:13.554 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.516+0000 7f633ffff640 1 -- 172.21.15.3:0/1857497576 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f6340092050 con 0x7f634c054520 2024-05-09T06:36:13.554 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.514+0000 7f1901870640 1 -- 172.21.15.3:0/1720337932 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f18fc0d3f40 msgr2=0x7f18fc0b77a0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-09T06:36:13.554 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.514+0000 7f1901870640 1 --2- 172.21.15.3:0/1720337932 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f18fc0d3f40 0x7f18fc0b77a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:13.554 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.514+0000 7f1901870640 1 -- 172.21.15.3:0/1720337932 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f18fc15e420 msgr2=0x7f18fc160380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:13.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.514+0000 7f1901870640 1 --2- 172.21.15.3:0/1720337932 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f18fc15e420 0x7f18fc160380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:13.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.514+0000 7f1901870640 1 -- 172.21.15.3:0/1720337932 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18fc0b8220 con 0x7f18fc0cfdd0 2024-05-09T06:36:13.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.514+0000 7f1902071640 1 --2- 172.21.15.3:0/1720337932 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f18fc15e420 0x7f18fc160380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:36:13.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.515+0000 7f1901870640 1 --2- 172.21.15.3:0/1720337932 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f18fc0cfdd0 0x7f18fc15cea0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f18f8009870 tx=0x7f18f80046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:13.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.516+0000 7f18f27fc640 1 -- 172.21.15.3:0/1720337932 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18f8041070 con 0x7f18fc0cfdd0 2024-05-09T06:36:13.556 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.516+0000 7f18f27fc640 1 -- 172.21.15.3:0/1720337932 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f18f803c870 con 0x7f18fc0cfdd0 2024-05-09T06:36:13.556 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.516+0000 7f18f27fc640 1 -- 172.21.15.3:0/1720337932 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18f8037040 con 0x7f18fc0cfdd0 2024-05-09T06:36:13.556 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.516+0000 7f1903ad4640 1 -- 172.21.15.3:0/1720337932 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18fc0b8500 con 0x7f18fc0cfdd0 2024-05-09T06:36:13.556 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.516+0000 7f1903ad4640 1 -- 172.21.15.3:0/1720337932 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18fc0ccba0 con 0x7f18fc0cfdd0 2024-05-09T06:36:13.556 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.516+0000 7f289affd640 1 -- 172.21.15.3:0/2160528106 <== mon.2 v2:172.21.15.190:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f289c007070 con 0x7f28a4072f10 2024-05-09T06:36:13.556 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.516+0000 7f3ffcff9640 1 -- 172.21.15.3:0/3464323938 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4000093400 con 0x7f3ff804ffc0 2024-05-09T06:36:13.556 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.516+0000 7fe2e6ffd640 1 -- 172.21.15.3:0/475537639 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe2f0004020 con 0x7fe2f80cf600 2024-05-09T06:36:13.557 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.517+0000 7f289affd640 1 --2- 172.21.15.3:0/2160528106 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f288806be30 0x7f288806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:13.557 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.517+0000 7f3ffcff9640 1 --2- 172.21.15.3:0/3464323938 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f3fe406be30 0x7f3fe406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:13.557 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.517+0000 7f289affd640 1 -- 172.21.15.3:0/2160528106 <== mon.2 v2:172.21.15.190:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f289c08acc0 con 0x7f28a4072f10 2024-05-09T06:36:13.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.517+0000 7f28a9b0b640 1 --2- 172.21.15.3:0/2160528106 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f288806be30 0x7f288806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:13.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.517+0000 7f83d10f5640 1 --2- 172.21.15.3:0/2359566183 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f83b006be30 0x7f83b006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:13.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.517+0000 7f83d20f7640 1 --2- 172.21.15.3:0/2359566183 >> [v2:172.21.15.190:6800/2143624576,v1:172.21.15.190:6801/2143624576] conn(0x7f83b0074050 0x7f83b0076470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:13.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.517+0000 7f3ffcff9640 1 -- 172.21.15.3:0/3464323938 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f4000106d80 con 0x7f3ff804ffc0 2024-05-09T06:36:13.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.517+0000 7f3ffffff640 1 --2- 172.21.15.3:0/3464323938 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f3fe406be30 0x7f3fe406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:13.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.517+0000 7f3ffcff9640 1 --2- 172.21.15.3:0/3464323938 >> [v2:172.21.15.3:6816/1897849827,v1:172.21.15.3:6817/1897849827] conn(0x7f3fe4074050 0x7f3fe4076470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:13.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.517+0000 7f3ffcff9640 1 -- 172.21.15.3:0/3464323938 --> [v2:172.21.15.3:6816/1897849827,v1:172.21.15.3:6817/1897849827] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3fe4076b20 con 0x7f3fe4074050 2024-05-09T06:36:13.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.517+0000 7f3ffcff9640 1 -- 172.21.15.3:0/3464323938 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f40001093e0 con 0x7f3ff804ffc0 2024-05-09T06:36:13.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.517+0000 7f6353fff640 1 --2- 172.21.15.3:0/1857497576 >> [v2:172.21.15.3:6808/801295451,v1:172.21.15.3:6809/801295451] conn(0x7f6338074100 0x7f6338076520 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:13.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.517+0000 7fe2e6ffd640 1 --2- 172.21.15.3:0/475537639 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fe2d806c030 0x7fe2d806e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:13.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.517+0000 7f1903ad4640 1 -- 172.21.15.3:0/1720337932 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f18fc069550 con 0x7f18fc0cfdd0 2024-05-09T06:36:13.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.517+0000 7fe2e6ffd640 1 -- 172.21.15.3:0/475537639 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fe2f0057240 con 0x7fe2f80cf600 2024-05-09T06:36:13.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.518+0000 7f63537fe640 1 --2- 172.21.15.3:0/1857497576 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f633806bf00 0x7f633806e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:13.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.518+0000 7fe2e6ffd640 1 --2- 172.21.15.3:0/475537639 >> [v2:172.21.15.190:6824/342414248,v1:172.21.15.190:6825/342414248] conn(0x7fe2d8074d60 0x7fe2d8077200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:13.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.518+0000 7fe2e6ffd640 1 -- 172.21.15.3:0/475537639 --> [v2:172.21.15.190:6824/342414248,v1:172.21.15.190:6825/342414248] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe2d80778d0 con 0x7fe2d8074d60 2024-05-09T06:36:13.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.518+0000 7fe2e6ffd640 1 -- 172.21.15.3:0/475537639 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fe2f0055bd0 con 0x7fe2f80cf600 2024-05-09T06:36:13.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.518+0000 7fe2f5d9b640 1 --2- 172.21.15.3:0/475537639 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fe2d806c030 0x7fe2d806e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:13.562 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.518+0000 7f4004bd5640 1 --2- 172.21.15.3:0/3464323938 >> [v2:172.21.15.3:6816/1897849827,v1:172.21.15.3:6817/1897849827] conn(0x7f3fe4074050 0x7f3fe4076470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:13.562 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.518+0000 7fe2f6d9d640 1 --2- 172.21.15.3:0/475537639 >> [v2:172.21.15.190:6824/342414248,v1:172.21.15.190:6825/342414248] conn(0x7fe2d8074d60 0x7fe2d8077200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:13.562 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.518+0000 7f28a9b0b640 1 --2- 172.21.15.3:0/2160528106 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f288806be30 0x7f288806e2f0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f28a0037000 tx=0x7f28a000b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:13.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.518+0000 7f28ac570640 1 --2- 172.21.15.3:0/2160528106 >> [v2:172.21.15.3:6824/1903389857,v1:172.21.15.3:6825/1903389857] conn(0x7f288c001650 0x7f288c003b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:13.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.518+0000 7f28ac570640 1 -- 172.21.15.3:0/2160528106 --> [v2:172.21.15.3:6824/1903389857,v1:172.21.15.3:6825/1903389857] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f288c006bb0 con 0x7f288c001650 2024-05-09T06:36:13.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.518+0000 7f4004bd5640 1 --2- 172.21.15.3:0/3464323938 >> [v2:172.21.15.3:6816/1897849827,v1:172.21.15.3:6817/1897849827] conn(0x7f3fe4074050 0x7f3fe4076470 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:13.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.518+0000 7f83d10f5640 1 --2- 172.21.15.3:0/2359566183 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f83b006be30 0x7f83b006e2f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f83c40e2920 tx=0x7f83bc008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:13.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.519+0000 7f63537fe640 1 --2- 172.21.15.3:0/1857497576 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f633806bf00 0x7f633806e3c0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f6348000f60 tx=0x7f6348007880 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:13.564 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.519+0000 7ff569ffb640 1 -- 172.21.15.3:0/1686658755 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff56c019020 con 0x7ff57415adf0 2024-05-09T06:36:13.564 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.519+0000 7f28aab0d640 1 --2- 172.21.15.3:0/2160528106 >> [v2:172.21.15.3:6824/1903389857,v1:172.21.15.3:6825/1903389857] conn(0x7f288c001650 0x7f288c003b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:13.564 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.519+0000 7f83d20f7640 1 --2- 172.21.15.3:0/2359566183 >> [v2:172.21.15.190:6800/2143624576,v1:172.21.15.190:6801/2143624576] conn(0x7f83b0074050 0x7f83b0076470 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:13.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.519+0000 7f633ffff640 1 -- 172.21.15.3:0/1857497576 <== osd.2 v2:172.21.15.3:6808/801295451 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f634000ee50 con 0x7f6338074100 2024-05-09T06:36:13.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.519+0000 7f3ffffff640 1 --2- 172.21.15.3:0/3464323938 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f3fe406be30 0x7f3fe406e2f0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f3ff4009560 tx=0x7f3ff402e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:13.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.519+0000 7fe2f6d9d640 1 --2- 172.21.15.3:0/475537639 >> [v2:172.21.15.190:6824/342414248,v1:172.21.15.190:6825/342414248] conn(0x7fe2d8074d60 0x7fe2d8077200 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:13.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.519+0000 7fe2f5d9b640 1 --2- 172.21.15.3:0/475537639 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fe2d806c030 0x7fe2d806e4f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fe2f81fc6a0 tx=0x7fe2e8006b50 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:13.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.519+0000 7f18f27fc640 1 -- 172.21.15.3:0/1720337932 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f18f8044070 con 0x7f18fc0cfdd0 2024-05-09T06:36:13.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.519+0000 7f0b927fc640 1 -- 172.21.15.3:0/4243570696 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0b8c048070 con 0x7f0b940df3d0 2024-05-09T06:36:13.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.519+0000 7ff569ffb640 1 --2- 172.21.15.3:0/1686658755 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7ff55806bf00 0x7ff55806e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:13.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.519+0000 7ff569ffb640 1 -- 172.21.15.3:0/1686658755 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7ff56c08b290 con 0x7ff57415adf0 2024-05-09T06:36:13.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.519+0000 7f28aab0d640 1 --2- 172.21.15.3:0/2160528106 >> [v2:172.21.15.3:6824/1903389857,v1:172.21.15.3:6825/1903389857] conn(0x7f288c001650 0x7f288c003b10 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:13.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.519+0000 7ff569ffb640 1 --2- 172.21.15.3:0/1686658755 >> [v2:172.21.15.3:6800/926805498,v1:172.21.15.3:6801/926805498] conn(0x7ff558074120 0x7ff558076540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:13.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.519+0000 7ff569ffb640 1 -- 172.21.15.3:0/1686658755 --> [v2:172.21.15.3:6800/926805498,v1:172.21.15.3:6801/926805498] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff558076bf0 con 0x7ff558074120 2024-05-09T06:36:13.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.519+0000 7ff569ffb640 1 -- 172.21.15.3:0/1686658755 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7ff56c08b680 con 0x7ff57415adf0 2024-05-09T06:36:13.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.519+0000 7ff5797d4640 1 --2- 172.21.15.3:0/1686658755 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7ff55806bf00 0x7ff55806e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:13.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.519+0000 7f18f27fc640 1 --2- 172.21.15.3:0/1720337932 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f18e006bf00 0x7f18e006e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:13.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.520+0000 7f0b927fc640 1 --2- 172.21.15.3:0/4243570696 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f0b8006bf00 0x7f0b8006e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:13.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.520+0000 7f3ffcff9640 1 -- 172.21.15.3:0/3464323938 <== osd.0 v2:172.21.15.3:6816/1897849827 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3fe4076b20 con 0x7f3fe4074050 2024-05-09T06:36:13.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.520+0000 7ff578fd3640 1 --2- 172.21.15.3:0/1686658755 >> [v2:172.21.15.3:6800/926805498,v1:172.21.15.3:6801/926805498] conn(0x7ff558074120 0x7ff558076540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:13.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.520+0000 7f83c27fc640 1 -- 172.21.15.3:0/2359566183 <== osd.4 v2:172.21.15.190:6800/2143624576 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f83c801fec0 con 0x7f83b0074050 2024-05-09T06:36:13.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.520+0000 7fe2e6ffd640 1 -- 172.21.15.3:0/475537639 <== osd.6 v2:172.21.15.190:6824/342414248 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe2d80778d0 con 0x7fe2d8074d60 2024-05-09T06:36:13.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.520+0000 7f18f27fc640 1 -- 172.21.15.3:0/1720337932 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f18f80ab910 con 0x7f18fc0cfdd0 2024-05-09T06:36:13.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.520+0000 7f0ba1c59640 1 --2- 172.21.15.3:0/4243570696 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f0b8006bf00 0x7f0b8006e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:13.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.520+0000 7f0b927fc640 1 -- 172.21.15.3:0/4243570696 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f0b8c0afe20 con 0x7f0b940df3d0 2024-05-09T06:36:13.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.520+0000 7f289affd640 1 -- 172.21.15.3:0/2160528106 <== osd.3 v2:172.21.15.3:6824/1903389857 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f288c006bb0 con 0x7f288c001650 2024-05-09T06:36:13.570 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.520+0000 7ff578fd3640 1 --2- 172.21.15.3:0/1686658755 >> [v2:172.21.15.3:6800/926805498,v1:172.21.15.3:6801/926805498] conn(0x7ff558074120 0x7ff558076540 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:13.570 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.520+0000 7ff5797d4640 1 --2- 172.21.15.3:0/1686658755 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7ff55806bf00 0x7ff55806e3c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ff570009ad0 tx=0x7ff570008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:13.570 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.520+0000 7f0ba1c59640 1 --2- 172.21.15.3:0/4243570696 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f0b8006bf00 0x7f0b8006e3c0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f0b980010e0 tx=0x7f0b98007950 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:13.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.520+0000 7f0b927fc640 1 --2- 172.21.15.3:0/4243570696 >> [v2:172.21.15.190:6812/1766923126,v1:172.21.15.190:6814/1766923126] conn(0x7f0b80074120 0x7f0b80076540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:13.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.521+0000 7f0b927fc640 1 -- 172.21.15.3:0/4243570696 --> [v2:172.21.15.190:6812/1766923126,v1:172.21.15.190:6814/1766923126] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0b8c040ea0 con 0x7f0b80074120 2024-05-09T06:36:13.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.521+0000 7f0b927fc640 1 -- 172.21.15.3:0/4243570696 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f0b8c0b0210 con 0x7f0b940df3d0 2024-05-09T06:36:13.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.520+0000 7f18f27fc640 1 --2- 172.21.15.3:0/1720337932 >> [v2:172.21.15.190:6808/537849891,v1:172.21.15.190:6809/537849891] conn(0x7f18e0074180 0x7f18e00765a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:13.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.520+0000 7f18f27fc640 1 -- 172.21.15.3:0/1720337932 --> [v2:172.21.15.190:6808/537849891,v1:172.21.15.190:6809/537849891] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f18f804be80 con 0x7f18e0074180 2024-05-09T06:36:13.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.520+0000 7f18f27fc640 1 -- 172.21.15.3:0/1720337932 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f18f80b02a0 con 0x7f18fc0cfdd0 2024-05-09T06:36:13.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.520+0000 7f190106f640 1 --2- 172.21.15.3:0/1720337932 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f18e006bf00 0x7f18e006e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:13.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.520+0000 7f1902071640 1 --2- 172.21.15.3:0/1720337932 >> [v2:172.21.15.190:6808/537849891,v1:172.21.15.190:6809/537849891] conn(0x7f18e0074180 0x7f18e00765a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:13.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.521+0000 7f0ba1458640 1 --2- 172.21.15.3:0/4243570696 >> [v2:172.21.15.190:6812/1766923126,v1:172.21.15.190:6814/1766923126] conn(0x7f0b80074120 0x7f0b80076540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:13.572 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.521+0000 7f190106f640 1 --2- 172.21.15.3:0/1720337932 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f18e006bf00 0x7f18e006e3c0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f18fc1fbb80 tx=0x7f18f4007c20 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:13.572 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.522+0000 7f1902071640 1 --2- 172.21.15.3:0/1720337932 >> [v2:172.21.15.190:6808/537849891,v1:172.21.15.190:6809/537849891] conn(0x7f18e0074180 0x7f18e00765a0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:13.572 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.522+0000 7f0ba1458640 1 --2- 172.21.15.3:0/4243570696 >> [v2:172.21.15.190:6812/1766923126,v1:172.21.15.190:6814/1766923126] conn(0x7f0b80074120 0x7f0b80076540 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:13.572 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.522+0000 7ff569ffb640 1 -- 172.21.15.3:0/1686658755 <== osd.1 v2:172.21.15.3:6800/926805498 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff558076bf0 con 0x7ff558074120 2024-05-09T06:36:13.573 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.522+0000 7f18f27fc640 1 -- 172.21.15.3:0/1720337932 <== osd.7 v2:172.21.15.190:6808/537849891 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f18f804be80 con 0x7f18e0074180 2024-05-09T06:36:13.573 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.524+0000 7f0b927fc640 1 -- 172.21.15.3:0/4243570696 <== osd.5 v2:172.21.15.190:6812/1766923126 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0b8c040ea0 con 0x7f0b80074120 2024-05-09T06:36:13.573 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.549+0000 7f6358b73640 1 -- 172.21.15.3:0/1857497576 --> [v2:172.21.15.3:6808/801295451,v1:172.21.15.3:6809/801295451] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f634c105370 con 0x7f6338074100 2024-05-09T06:36:13.573 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.550+0000 7f633ffff640 1 -- 172.21.15.3:0/1857497576 <== osd.2 v2:172.21.15.3:6808/801295451 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f634c105370 con 0x7f6338074100 2024-05-09T06:36:13.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.550+0000 7f633dffb640 1 -- 172.21.15.3:0/1857497576 >> [v2:172.21.15.3:6808/801295451,v1:172.21.15.3:6809/801295451] conn(0x7f6338074100 msgr2=0x7f6338076520 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:13.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.550+0000 7f633dffb640 1 --2- 172.21.15.3:0/1857497576 >> [v2:172.21.15.3:6808/801295451,v1:172.21.15.3:6809/801295451] conn(0x7f6338074100 0x7f6338076520 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:13.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.551+0000 7f633dffb640 1 -- 172.21.15.3:0/1857497576 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f633806bf00 msgr2=0x7f633806e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:13.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.551+0000 7f633dffb640 1 --2- 172.21.15.3:0/1857497576 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f633806bf00 0x7f633806e3c0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f6348000f60 tx=0x7f6348007880 comp rx=0 tx=0).stop 2024-05-09T06:36:13.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.551+0000 7f633dffb640 1 -- 172.21.15.3:0/1857497576 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f634c054520 msgr2=0x7f634c0f7030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:13.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.551+0000 7f633dffb640 1 --2- 172.21.15.3:0/1857497576 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f634c054520 0x7f634c0f7030 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f634000d650 tx=0x7f634000db10 comp rx=0 tx=0).stop 2024-05-09T06:36:13.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.551+0000 7f6353fff640 1 -- 172.21.15.3:0/1857497576 reap_dead start 2024-05-09T06:36:13.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.551+0000 7f633dffb640 1 -- 172.21.15.3:0/1857497576 shutdown_connections 2024-05-09T06:36:13.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.551+0000 7f633dffb640 1 -- 172.21.15.3:0/1857497576 >> 172.21.15.3:0/1857497576 conn(0x7f634c009fb0 msgr2=0x7f634c101490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:13.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.551+0000 7f83d28f8640 1 -- 172.21.15.3:0/2359566183 --> [v2:172.21.15.190:6800/2143624576,v1:172.21.15.190:6801/2143624576] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f83b4002dd0 con 0x7f83b0074050 2024-05-09T06:36:13.576 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.552+0000 7f633dffb640 1 -- 172.21.15.3:0/1857497576 shutdown_connections 2024-05-09T06:36:13.576 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.552+0000 7f633dffb640 1 -- 172.21.15.3:0/1857497576 wait complete. 2024-05-09T06:36:13.576 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.552+0000 7f83c27fc640 1 -- 172.21.15.3:0/2359566183 <== osd.4 v2:172.21.15.190:6800/2143624576 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f83b4002dd0 con 0x7f83b0074050 2024-05-09T06:36:13.578 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.552+0000 7f83d28f8640 1 -- 172.21.15.3:0/2359566183 >> [v2:172.21.15.190:6800/2143624576,v1:172.21.15.190:6801/2143624576] conn(0x7f83b0074050 msgr2=0x7f83b0076470 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:13.578 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.553+0000 7f83d28f8640 1 --2- 172.21.15.3:0/2359566183 >> [v2:172.21.15.190:6800/2143624576,v1:172.21.15.190:6801/2143624576] conn(0x7f83b0074050 0x7f83b0076470 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:13.578 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.553+0000 7f83d28f8640 1 -- 172.21.15.3:0/2359566183 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f83b006be30 msgr2=0x7f83b006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:13.578 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.553+0000 7f83d28f8640 1 --2- 172.21.15.3:0/2359566183 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f83b006be30 0x7f83b006e2f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f83c40e2920 tx=0x7f83bc008040 comp rx=0 tx=0).stop 2024-05-09T06:36:13.578 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.553+0000 7f83d28f8640 1 -- 172.21.15.3:0/2359566183 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f83c40501a0 msgr2=0x7f83c40c6b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:13.578 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.553+0000 7f83d28f8640 1 --2- 172.21.15.3:0/2359566183 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f83c40501a0 0x7f83c40c6b60 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f83c800b420 tx=0x7f83c800b8e0 comp rx=0 tx=0).stop 2024-05-09T06:36:13.578 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.553+0000 7f83d20f7640 1 -- 172.21.15.3:0/2359566183 reap_dead start 2024-05-09T06:36:13.578 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.553+0000 7f83d28f8640 1 -- 172.21.15.3:0/2359566183 shutdown_connections 2024-05-09T06:36:13.578 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.553+0000 7f83d28f8640 1 -- 172.21.15.3:0/2359566183 >> 172.21.15.3:0/2359566183 conn(0x7f83c4009fb0 msgr2=0x7f83c4034480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:13.578 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.554+0000 7f83d28f8640 1 -- 172.21.15.3:0/2359566183 shutdown_connections 2024-05-09T06:36:13.578 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.554+0000 7f83d28f8640 1 -- 172.21.15.3:0/2359566183 wait complete. 2024-05-09T06:36:13.579 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.554+0000 7f40053d6640 1 -- 172.21.15.3:0/3464323938 --> [v2:172.21.15.3:6816/1897849827,v1:172.21.15.3:6817/1897849827] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f3ff80e9a40 con 0x7f3fe4074050 2024-05-09T06:36:13.580 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.554+0000 7f28ac570640 1 -- 172.21.15.3:0/2160528106 --> [v2:172.21.15.3:6824/1903389857,v1:172.21.15.3:6825/1903389857] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f288c005c90 con 0x7f288c001650 2024-05-09T06:36:13.580 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.554+0000 7f3ffcff9640 1 -- 172.21.15.3:0/3464323938 <== osd.0 v2:172.21.15.3:6816/1897849827 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f3ff80e9a40 con 0x7f3fe4074050 2024-05-09T06:36:13.581 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.555+0000 7f289affd640 1 -- 172.21.15.3:0/2160528106 <== osd.3 v2:172.21.15.3:6824/1903389857 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f288c005c90 con 0x7f288c001650 2024-05-09T06:36:13.581 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.555+0000 7fe2fc8e7640 1 -- 172.21.15.3:0/475537639 --> [v2:172.21.15.190:6824/342414248,v1:172.21.15.190:6825/342414248] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fe2f806c850 con 0x7fe2d8074d60 2024-05-09T06:36:13.581 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.555+0000 7fe2e6ffd640 1 -- 172.21.15.3:0/475537639 <== osd.6 v2:172.21.15.190:6824/342414248 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fe2f806c850 con 0x7fe2d8074d60 2024-05-09T06:36:13.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.555+0000 7f1903ad4640 1 -- 172.21.15.3:0/1720337932 --> [v2:172.21.15.190:6808/537849891,v1:172.21.15.190:6809/537849891] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f18fc0d50a0 con 0x7f18e0074180 2024-05-09T06:36:13.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.555+0000 7f28ac570640 1 -- 172.21.15.3:0/2160528106 >> [v2:172.21.15.3:6824/1903389857,v1:172.21.15.3:6825/1903389857] conn(0x7f288c001650 msgr2=0x7f288c003b10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:13.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.555+0000 7f28ac570640 1 --2- 172.21.15.3:0/2160528106 >> [v2:172.21.15.3:6824/1903389857,v1:172.21.15.3:6825/1903389857] conn(0x7f288c001650 0x7f288c003b10 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:13.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.555+0000 7f3fea7fc640 1 -- 172.21.15.3:0/3464323938 >> [v2:172.21.15.3:6816/1897849827,v1:172.21.15.3:6817/1897849827] conn(0x7f3fe4074050 msgr2=0x7f3fe4076470 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:13.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.555+0000 7f3fea7fc640 1 --2- 172.21.15.3:0/3464323938 >> [v2:172.21.15.3:6816/1897849827,v1:172.21.15.3:6817/1897849827] conn(0x7f3fe4074050 0x7f3fe4076470 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:13.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.555+0000 7f28ac570640 1 -- 172.21.15.3:0/2160528106 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f288806be30 msgr2=0x7f288806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:13.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.555+0000 7f28ac570640 1 --2- 172.21.15.3:0/2160528106 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f288806be30 0x7f288806e2f0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f28a0037000 tx=0x7f28a000b040 comp rx=0 tx=0).stop 2024-05-09T06:36:13.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.555+0000 7f0ba3ebd640 1 -- 172.21.15.3:0/4243570696 --> [v2:172.21.15.190:6812/1766923126,v1:172.21.15.190:6814/1766923126] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f0b94054b10 con 0x7f0b80074120 2024-05-09T06:36:13.584 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.555+0000 7fe2fc8e7640 1 -- 172.21.15.3:0/475537639 >> [v2:172.21.15.190:6824/342414248,v1:172.21.15.190:6825/342414248] conn(0x7fe2d8074d60 msgr2=0x7fe2d8077200 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:13.584 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.556+0000 7fe2fc8e7640 1 --2- 172.21.15.3:0/475537639 >> [v2:172.21.15.190:6824/342414248,v1:172.21.15.190:6825/342414248] conn(0x7fe2d8074d60 0x7fe2d8077200 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:13.584 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.555+0000 7f18f27fc640 1 -- 172.21.15.3:0/1720337932 <== osd.7 v2:172.21.15.190:6808/537849891 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f18fc0d50a0 con 0x7f18e0074180 2024-05-09T06:36:13.584 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.556+0000 7f28ac570640 1 -- 172.21.15.3:0/2160528106 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f28a4072f10 msgr2=0x7f28a4146730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:13.584 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.556+0000 7f28ac570640 1 --2- 172.21.15.3:0/2160528106 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f28a4072f10 0x7f28a4146730 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f289c00c7b0 tx=0x7f289c00cc70 comp rx=0 tx=0).stop 2024-05-09T06:36:13.584 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.556+0000 7f28aab0d640 1 -- 172.21.15.3:0/2160528106 reap_dead start 2024-05-09T06:36:13.585 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.556+0000 7f3fea7fc640 1 -- 172.21.15.3:0/3464323938 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f3fe406be30 msgr2=0x7f3fe406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:13.585 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.556+0000 7f3fea7fc640 1 --2- 172.21.15.3:0/3464323938 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f3fe406be30 0x7f3fe406e2f0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f3ff4009560 tx=0x7f3ff402e000 comp rx=0 tx=0).stop 2024-05-09T06:36:13.585 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.556+0000 7f3fea7fc640 1 -- 172.21.15.3:0/3464323938 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3ff804ffc0 msgr2=0x7f3ff80da7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:13.585 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.556+0000 7f3fea7fc640 1 --2- 172.21.15.3:0/3464323938 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3ff804ffc0 0x7f3ff80da7f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f4000000b60 tx=0x7f4000068dc0 comp rx=0 tx=0).stop 2024-05-09T06:36:13.585 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.556+0000 7f4004bd5640 1 -- 172.21.15.3:0/3464323938 reap_dead start 2024-05-09T06:36:13.585 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.556+0000 7f3fea7fc640 1 -- 172.21.15.3:0/3464323938 shutdown_connections 2024-05-09T06:36:13.585 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.556+0000 7f3fea7fc640 1 -- 172.21.15.3:0/3464323938 >> 172.21.15.3:0/3464323938 conn(0x7f3ff8009fa0 msgr2=0x7f3ff8052c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:13.585 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.556+0000 7ff5577fe640 1 -- 172.21.15.3:0/1686658755 --> [v2:172.21.15.3:6800/926805498,v1:172.21.15.3:6801/926805498] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ff5740682e0 con 0x7ff558074120 2024-05-09T06:36:13.585 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.556+0000 7f28ac570640 1 -- 172.21.15.3:0/2160528106 shutdown_connections 2024-05-09T06:36:13.586 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.556+0000 7f28ac570640 1 -- 172.21.15.3:0/2160528106 >> 172.21.15.3:0/2160528106 conn(0x7f28a406b250 msgr2=0x7f28a40b18f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:13.586 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.556+0000 7f0b927fc640 1 -- 172.21.15.3:0/4243570696 <== osd.5 v2:172.21.15.190:6812/1766923126 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f0b94054b10 con 0x7f0b80074120 2024-05-09T06:36:13.586 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.556+0000 7fe2fc8e7640 1 -- 172.21.15.3:0/475537639 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fe2d806c030 msgr2=0x7fe2d806e4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:13.586 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.556+0000 7fe2fc8e7640 1 --2- 172.21.15.3:0/475537639 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fe2d806c030 0x7fe2d806e4f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fe2f81fc6a0 tx=0x7fe2e8006b50 comp rx=0 tx=0).stop 2024-05-09T06:36:13.586 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.556+0000 7fe2fc8e7640 1 -- 172.21.15.3:0/475537639 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe2f80cf600 msgr2=0x7fe2f80ca4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:13.586 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.556+0000 7fe2fc8e7640 1 --2- 172.21.15.3:0/475537639 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe2f80cf600 0x7fe2f80ca4c0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fe2f000ccd0 tx=0x7fe2f000a5c0 comp rx=0 tx=0).stop 2024-05-09T06:36:13.587 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.556+0000 7fe2f6d9d640 1 -- 172.21.15.3:0/475537639 reap_dead start 2024-05-09T06:36:13.587 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.556+0000 7fe2fc8e7640 1 -- 172.21.15.3:0/475537639 shutdown_connections 2024-05-09T06:36:13.587 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.556+0000 7fe2fc8e7640 1 -- 172.21.15.3:0/475537639 >> 172.21.15.3:0/475537639 conn(0x7fe2f80d04a0 msgr2=0x7fe2f806bd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:13.587 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.556+0000 7f28ac570640 1 -- 172.21.15.3:0/2160528106 shutdown_connections 2024-05-09T06:36:13.587 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.556+0000 7f3fea7fc640 1 -- 172.21.15.3:0/3464323938 shutdown_connections 2024-05-09T06:36:13.587 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.556+0000 7f3fea7fc640 1 -- 172.21.15.3:0/3464323938 wait complete. 2024-05-09T06:36:13.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.556+0000 7f28ac570640 1 -- 172.21.15.3:0/2160528106 wait complete. 2024-05-09T06:36:13.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.556+0000 7ff569ffb640 1 -- 172.21.15.3:0/1686658755 <== osd.1 v2:172.21.15.3:6800/926805498 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7ff5740682e0 con 0x7ff558074120 2024-05-09T06:36:13.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.557+0000 7fe2fc8e7640 1 -- 172.21.15.3:0/475537639 shutdown_connections 2024-05-09T06:36:13.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.557+0000 7fe2fc8e7640 1 -- 172.21.15.3:0/475537639 wait complete. 2024-05-09T06:36:13.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.557+0000 7f1903ad4640 1 -- 172.21.15.3:0/1720337932 >> [v2:172.21.15.190:6808/537849891,v1:172.21.15.190:6809/537849891] conn(0x7f18e0074180 msgr2=0x7f18e00765a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:13.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.557+0000 7f1903ad4640 1 --2- 172.21.15.3:0/1720337932 >> [v2:172.21.15.190:6808/537849891,v1:172.21.15.190:6809/537849891] conn(0x7f18e0074180 0x7f18e00765a0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:13.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.557+0000 7f0ba3ebd640 1 -- 172.21.15.3:0/4243570696 >> [v2:172.21.15.190:6812/1766923126,v1:172.21.15.190:6814/1766923126] conn(0x7f0b80074120 msgr2=0x7f0b80076540 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:13.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.557+0000 7f0ba3ebd640 1 --2- 172.21.15.3:0/4243570696 >> [v2:172.21.15.190:6812/1766923126,v1:172.21.15.190:6814/1766923126] conn(0x7f0b80074120 0x7f0b80076540 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:13.590 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.557+0000 7f1903ad4640 1 -- 172.21.15.3:0/1720337932 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f18e006bf00 msgr2=0x7f18e006e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:13.590 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.557+0000 7f1903ad4640 1 --2- 172.21.15.3:0/1720337932 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f18e006bf00 0x7f18e006e3c0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f18fc1fbb80 tx=0x7f18f4007c20 comp rx=0 tx=0).stop 2024-05-09T06:36:13.590 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.557+0000 7ff5577fe640 1 -- 172.21.15.3:0/1686658755 >> [v2:172.21.15.3:6800/926805498,v1:172.21.15.3:6801/926805498] conn(0x7ff558074120 msgr2=0x7ff558076540 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:13.590 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.557+0000 7ff5577fe640 1 --2- 172.21.15.3:0/1686658755 >> [v2:172.21.15.3:6800/926805498,v1:172.21.15.3:6801/926805498] conn(0x7ff558074120 0x7ff558076540 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:13.590 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.557+0000 7f0ba3ebd640 1 -- 172.21.15.3:0/4243570696 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f0b8006bf00 msgr2=0x7f0b8006e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:13.591 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.557+0000 7f1903ad4640 1 -- 172.21.15.3:0/1720337932 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f18fc0cfdd0 msgr2=0x7f18fc15cea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:13.591 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.557+0000 7f1903ad4640 1 --2- 172.21.15.3:0/1720337932 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f18fc0cfdd0 0x7f18fc15cea0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f18f8009870 tx=0x7f18f80046c0 comp rx=0 tx=0).stop 2024-05-09T06:36:13.591 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.557+0000 7f1902071640 1 -- 172.21.15.3:0/1720337932 reap_dead start 2024-05-09T06:36:13.591 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.557+0000 7f1903ad4640 1 -- 172.21.15.3:0/1720337932 shutdown_connections 2024-05-09T06:36:13.591 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.557+0000 7f1903ad4640 1 -- 172.21.15.3:0/1720337932 >> 172.21.15.3:0/1720337932 conn(0x7f18fc08dbc0 msgr2=0x7f18fc0d4390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:13.591 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.557+0000 7f0ba3ebd640 1 --2- 172.21.15.3:0/4243570696 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f0b8006bf00 0x7f0b8006e3c0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f0b980010e0 tx=0x7f0b98007950 comp rx=0 tx=0).stop 2024-05-09T06:36:13.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.558+0000 7ff5577fe640 1 -- 172.21.15.3:0/1686658755 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7ff55806bf00 msgr2=0x7ff55806e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:13.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.558+0000 7ff5577fe640 1 --2- 172.21.15.3:0/1686658755 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7ff55806bf00 0x7ff55806e3c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ff570009ad0 tx=0x7ff570008040 comp rx=0 tx=0).stop 2024-05-09T06:36:13.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.558+0000 7ff5577fe640 1 -- 172.21.15.3:0/1686658755 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff57415adf0 msgr2=0x7ff5740cec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:13.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.558+0000 7ff5577fe640 1 --2- 172.21.15.3:0/1686658755 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff57415adf0 0x7ff5740cec10 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7ff56c00c780 tx=0x7ff56c00cc40 comp rx=0 tx=0).stop 2024-05-09T06:36:13.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.557+0000 7f0ba3ebd640 1 -- 172.21.15.3:0/4243570696 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0b940df3d0 msgr2=0x7f0b940d8c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:13.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.557+0000 7f0ba3ebd640 1 --2- 172.21.15.3:0/4243570696 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0b940df3d0 0x7f0b940d8c80 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f0b8c009870 tx=0x7f0b8c0046c0 comp rx=0 tx=0).stop 2024-05-09T06:36:13.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.557+0000 7f0ba245a640 1 -- 172.21.15.3:0/4243570696 reap_dead start 2024-05-09T06:36:13.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.558+0000 7f0ba3ebd640 1 -- 172.21.15.3:0/4243570696 shutdown_connections 2024-05-09T06:36:13.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.558+0000 7f0ba3ebd640 1 -- 172.21.15.3:0/4243570696 >> 172.21.15.3:0/4243570696 conn(0x7f0b9400a3f0 msgr2=0x7f0b940e0de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:13.593 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.558+0000 7f1903ad4640 1 -- 172.21.15.3:0/1720337932 shutdown_connections 2024-05-09T06:36:13.593 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.558+0000 7f1903ad4640 1 -- 172.21.15.3:0/1720337932 wait complete. 2024-05-09T06:36:13.593 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.558+0000 7ff579fd5640 1 -- 172.21.15.3:0/1686658755 reap_dead start 2024-05-09T06:36:13.593 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.558+0000 7ff5577fe640 1 -- 172.21.15.3:0/1686658755 shutdown_connections 2024-05-09T06:36:13.593 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.558+0000 7ff5577fe640 1 -- 172.21.15.3:0/1686658755 >> 172.21.15.3:0/1686658755 conn(0x7ff5740d0a70 msgr2=0x7ff57415b210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:13.594 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.558+0000 7f0ba3ebd640 1 -- 172.21.15.3:0/4243570696 shutdown_connections 2024-05-09T06:36:13.594 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.559+0000 7f0ba3ebd640 1 -- 172.21.15.3:0/4243570696 wait complete. 2024-05-09T06:36:13.594 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.559+0000 7ff5577fe640 1 -- 172.21.15.3:0/1686658755 shutdown_connections 2024-05-09T06:36:13.594 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.559+0000 7ff5577fe640 1 -- 172.21.15.3:0/1686658755 wait complete. 2024-05-09T06:36:13.595 INFO:teuthology.orchestra.run.smithi003.stdout:55834574854 2024-05-09T06:36:13.595 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-05-09T06:36:13.596 INFO:teuthology.orchestra.run.smithi003.stdout:55834574854 2024-05-09T06:36:13.596 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-05-09T06:36:13.597 INFO:teuthology.orchestra.run.smithi003.stdout:55834574855 2024-05-09T06:36:13.597 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-05-09T06:36:13.598 INFO:teuthology.orchestra.run.smithi003.stdout:55834574855 2024-05-09T06:36:13.598 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-05-09T06:36:13.599 INFO:teuthology.orchestra.run.smithi003.stdout:55834574855 2024-05-09T06:36:13.599 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-05-09T06:36:13.600 INFO:teuthology.orchestra.run.smithi003.stdout:55834574855 2024-05-09T06:36:13.600 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-05-09T06:36:13.601 INFO:teuthology.orchestra.run.smithi003.stdout:55834574854 2024-05-09T06:36:13.601 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-05-09T06:36:13.602 INFO:teuthology.orchestra.run.smithi003.stdout:55834574854 2024-05-09T06:36:13.602 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-05-09T06:36:13.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.835+0000 7fc0b51a2640 1 Processor -- start 2024-05-09T06:36:13.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.836+0000 7fc0b51a2640 1 -- start start 2024-05-09T06:36:13.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.834+0000 7f926cea7640 1 Processor -- start 2024-05-09T06:36:13.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.834+0000 7f926cea7640 1 -- start start 2024-05-09T06:36:13.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.835+0000 7f926cea7640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f92680cc2c0 0x7f9268166b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:13.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.835+0000 7f926cea7640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f92680cc6a0 0x7f92680b3f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:13.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.835+0000 7f926cea7640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9268167070 con 0x7f9268162ae0 2024-05-09T06:36:13.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.835+0000 7f926cea7640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f926806d2e0 con 0x7f92680cc2c0 2024-05-09T06:36:13.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.835+0000 7f926cea7640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f92680727c0 con 0x7f92680cc6a0 2024-05-09T06:36:13.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.836+0000 7f92677fe640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f92680cc2c0 0x7f9268166b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:13.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.836+0000 7f92677fe640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f92680cc2c0 0x7f9268166b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49540/0 (socket says 172.21.15.3:49540) 2024-05-09T06:36:13.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.836+0000 7f92677fe640 1 -- 172.21.15.3:0/2915407568 learned_addr learned my addr 172.21.15.3:0/2915407568 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:36:13.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.836+0000 7f9266ffd640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f92680cc6a0 0x7f92680b3f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:13.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.836+0000 7f7b523fb640 1 Processor -- start 2024-05-09T06:36:13.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.836+0000 7f7b523fb640 1 -- start start 2024-05-09T06:36:13.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.836+0000 7f9266ffd640 1 -- 172.21.15.3:0/2915407568 >> v2:172.21.15.3:3300/0 conn(0x7f92680cc2c0 msgr2=0x7f9268166b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:13.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.836+0000 7f9266ffd640 1 --2- 172.21.15.3:0/2915407568 >> v2:172.21.15.3:3300/0 conn(0x7f92680cc2c0 0x7f9268166b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:13.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.836+0000 7f9266ffd640 1 -- 172.21.15.3:0/2915407568 >> v1:172.21.15.190:6789/0 conn(0x7f9268162ae0 legacy=0x7f926814b5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:13.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.836+0000 7f9266ffd640 1 -- 172.21.15.3:0/2915407568 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9268163060 con 0x7f92680cc6a0 2024-05-09T06:36:13.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.836+0000 7f92677fe640 1 --2- 172.21.15.3:0/2915407568 >> v2:172.21.15.3:3300/0 conn(0x7f92680cc2c0 0x7f9268166b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T06:36:13.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.836+0000 7f24c331d640 1 Processor -- start 2024-05-09T06:36:13.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.837+0000 7f24c331d640 1 -- start start 2024-05-09T06:36:13.821 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.837+0000 7fc0b51a2640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc0b0144910 0x7fc0b00cf8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:13.821 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.837+0000 7fc0b51a2640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fc0b0072970 0x7fc0b0071080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:13.821 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.837+0000 7f7b523fb640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f7b4c0706b0 0x7f7b4c15d600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:13.821 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.837+0000 7f7b523fb640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7b4c15c700 con 0x7f7b4c06b410 2024-05-09T06:36:13.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.837+0000 7f7b523fb640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7b4c15c9e0 con 0x7f7b4c06e3f0 2024-05-09T06:36:13.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.837+0000 7f7b523fb640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f7b4c15ccc0 con 0x7f7b4c0706b0 2024-05-09T06:36:13.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.837+0000 7fc0b51a2640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc0b0144cf0 con 0x7fc0b00b4130 2024-05-09T06:36:13.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.837+0000 7fc0aed9d640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc0b0144910 0x7fc0b00cf8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:13.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.838+0000 7fc0af59e640 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7fc0b00b4130 0x7fc0b006e250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.3:58434/0 (socket says 172.21.15.3:58434) 2024-05-09T06:36:13.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.838+0000 7fc0af59e640 1 -- 172.21.15.3:0/1227817894 learned_addr learned my addr 172.21.15.3:0/1227817894 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-05-09T06:36:13.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.838+0000 7fc0b51a2640 1 -- 172.21.15.3:0/1227817894 --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7fc0b00724e0 con 0x7fc0b0072970 2024-05-09T06:36:13.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.838+0000 7fc0b51a2640 1 -- 172.21.15.3:0/1227817894 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fc0b00718d0 con 0x7fc0b0144910 2024-05-09T06:36:13.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.838+0000 7f7b4bfff640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f7b4c0706b0 0x7f7b4c15d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:13.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.838+0000 7f7b4bfff640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f7b4c0706b0 0x7f7b4c15d600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.3:47738/0 (socket says 172.21.15.3:47738) 2024-05-09T06:36:13.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.838+0000 7f7b4bfff640 1 -- 172.21.15.3:0/1717621930 learned_addr learned my addr 172.21.15.3:0/1717621930 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:36:13.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.838+0000 7fc0ae59c640 1 --2- 172.21.15.3:0/1227817894 >> v2:172.21.15.190:3300/0 conn(0x7fc0b0072970 0x7fc0b0071080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:13.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.838+0000 7fc0aed9d640 1 -- 172.21.15.3:0/1227817894 >> v2:172.21.15.190:3300/0 conn(0x7fc0b0072970 msgr2=0x7fc0b0071080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:13.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.838+0000 7f7b4bfff640 1 -- 172.21.15.3:0/1717621930 >> v1:172.21.15.190:6789/0 conn(0x7f7b4c06e3f0 legacy=0x7f7b4c15fbe0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T06:36:13.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.838+0000 7f7b4bfff640 1 -- 172.21.15.3:0/1717621930 >> v1:172.21.15.3:6789/0 conn(0x7f7b4c06b410 legacy=0x7f7b4c06dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:13.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.838+0000 7f7b4bfff640 1 -- 172.21.15.3:0/1717621930 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b4c15cfa0 con 0x7f7b4c0706b0 2024-05-09T06:36:13.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.838+0000 7f24c331d640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f24bc160000 0x7f24bc070d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:13.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.838+0000 7f24c331d640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f24bc1603e0 0x7f24bc160870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:13.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.838+0000 7f24c331d640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f24bc0b4a10 con 0x7f24bc070360 2024-05-09T06:36:13.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.838+0000 7f24c08b8640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f24bc070360 0x7f24bc0d0660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:49238/0 (socket says 172.21.15.3:49238) 2024-05-09T06:36:13.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.838+0000 7f24c331d640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f24bc160f10 con 0x7f24bc1603e0 2024-05-09T06:36:13.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.838+0000 7f24c08b8640 1 -- 172.21.15.3:0/3492620125 learned_addr learned my addr 172.21.15.3:0/3492620125 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-05-09T06:36:13.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.838+0000 7f24c331d640 1 -- 172.21.15.3:0/3492620125 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f24bc06fe70 con 0x7f24bc160000 2024-05-09T06:36:13.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.838+0000 7f24c10b9640 1 --2- 172.21.15.3:0/3492620125 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f24bc160000 0x7f24bc070d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:13.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.838+0000 7f24c18ba640 1 --2- 172.21.15.3:0/3492620125 >> v2:172.21.15.190:3300/0 conn(0x7f24bc1603e0 0x7f24bc160870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:13.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.838+0000 7f9266ffd640 1 --2- 172.21.15.3:0/2915407568 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f92680cc6a0 0x7f92680b3f50 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f92680b2670 tx=0x7f925c02a160 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3d88350b7c84fa3d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:13.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.839+0000 7febd7a7d640 1 Processor -- start 2024-05-09T06:36:13.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.839+0000 7f92667fc640 1 -- 172.21.15.3:0/2915407568 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f925c037070 con 0x7f92680cc6a0 2024-05-09T06:36:13.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.839+0000 7f92667fc640 1 -- 172.21.15.3:0/2915407568 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f925c032440 con 0x7f92680cc6a0 2024-05-09T06:36:13.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.839+0000 7f24c10b9640 1 -- 172.21.15.3:0/3492620125 >> v2:172.21.15.190:3300/0 conn(0x7f24bc1603e0 msgr2=0x7f24bc160870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:13.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.839+0000 7f24c10b9640 1 --2- 172.21.15.3:0/3492620125 >> v2:172.21.15.190:3300/0 conn(0x7f24bc1603e0 0x7f24bc160870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:13.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.839+0000 7f24c10b9640 1 -- 172.21.15.3:0/3492620125 >> v1:172.21.15.3:6789/0 conn(0x7f24bc070360 legacy=0x7f24bc0d0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:13.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.839+0000 7f24c10b9640 1 -- 172.21.15.3:0/3492620125 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24bc15fd60 con 0x7f24bc160000 2024-05-09T06:36:13.828 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.838+0000 7fc0aed9d640 1 --2- 172.21.15.3:0/1227817894 >> v2:172.21.15.190:3300/0 conn(0x7fc0b0072970 0x7fc0b0071080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:13.828 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.838+0000 7fc0aed9d640 1 -- 172.21.15.3:0/1227817894 >> v1:172.21.15.190:6789/0 conn(0x7fc0b00b4130 legacy=0x7fc0b006e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:13.828 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.838+0000 7fc0aed9d640 1 -- 172.21.15.3:0/1227817894 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0b0071bb0 con 0x7fc0b0144910 2024-05-09T06:36:13.828 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.838+0000 7fc0ae59c640 1 --2- 172.21.15.3:0/1227817894 >> v2:172.21.15.190:3300/0 conn(0x7fc0b0072970 0x7fc0b0071080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T06:36:13.828 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.839+0000 7f7b4bfff640 1 --2- 172.21.15.3:0/1717621930 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7b4c0706b0 0x7f7b4c15d600 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f7b44009e50 tx=0x7f7b4402a6f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5cab233f5c831482 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:13.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.839+0000 7f597d968640 1 Processor -- start 2024-05-09T06:36:13.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.839+0000 7f597d968640 1 -- start start 2024-05-09T06:36:13.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.839+0000 7f910a87b640 1 Processor -- start 2024-05-09T06:36:13.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.839+0000 7fc0aed9d640 1 --2- 172.21.15.3:0/1227817894 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc0b0144910 0x7fc0b00cf8c0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fc0a00097a0 tx=0x7fc0a002a130 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3a078e79a4185c17 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:13.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.840+0000 7f7b4affd640 1 -- 172.21.15.3:0/1717621930 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b44036070 con 0x7f7b4c0706b0 2024-05-09T06:36:13.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.840+0000 7f7b4affd640 1 -- 172.21.15.3:0/1717621930 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b44031600 con 0x7f7b4c0706b0 2024-05-09T06:36:13.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.840+0000 7f7b4affd640 1 -- 172.21.15.3:0/1717621930 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b440319d0 con 0x7f7b4c0706b0 2024-05-09T06:36:13.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.839+0000 7febd7a7d640 1 -- start start 2024-05-09T06:36:13.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.840+0000 7f92667fc640 1 -- 172.21.15.3:0/2915407568 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f925c032810 con 0x7f92680cc6a0 2024-05-09T06:36:13.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.840+0000 7f24c10b9640 1 --2- 172.21.15.3:0/3492620125 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f24bc160000 0x7f24bc070d90 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f24b80097a0 tx=0x7f24b802e470 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c51a2ce5de8dc581 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:13.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.840+0000 7f24b3fff640 1 -- 172.21.15.3:0/3492620125 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24b803a070 con 0x7f24bc160000 2024-05-09T06:36:13.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.840+0000 7f24b3fff640 1 -- 172.21.15.3:0/3492620125 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f24b8033710 con 0x7f24bc160000 2024-05-09T06:36:13.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.840+0000 7f24b3fff640 1 -- 172.21.15.3:0/3492620125 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24b8033ae0 con 0x7f24bc160000 2024-05-09T06:36:13.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.840+0000 7f926cea7640 1 -- 172.21.15.3:0/2915407568 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f92680cc6a0 msgr2=0x7f92680b3f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:13.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.840+0000 7f926cea7640 1 --2- 172.21.15.3:0/2915407568 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f92680cc6a0 0x7f92680b3f50 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f92680b2670 tx=0x7f925c02a160 comp rx=0 tx=0).stop 2024-05-09T06:36:13.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.839+0000 7f910a87b640 1 -- start start 2024-05-09T06:36:13.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.840+0000 7fc0add9b640 1 -- 172.21.15.3:0/1227817894 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0a0037070 con 0x7fc0b0144910 2024-05-09T06:36:13.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.840+0000 7f926cea7640 1 -- 172.21.15.3:0/2915407568 shutdown_connections 2024-05-09T06:36:13.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.840+0000 7f926cea7640 1 --2- 172.21.15.3:0/2915407568 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f92680cc6a0 0x7f92680b3f50 secure :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f92680b2670 tx=0x7f925c02a160 comp rx=0 tx=0).stop 2024-05-09T06:36:13.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.840+0000 7f926cea7640 1 --2- 172.21.15.3:0/2915407568 >> v2:172.21.15.3:3300/0 conn(0x7f92680cc2c0 0x7f9268166b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:13.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.840+0000 7f926cea7640 1 -- 172.21.15.3:0/2915407568 >> 172.21.15.3:0/2915407568 conn(0x7f926806bb30 msgr2=0x7f92680b1b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:13.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.840+0000 7f910a87b640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f910415e6f0 0x7f91040b4170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:13.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.841+0000 7fc0add9b640 1 -- 172.21.15.3:0/1227817894 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc0a002ae40 con 0x7fc0b0144910 2024-05-09T06:36:13.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.841+0000 7fc0add9b640 1 -- 172.21.15.3:0/1227817894 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0a0032660 con 0x7fc0b0144910 2024-05-09T06:36:13.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.841+0000 7f7b523fb640 1 -- 172.21.15.3:0/1717621930 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7b4c0706b0 msgr2=0x7f7b4c15d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:13.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.840+0000 7f597d968640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f597815ea40 0x7f59780cdbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:13.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.841+0000 7f597d968640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5978072a80 con 0x7f597815d8a0 2024-05-09T06:36:13.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.841+0000 7f597d968640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5978072d60 con 0x7f597806e250 2024-05-09T06:36:13.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.841+0000 7f597d968640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f59780ce110 con 0x7f597815ea40 2024-05-09T06:36:13.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.841+0000 7f59777fe640 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7f597806e250 0x7f59780b35e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.3:58454/0 (socket says 172.21.15.3:58454) 2024-05-09T06:36:13.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.841+0000 7f59777fe640 1 -- 172.21.15.3:0/3617954560 learned_addr learned my addr 172.21.15.3:0/3617954560 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-05-09T06:36:13.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.841+0000 7f9103fff640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f910415e6f0 0x7f91040b4170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:13.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.841+0000 7f9103fff640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f910415e6f0 0x7f91040b4170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.3:47764/0 (socket says 172.21.15.3:47764) 2024-05-09T06:36:13.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.841+0000 7f7b523fb640 1 --2- 172.21.15.3:0/1717621930 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7b4c0706b0 0x7f7b4c15d600 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f7b44009e50 tx=0x7f7b4402a6f0 comp rx=0 tx=0).stop 2024-05-09T06:36:13.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.841+0000 7f91037fe640 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7f91040d2000 0x7f91040d26e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.3:58470/0 (socket says 172.21.15.3:58470) 2024-05-09T06:36:13.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.841+0000 7fe76759e640 1 Processor -- start 2024-05-09T06:36:13.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.841+0000 7f9103fff640 1 -- 172.21.15.3:0/19884217 learned_addr learned my addr 172.21.15.3:0/19884217 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:36:13.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.841+0000 7f7b523fb640 1 -- 172.21.15.3:0/1717621930 shutdown_connections 2024-05-09T06:36:13.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.841+0000 7f7b523fb640 1 --2- 172.21.15.3:0/1717621930 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7b4c0706b0 0x7f7b4c15d600 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:13.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.841+0000 7f7b523fb640 1 -- 172.21.15.3:0/1717621930 >> 172.21.15.3:0/1717621930 conn(0x7f7b4c06ce50 msgr2=0x7f7b4c0b2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:13.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.841+0000 7fe76759e640 1 -- start start 2024-05-09T06:36:13.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.841+0000 7f910a87b640 1 --2- 172.21.15.3:0/19884217 >> v2:172.21.15.3:3300/0 conn(0x7f91040d2df0 0x7f91040cf010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:13.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.841+0000 7f24c331d640 1 -- 172.21.15.3:0/3492620125 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f24bc160000 msgr2=0x7f24bc070d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:13.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.841+0000 7f24c331d640 1 --2- 172.21.15.3:0/3492620125 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f24bc160000 0x7f24bc070d90 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f24b80097a0 tx=0x7f24b802e470 comp rx=0 tx=0).stop 2024-05-09T06:36:13.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.841+0000 7f926cea7640 1 -- 172.21.15.3:0/2915407568 shutdown_connections 2024-05-09T06:36:13.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.841+0000 7f910a87b640 1 -- 172.21.15.3:0/19884217 --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f91040b49d0 con 0x7f91040d2000 2024-05-09T06:36:13.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.841+0000 7f910a87b640 1 -- 172.21.15.3:0/19884217 --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f91040b4cb0 con 0x7f91040d2df0 2024-05-09T06:36:13.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.841+0000 7f910a87b640 1 -- 172.21.15.3:0/19884217 --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f91040b4f90 con 0x7f910415e6f0 2024-05-09T06:36:13.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.841+0000 7f24c331d640 1 -- 172.21.15.3:0/3492620125 shutdown_connections 2024-05-09T06:36:13.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.841+0000 7f24c331d640 1 --2- 172.21.15.3:0/3492620125 >> v2:172.21.15.190:3300/0 conn(0x7f24bc1603e0 0x7f24bc160870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:13.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.841+0000 7f24c331d640 1 --2- 172.21.15.3:0/3492620125 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f24bc160000 0x7f24bc070d90 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:13.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.841+0000 7f24c331d640 1 -- 172.21.15.3:0/3492620125 >> 172.21.15.3:0/3492620125 conn(0x7f24bc0b3ae0 msgr2=0x7f24bc0b3ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:13.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.841+0000 7f5975ffb640 1 -- 172.21.15.3:0/3617954560 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2131556049 0 0) 0x7f5978072d60 con 0x7f597806e250 2024-05-09T06:36:13.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.841+0000 7f5975ffb640 1 -- 172.21.15.3:0/3617954560 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5960003520 con 0x7f597806e250 2024-05-09T06:36:13.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.841+0000 7f7b523fb640 1 -- 172.21.15.3:0/1717621930 shutdown_connections 2024-05-09T06:36:13.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.841+0000 7f9102ffd640 1 -- 172.21.15.3:0/19884217 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3333859284 0 0) 0x7f91040b49d0 con 0x7f91040d2000 2024-05-09T06:36:13.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.841+0000 7f9102ffd640 1 -- 172.21.15.3:0/19884217 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f90ec003520 con 0x7f91040d2000 2024-05-09T06:36:13.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.841+0000 7febd7a7d640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7febd00cc270 0x7febd0166220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:13.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.841+0000 7febd7a7d640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7febd00cc650 0x7febd0163460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:13.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.841+0000 7febd7a7d640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7febd01639a0 0x7febd006e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:13.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.841+0000 7febd7a7d640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7febd0166a70 con 0x7febd00cc270 2024-05-09T06:36:13.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.841+0000 7febd7a7d640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7febd0166d50 con 0x7febd01639a0 2024-05-09T06:36:13.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.841+0000 7febd7a7d640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7febd0167030 con 0x7febd00cc650 2024-05-09T06:36:13.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.841+0000 7f9108e18640 1 --2- 172.21.15.3:0/19884217 >> v2:172.21.15.3:3300/0 conn(0x7f91040d2df0 0x7f91040cf010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:13.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.842+0000 7f7b523fb640 1 -- 172.21.15.3:0/1717621930 wait complete. 2024-05-09T06:36:13.843 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.842+0000 7f59767fc640 1 --2- 172.21.15.3:0/3617954560 >> v2:172.21.15.3:3300/0 conn(0x7f597815ea40 0x7f59780cdbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:13.843 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.842+0000 7fc0b51a2640 1 -- 172.21.15.3:0/1227817894 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc0b0144910 msgr2=0x7fc0b00cf8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:13.843 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.842+0000 7fc0b51a2640 1 --2- 172.21.15.3:0/1227817894 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc0b0144910 0x7fc0b00cf8c0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fc0a00097a0 tx=0x7fc0a002a130 comp rx=0 tx=0).stop 2024-05-09T06:36:13.844 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.842+0000 7febd601a640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7febd01639a0 0x7febd006e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:13.844 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.842+0000 7febd601a640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7febd01639a0 0x7febd006e250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.3:47768/0 (socket says 172.21.15.3:47768) 2024-05-09T06:36:13.844 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.842+0000 7febd5819640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7febd00cc270 0x7febd0166220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:13.844 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.842+0000 7febd601a640 1 -- 172.21.15.3:0/2772657691 learned_addr learned my addr 172.21.15.3:0/2772657691 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:36:13.844 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.842+0000 7fc0b51a2640 1 -- 172.21.15.3:0/1227817894 shutdown_connections 2024-05-09T06:36:13.845 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.842+0000 7fc0b51a2640 1 --2- 172.21.15.3:0/1227817894 >> v2:172.21.15.190:3300/0 conn(0x7fc0b0072970 0x7fc0b0071080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:13.845 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.842+0000 7fc0b51a2640 1 --2- 172.21.15.3:0/1227817894 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc0b0144910 0x7fc0b00cf8c0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:13.845 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.842+0000 7fc0b51a2640 1 -- 172.21.15.3:0/1227817894 >> 172.21.15.3:0/1227817894 conn(0x7fc0b00b1c20 msgr2=0x7fc0b00b2030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:13.845 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.842+0000 7f5975ffb640 1 -- 172.21.15.3:0/3617954560 <== mon.1 v1:172.21.15.190:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3836170844 0 0) 0x7f5960003520 con 0x7f597806e250 2024-05-09T06:36:13.845 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.842+0000 7febd5819640 1 -- 172.21.15.3:0/2772657691 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7febd00cc650 msgr2=0x7febd0163460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:13.846 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.841+0000 7f926cea7640 1 -- 172.21.15.3:0/2915407568 wait complete. 2024-05-09T06:36:13.846 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.843+0000 7fc0b51a2640 1 -- 172.21.15.3:0/1227817894 shutdown_connections 2024-05-09T06:36:13.846 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.843+0000 7fc0b51a2640 1 -- 172.21.15.3:0/1227817894 wait complete. 2024-05-09T06:36:13.846 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.843+0000 7f7b523fb640 1 Processor -- start 2024-05-09T06:36:13.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.842+0000 7f5975ffb640 1 -- 172.21.15.3:0/3617954560 >> v2:172.21.15.3:3300/0 conn(0x7f597815ea40 msgr2=0x7f59780cdbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:13.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.842+0000 7f5975ffb640 1 --2- 172.21.15.3:0/3617954560 >> v2:172.21.15.3:3300/0 conn(0x7f597815ea40 0x7f59780cdbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:13.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.842+0000 7f5975ffb640 1 -- 172.21.15.3:0/3617954560 >> v1:172.21.15.3:6789/0 conn(0x7f597815d8a0 legacy=0x7f59780722a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:13.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.842+0000 7f5975ffb640 1 -- 172.21.15.3:0/3617954560 --> v1:172.21.15.190:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5978073040 con 0x7f597806e250 2024-05-09T06:36:13.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.842+0000 7f5975ffb640 1 -- 172.21.15.3:0/3617954560 <== mon.1 v1:172.21.15.190:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 563312031 0 0) 0x7f5970005030 con 0x7f597806e250 2024-05-09T06:36:13.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.842+0000 7f5975ffb640 1 -- 172.21.15.3:0/3617954560 >> v1:172.21.15.190:6789/0 conn(0x7f597806e250 legacy=0x7f59780b35e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:13.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.842+0000 7f5975ffb640 1 --2- 172.21.15.3:0/3617954560 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f59600039b0 0x7f5960003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:13.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.842+0000 7f5975ffb640 1 --2- 172.21.15.3:0/3617954560 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5960004800 0x7f5960004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:13.848 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.842+0000 7f5975ffb640 1 --2- 172.21.15.3:0/3617954560 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f59600051c0 0x7f596000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:13.848 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.842+0000 7febd5018640 1 --2- 172.21.15.3:0/2772657691 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7febd00cc650 0x7febd0163460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:13.848 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.843+0000 7febd5819640 1 --2- 172.21.15.3:0/2772657691 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7febd00cc650 0x7febd0163460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:13.848 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.843+0000 7febd5819640 1 -- 172.21.15.3:0/2772657691 >> v2:172.21.15.190:3300/0 conn(0x7febd01639a0 msgr2=0x7febd006e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:13.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.842+0000 7f9103fff640 1 -- 172.21.15.3:0/19884217 >> v2:172.21.15.3:3300/0 conn(0x7f91040d2df0 msgr2=0x7f91040cf010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:13.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.842+0000 7f9103fff640 1 --2- 172.21.15.3:0/19884217 >> v2:172.21.15.3:3300/0 conn(0x7f91040d2df0 0x7f91040cf010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:13.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.842+0000 7f9103fff640 1 -- 172.21.15.3:0/19884217 >> v1:172.21.15.190:6789/0 conn(0x7f91040d2000 legacy=0x7f91040d26e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:13.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.842+0000 7f9103fff640 1 -- 172.21.15.3:0/19884217 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91040b5270 con 0x7f910415e6f0 2024-05-09T06:36:13.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.843+0000 7f9103fff640 1 --2- 172.21.15.3:0/19884217 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f910415e6f0 0x7f91040b4170 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f910409daa0 tx=0x7f90f402a2c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1948885a6cbe153f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:13.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.843+0000 7f9102ffd640 1 -- 172.21.15.3:0/19884217 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90f4037070 con 0x7f910415e6f0 2024-05-09T06:36:13.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.843+0000 7f9102ffd640 1 -- 172.21.15.3:0/19884217 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90f4032640 con 0x7f910415e6f0 2024-05-09T06:36:13.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.843+0000 7f9102ffd640 1 -- 172.21.15.3:0/19884217 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90f4032a50 con 0x7f910415e6f0 2024-05-09T06:36:13.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.843+0000 7fe76759e640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fe7600ead00 0x7fe76004fa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:13.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.843+0000 7fe76759e640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fe760006ef0 0x7fe760007360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:13.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.843+0000 7fe76759e640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe760053720 con 0x7fe76004ffd0 2024-05-09T06:36:13.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.843+0000 7fe76759e640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fe7600078a0 con 0x7fe760006ef0 2024-05-09T06:36:13.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.843+0000 7fe76759e640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7fe760054330 con 0x7fe7600ead00 2024-05-09T06:36:13.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.843+0000 7fe765d9b640 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7fe76004ffd0 0x7fe760037bd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.3:58482/0 (socket says 172.21.15.3:58482) 2024-05-09T06:36:13.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.843+0000 7fe765d9b640 1 -- 172.21.15.3:0/3871196120 learned_addr learned my addr 172.21.15.3:0/3871196120 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-05-09T06:36:13.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.843+0000 7fe76659c640 1 --2- 172.21.15.3:0/3871196120 >> v2:172.21.15.190:3300/0 conn(0x7fe7600ead00 0x7fe76004fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:13.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.843+0000 7fe766d9d640 1 --2- 172.21.15.3:0/3871196120 >> v2:172.21.15.3:3300/0 conn(0x7fe760006ef0 0x7fe760007360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:13.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.843+0000 7fe76659c640 1 -- 172.21.15.3:0/3871196120 >> v2:172.21.15.3:3300/0 conn(0x7fe760006ef0 msgr2=0x7fe760007360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:13.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.843+0000 7fe76659c640 1 --2- 172.21.15.3:0/3871196120 >> v2:172.21.15.3:3300/0 conn(0x7fe760006ef0 0x7fe760007360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:13.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.843+0000 7fe76659c640 1 -- 172.21.15.3:0/3871196120 >> v1:172.21.15.190:6789/0 conn(0x7fe76004ffd0 legacy=0x7fe760037bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:13.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.843+0000 7fe76659c640 1 -- 172.21.15.3:0/3871196120 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe760053a00 con 0x7fe7600ead00 2024-05-09T06:36:13.851 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.843+0000 7fe76559a640 1 -- 172.21.15.3:0/3871196120 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3001593663 0 0) 0x7fe760053720 con 0x7fe76004ffd0 2024-05-09T06:36:13.851 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.842+0000 7f5975ffb640 1 -- 172.21.15.3:0/3617954560 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f596000c130 con 0x7f59600051c0 2024-05-09T06:36:13.851 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.842+0000 7f5975ffb640 1 -- 172.21.15.3:0/3617954560 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f596000c2e0 con 0x7f5960004800 2024-05-09T06:36:13.851 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.842+0000 7f5975ffb640 1 -- 172.21.15.3:0/3617954560 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f596000c4e0 con 0x7f59600039b0 2024-05-09T06:36:13.851 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.843+0000 7f59767fc640 1 --2- 172.21.15.3:0/3617954560 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5960004800 0x7f5960004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:13.851 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.843+0000 7f59777fe640 1 --2- 172.21.15.3:0/3617954560 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f59600051c0 0x7f596000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:13.851 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.843+0000 7f5976ffd640 1 --2- 172.21.15.3:0/3617954560 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f59600039b0 0x7f5960003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:13.851 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.843+0000 7f59777fe640 1 -- 172.21.15.3:0/3617954560 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f59600039b0 msgr2=0x7f5960003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:13.851 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.843+0000 7f59777fe640 1 --2- 172.21.15.3:0/3617954560 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f59600039b0 0x7f5960003dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:13.851 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.843+0000 7f59777fe640 1 -- 172.21.15.3:0/3617954560 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5960004800 msgr2=0x7f5960004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:13.852 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.843+0000 7f59777fe640 1 --2- 172.21.15.3:0/3617954560 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5960004800 0x7f5960004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:13.852 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.843+0000 7f59777fe640 1 -- 172.21.15.3:0/3617954560 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5978073040 con 0x7f59600051c0 2024-05-09T06:36:13.852 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.843+0000 7febd5819640 1 --2- 172.21.15.3:0/2772657691 >> v2:172.21.15.190:3300/0 conn(0x7febd01639a0 0x7febd006e250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:13.852 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.843+0000 7febd5819640 1 -- 172.21.15.3:0/2772657691 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febd0162f40 con 0x7febd00cc270 2024-05-09T06:36:13.853 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.843+0000 7f926cea7640 1 Processor -- start 2024-05-09T06:36:13.853 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.844+0000 7f926cea7640 1 -- start start 2024-05-09T06:36:13.853 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.844+0000 7febd5819640 1 --2- 172.21.15.3:0/2772657691 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7febd00cc270 0x7febd0166220 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7febd014c370 tx=0x7febcc02a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3528b9f4b4eed9a6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:13.853 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.844+0000 7febd5018640 1 --2- 172.21.15.3:0/2772657691 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7febd00cc650 0x7febd0163460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:36:13.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.844+0000 7f24c331d640 1 -- 172.21.15.3:0/3492620125 shutdown_connections 2024-05-09T06:36:13.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.844+0000 7f24c331d640 1 -- 172.21.15.3:0/3492620125 wait complete. 2024-05-09T06:36:13.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.845+0000 7f926cea7640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f92680cc2c0 0x7f9268141fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:13.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.845+0000 7f59777fe640 1 --2- 172.21.15.3:0/3617954560 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f59600051c0 0x7f596000b9c0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f59700015b0 tx=0x7f5960005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7490b36fbddbfce server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:13.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.845+0000 7f59777fe640 1 -- 172.21.15.3:0/3617954560 reap_dead start 2024-05-09T06:36:13.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.844+0000 7febd601a640 1 --2- 172.21.15.3:0/2772657691 >> v2:172.21.15.190:3300/0 conn(0x7febd01639a0 0x7febd006e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:36:13.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.845+0000 7febd4817640 1 -- 172.21.15.3:0/2772657691 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febcc037070 con 0x7febd00cc270 2024-05-09T06:36:13.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.845+0000 7febd4817640 1 -- 172.21.15.3:0/2772657691 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7febcc032440 con 0x7febd00cc270 2024-05-09T06:36:13.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.845+0000 7f910a87b640 1 -- 172.21.15.3:0/19884217 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f910415e6f0 msgr2=0x7f91040b4170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:13.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.845+0000 7f910a87b640 1 --2- 172.21.15.3:0/19884217 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f910415e6f0 0x7f91040b4170 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f910409daa0 tx=0x7f90f402a2c0 comp rx=0 tx=0).stop 2024-05-09T06:36:13.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.845+0000 7f910a87b640 1 -- 172.21.15.3:0/19884217 shutdown_connections 2024-05-09T06:36:13.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.845+0000 7f910a87b640 1 --2- 172.21.15.3:0/19884217 >> v2:172.21.15.3:3300/0 conn(0x7f91040d2df0 0x7f91040cf010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:13.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.845+0000 7f910a87b640 1 --2- 172.21.15.3:0/19884217 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f910415e6f0 0x7f91040b4170 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:13.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.845+0000 7f910a87b640 1 -- 172.21.15.3:0/19884217 >> 172.21.15.3:0/19884217 conn(0x7f910409d500 msgr2=0x7f91040d4220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:13.856 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.845+0000 7f910a87b640 1 -- 172.21.15.3:0/19884217 shutdown_connections 2024-05-09T06:36:13.856 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.845+0000 7fe76659c640 1 --2- 172.21.15.3:0/3871196120 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe7600ead00 0x7fe76004fa90 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fe760007ab0 tx=0x7fe75802a1a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=192f2e72ec68a1ba server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:13.856 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.845+0000 7fe76559a640 1 -- 172.21.15.3:0/3871196120 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe758037070 con 0x7fe7600ead00 2024-05-09T06:36:13.856 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.845+0000 7f926cea7640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9268162ae0 0x7f92680cf350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:13.856 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.845+0000 7f910a87b640 1 -- 172.21.15.3:0/19884217 wait complete. 2024-05-09T06:36:13.857 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.845+0000 7fe76559a640 1 -- 172.21.15.3:0/3871196120 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe758032440 con 0x7fe7600ead00 2024-05-09T06:36:13.857 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.845+0000 7f926cea7640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9268147ec0 0x7f9268160990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:13.857 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.845+0000 7f92677fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f92680cc2c0 0x7f9268141fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:13.857 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.845+0000 7f92677fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f92680cc2c0 0x7f9268141fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.3:47790/0 (socket says 172.21.15.3:47790) 2024-05-09T06:36:13.857 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.845+0000 7f92677fe640 1 -- 172.21.15.3:0/3133696160 learned_addr learned my addr 172.21.15.3:0/3133696160 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:36:13.858 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.845+0000 7f926cea7640 1 -- 172.21.15.3:0/3133696160 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f9268163900 con 0x7f9268162ae0 2024-05-09T06:36:13.858 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.846+0000 7f926cea7640 1 -- 172.21.15.3:0/3133696160 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f9268163620 con 0x7f92680cc2c0 2024-05-09T06:36:13.858 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.846+0000 7f9267fff640 1 --2- 172.21.15.3:0/3133696160 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9268147ec0 0x7f9268160990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:13.858 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.846+0000 7f926cea7640 1 -- 172.21.15.3:0/3133696160 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f92680b4490 con 0x7f9268147ec0 2024-05-09T06:36:13.858 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.846+0000 7fc0b51a2640 1 Processor -- start 2024-05-09T06:36:13.858 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.845+0000 7f7b523fb640 1 -- start start 2024-05-09T06:36:13.858 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.846+0000 7f7b523fb640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7b4c06b410 0x7f7b4c1fc320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:13.859 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.846+0000 7f7b523fb640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7b4c06e3f0 0x7f7b4c1f8490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:13.859 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.846+0000 7f7b523fb640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7b4c0706b0 0x7f7b4c0b0f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:13.859 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.846+0000 7f7b523fb640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f7b4c06ba20 con 0x7f7b4c0706b0 2024-05-09T06:36:13.859 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.846+0000 7f7b4bfff640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7b4c06b410 0x7f7b4c1fc320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:13.859 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.846+0000 7f7b4bfff640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7b4c06b410 0x7f7b4c1fc320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.3:45114/0 (socket says 172.21.15.3:45114) 2024-05-09T06:36:13.859 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.846+0000 7f7b4bfff640 1 -- 172.21.15.3:0/533500004 learned_addr learned my addr 172.21.15.3:0/533500004 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:36:13.859 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.846+0000 7f7b523fb640 1 -- 172.21.15.3:0/533500004 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f7b4c06b8a0 con 0x7f7b4c06e3f0 2024-05-09T06:36:13.859 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.846+0000 7f7b523fb640 1 -- 172.21.15.3:0/533500004 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f7b4c06bba0 con 0x7f7b4c06b410 2024-05-09T06:36:13.859 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.846+0000 7f7b4b7fe640 1 --2- 172.21.15.3:0/533500004 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7b4c06e3f0 0x7f7b4c1f8490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:13.859 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.846+0000 7febd7a7d640 1 -- 172.21.15.3:0/2772657691 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7febd00cc270 msgr2=0x7febd0166220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:13.859 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.846+0000 7febd7a7d640 1 --2- 172.21.15.3:0/2772657691 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7febd00cc270 0x7febd0166220 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7febd014c370 tx=0x7febcc02a140 comp rx=0 tx=0).stop 2024-05-09T06:36:13.860 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.846+0000 7fe76559a640 1 -- 172.21.15.3:0/3871196120 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe758032810 con 0x7fe7600ead00 2024-05-09T06:36:13.860 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.846+0000 7f7b50998640 1 --2- 172.21.15.3:0/533500004 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7b4c0706b0 0x7f7b4c0b0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:13.860 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.846+0000 7fe76759e640 1 -- 172.21.15.3:0/3871196120 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe7600ead00 msgr2=0x7fe76004fa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:13.860 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.846+0000 7febd7a7d640 1 -- 172.21.15.3:0/2772657691 shutdown_connections 2024-05-09T06:36:13.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.846+0000 7febd7a7d640 1 --2- 172.21.15.3:0/2772657691 >> v2:172.21.15.190:3300/0 conn(0x7febd01639a0 0x7febd006e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:13.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.846+0000 7febd7a7d640 1 --2- 172.21.15.3:0/2772657691 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7febd00cc650 0x7febd0163460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:13.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.846+0000 7f24c331d640 1 Processor -- start 2024-05-09T06:36:13.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.846+0000 7febd7a7d640 1 --2- 172.21.15.3:0/2772657691 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7febd00cc270 0x7febd0166220 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:13.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.846+0000 7febd7a7d640 1 -- 172.21.15.3:0/2772657691 >> 172.21.15.3:0/2772657691 conn(0x7febd00b1c20 msgr2=0x7febd00b2030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:13.862 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.846+0000 7fe76759e640 1 --2- 172.21.15.3:0/3871196120 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe7600ead00 0x7fe76004fa90 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fe760007ab0 tx=0x7fe75802a1a0 comp rx=0 tx=0).stop 2024-05-09T06:36:13.862 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.846+0000 7f24c331d640 1 -- start start 2024-05-09T06:36:13.862 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.846+0000 7f9266ffd640 1 --2- 172.21.15.3:0/3133696160 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9268162ae0 0x7f92680cf350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:13.862 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.847+0000 7fe76759e640 1 -- 172.21.15.3:0/3871196120 shutdown_connections 2024-05-09T06:36:13.863 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.847+0000 7fe76759e640 1 --2- 172.21.15.3:0/3871196120 >> v2:172.21.15.3:3300/0 conn(0x7fe760006ef0 0x7fe760007360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:13.863 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.847+0000 7fe76759e640 1 --2- 172.21.15.3:0/3871196120 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe7600ead00 0x7fe76004fa90 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:13.863 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.847+0000 7fe76759e640 1 -- 172.21.15.3:0/3871196120 >> 172.21.15.3:0/3871196120 conn(0x7fe760009fb0 msgr2=0x7fe76000a390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:13.863 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.847+0000 7f92677fe640 1 -- 172.21.15.3:0/3133696160 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9268147ec0 msgr2=0x7f9268160990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:13.863 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.847+0000 7f92677fe640 1 --2- 172.21.15.3:0/3133696160 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9268147ec0 0x7f9268160990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:13.863 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.847+0000 7f92677fe640 1 -- 172.21.15.3:0/3133696160 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9268162ae0 msgr2=0x7f92680cf350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:13.863 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.847+0000 7f92677fe640 1 --2- 172.21.15.3:0/3133696160 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9268162ae0 0x7f92680cf350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:13.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.847+0000 7f7b4bfff640 1 -- 172.21.15.3:0/533500004 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7b4c06e3f0 msgr2=0x7f7b4c1f8490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:13.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.847+0000 7f7b4bfff640 1 --2- 172.21.15.3:0/533500004 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7b4c06e3f0 0x7f7b4c1f8490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:13.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.847+0000 7f7b4bfff640 1 -- 172.21.15.3:0/533500004 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7b4c0706b0 msgr2=0x7f7b4c0b0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:13.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.847+0000 7f7b4bfff640 1 --2- 172.21.15.3:0/533500004 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7b4c0706b0 0x7f7b4c0b0f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:13.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.847+0000 7f7b4bfff640 1 -- 172.21.15.3:0/533500004 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b4c1f8d50 con 0x7f7b4c06b410 2024-05-09T06:36:13.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.847+0000 7f92677fe640 1 -- 172.21.15.3:0/3133696160 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9268148660 con 0x7f92680cc2c0 2024-05-09T06:36:13.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.847+0000 7f9267fff640 1 --2- 172.21.15.3:0/3133696160 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9268147ec0 0x7f9268160990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T06:36:13.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.847+0000 7f9266ffd640 1 --2- 172.21.15.3:0/3133696160 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9268162ae0 0x7f92680cf350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:36:13.865 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.847+0000 7febd7a7d640 1 -- 172.21.15.3:0/2772657691 shutdown_connections 2024-05-09T06:36:13.865 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.847+0000 7f92677fe640 1 --2- 172.21.15.3:0/3133696160 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f92680cc2c0 0x7f9268141fe0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f9260009f20 tx=0x7f926000c550 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:13.865 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.847+0000 7fc0b51a2640 1 -- start start 2024-05-09T06:36:13.866 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.847+0000 7f924ffff640 1 -- 172.21.15.3:0/3133696160 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9260014480 con 0x7f92680cc2c0 2024-05-09T06:36:13.866 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.847+0000 7f924ffff640 1 -- 172.21.15.3:0/3133696160 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9260019070 con 0x7f92680cc2c0 2024-05-09T06:36:13.866 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.847+0000 7f924ffff640 1 -- 172.21.15.3:0/3133696160 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9260014820 con 0x7f92680cc2c0 2024-05-09T06:36:13.866 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.847+0000 7f926cea7640 1 -- 172.21.15.3:0/3133696160 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9268148940 con 0x7f92680cc2c0 2024-05-09T06:36:13.866 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.847+0000 7f926cea7640 1 -- 172.21.15.3:0/3133696160 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9268149230 con 0x7f92680cc2c0 2024-05-09T06:36:13.866 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.847+0000 7f910a87b640 1 Processor -- start 2024-05-09T06:36:13.867 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.846+0000 7f5975ffb640 1 -- 172.21.15.3:0/3617954560 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5970008bd0 con 0x7f59600051c0 2024-05-09T06:36:13.867 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.846+0000 7f5975ffb640 1 -- 172.21.15.3:0/3617954560 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f597002f070 con 0x7f59600051c0 2024-05-09T06:36:13.867 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.846+0000 7f5975ffb640 1 -- 172.21.15.3:0/3617954560 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5970008bd0 con 0x7f59600051c0 2024-05-09T06:36:13.867 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.847+0000 7f597d968640 1 -- 172.21.15.3:0/3617954560 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f59600051c0 msgr2=0x7f596000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:13.867 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.847+0000 7f597d968640 1 --2- 172.21.15.3:0/3617954560 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f59600051c0 0x7f596000b9c0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f59700015b0 tx=0x7f5960005830 comp rx=0 tx=0).stop 2024-05-09T06:36:13.867 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.848+0000 7f597d968640 1 -- 172.21.15.3:0/3617954560 shutdown_connections 2024-05-09T06:36:13.867 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.848+0000 7f597d968640 1 --2- 172.21.15.3:0/3617954560 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f59600051c0 0x7f596000b9c0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:13.867 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.848+0000 7f597d968640 1 -- 172.21.15.3:0/3617954560 >> 172.21.15.3:0/3617954560 conn(0x7f59780b1c20 msgr2=0x7f59780b2030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:13.867 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.848+0000 7f597d968640 1 -- 172.21.15.3:0/3617954560 shutdown_connections 2024-05-09T06:36:13.867 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.848+0000 7f597d968640 1 -- 172.21.15.3:0/3617954560 wait complete. 2024-05-09T06:36:13.868 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.848+0000 7fe76759e640 1 -- 172.21.15.3:0/3871196120 shutdown_connections 2024-05-09T06:36:13.868 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.848+0000 7f24c331d640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f24bc070360 0x7f24bc158390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:13.868 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.848+0000 7f24c331d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f24bc160000 0x7f24bc156080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:13.869 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.848+0000 7f910a87b640 1 -- start start 2024-05-09T06:36:13.869 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.848+0000 7f24c331d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f24bc1603e0 0x7f24bc0c9790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:13.869 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.848+0000 7f24c331d640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f24bc0d0d70 con 0x7f24bc1603e0 2024-05-09T06:36:13.869 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.848+0000 7f24c331d640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f24bc06fb00 con 0x7f24bc160000 2024-05-09T06:36:13.869 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.848+0000 7f24c331d640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f24bc0cc9d0 con 0x7f24bc070360 2024-05-09T06:36:13.870 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.848+0000 7fe76759e640 1 -- 172.21.15.3:0/3871196120 wait complete. 2024-05-09T06:36:13.870 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.848+0000 7f24c18ba640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f24bc1603e0 0x7f24bc0c9790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:13.871 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.848+0000 7f24c18ba640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f24bc1603e0 0x7f24bc0c9790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49614/0 (socket says 172.21.15.3:49614) 2024-05-09T06:36:13.871 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.848+0000 7f24c18ba640 1 -- 172.21.15.3:0/2144740818 learned_addr learned my addr 172.21.15.3:0/2144740818 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:36:13.871 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.848+0000 7f24c10b9640 1 --2- 172.21.15.3:0/2144740818 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f24bc070360 0x7f24bc158390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:13.871 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.849+0000 7fc0b51a2640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc0b0072970 0x7fc0b013e560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:13.871 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.849+0000 7f24c10b9640 1 -- 172.21.15.3:0/2144740818 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f24bc160000 msgr2=0x7f24bc156080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:13.871 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.849+0000 7f24c10b9640 1 --2- 172.21.15.3:0/2144740818 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f24bc160000 0x7f24bc156080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:13.871 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.849+0000 7f24c10b9640 1 -- 172.21.15.3:0/2144740818 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f24bc1603e0 msgr2=0x7f24bc0c9790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:13.872 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.849+0000 7fc0b51a2640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc0b00b4130 0x7fc0b015b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:13.872 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.849+0000 7fc0aed9d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc0b0072970 0x7fc0b013e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:13.872 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.847+0000 7febd7a7d640 1 -- 172.21.15.3:0/2772657691 wait complete. 2024-05-09T06:36:13.872 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.849+0000 7febd7a7d640 1 Processor -- start 2024-05-09T06:36:13.872 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.849+0000 7febd7a7d640 1 -- start start 2024-05-09T06:36:13.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.849+0000 7fc0aed9d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc0b0072970 0x7fc0b013e560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.3:47820/0 (socket says 172.21.15.3:47820) 2024-05-09T06:36:13.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.849+0000 7fc0ae59c640 1 --2- 172.21.15.3:0/1844828524 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc0b00b4130 0x7fc0b015b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:13.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.849+0000 7f24c10b9640 1 --2- 172.21.15.3:0/2144740818 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f24bc1603e0 0x7f24bc0c9790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:13.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.849+0000 7f24c10b9640 1 -- 172.21.15.3:0/2144740818 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24bc1572f0 con 0x7f24bc070360 2024-05-09T06:36:13.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.849+0000 7f24c18ba640 1 --2- 172.21.15.3:0/2144740818 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f24bc1603e0 0x7f24bc0c9790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T06:36:13.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.849+0000 7fc0aed9d640 1 -- 172.21.15.3:0/1844828524 learned_addr learned my addr 172.21.15.3:0/1844828524 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:36:13.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.849+0000 7fc0b51a2640 1 --2- 172.21.15.3:0/1844828524 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc0b0144910 0x7fc0b01594c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:13.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.849+0000 7f7b4bfff640 1 --2- 172.21.15.3:0/533500004 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7b4c06b410 0x7f7b4c1fc320 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f7b44009e50 tx=0x7f7b44004a70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:13.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.849+0000 7f910a87b640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f91040d2000 0x7f9104152630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:13.875 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.849+0000 7f24c10b9640 1 --2- 172.21.15.3:0/2144740818 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f24bc070360 0x7f24bc158390 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f24b8035000 tx=0x7f24b80046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:13.875 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.849+0000 7fc0b51a2640 1 -- 172.21.15.3:0/1844828524 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fc0b00715c0 con 0x7fc0b00b4130 2024-05-09T06:36:13.875 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.849+0000 7f910a87b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f91040d2df0 0x7f91040c9180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:13.875 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.850+0000 7fe76759e640 1 Processor -- start 2024-05-09T06:36:13.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.850+0000 7f597d968640 1 Processor -- start 2024-05-09T06:36:13.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.850+0000 7f9103fff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f91040d2000 0x7f9104152630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:13.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.850+0000 7f9103fff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f91040d2000 0x7f9104152630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.3:47822/0 (socket says 172.21.15.3:47822) 2024-05-09T06:36:13.877 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.850+0000 7fc0af59e640 1 --2- 172.21.15.3:0/1844828524 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc0b0144910 0x7fc0b01594c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:13.877 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.850+0000 7fc0b51a2640 1 -- 172.21.15.3:0/1844828524 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fc0b0072170 con 0x7fc0b0072970 2024-05-09T06:36:13.877 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.850+0000 7fc0b51a2640 1 -- 172.21.15.3:0/1844828524 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fc0b00b45c0 con 0x7fc0b0144910 2024-05-09T06:36:13.877 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.850+0000 7f9103fff640 1 -- 172.21.15.3:0/1774738169 learned_addr learned my addr 172.21.15.3:0/1774738169 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:36:13.877 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.850+0000 7fe76759e640 1 -- start start 2024-05-09T06:36:13.878 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.850+0000 7f910a87b640 1 --2- 172.21.15.3:0/1774738169 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f910415e6f0 0x7f91040d0c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:13.878 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.850+0000 7f924ffff640 1 -- 172.21.15.3:0/3133696160 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9260005d10 con 0x7f92680cc2c0 2024-05-09T06:36:13.878 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.850+0000 7f24b1ffb640 1 -- 172.21.15.3:0/2144740818 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24b8044020 con 0x7f24bc070360 2024-05-09T06:36:13.878 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.850+0000 7f24c331d640 1 -- 172.21.15.3:0/2144740818 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24bc1575d0 con 0x7f24bc070360 2024-05-09T06:36:13.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.850+0000 7f24c331d640 1 -- 172.21.15.3:0/2144740818 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f24bc157b90 con 0x7f24bc070360 2024-05-09T06:36:13.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.850+0000 7f910a87b640 1 -- 172.21.15.3:0/1774738169 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f91040b46b0 con 0x7f91040d2df0 2024-05-09T06:36:13.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.850+0000 7f24b1ffb640 1 -- 172.21.15.3:0/2144740818 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f24b8033710 con 0x7f24bc070360 2024-05-09T06:36:13.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.850+0000 7f24b1ffb640 1 -- 172.21.15.3:0/2144740818 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24b803a040 con 0x7f24bc070360 2024-05-09T06:36:13.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.850+0000 7f9108e18640 1 --2- 172.21.15.3:0/1774738169 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f910415e6f0 0x7f91040d0c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:13.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.850+0000 7f910a87b640 1 -- 172.21.15.3:0/1774738169 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f910415ead0 con 0x7f91040d2000 2024-05-09T06:36:13.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.850+0000 7fc0af59e640 1 -- 172.21.15.3:0/1844828524 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc0b0072970 msgr2=0x7fc0b013e560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:13.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.850+0000 7fc0af59e640 1 --2- 172.21.15.3:0/1844828524 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc0b0072970 0x7fc0b013e560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:13.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.850+0000 7febd7a7d640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7febd00cc270 0x7febd0147a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:13.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.850+0000 7fc0af59e640 1 -- 172.21.15.3:0/1844828524 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc0b00b4130 msgr2=0x7fc0b015b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:13.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.850+0000 7fc0af59e640 1 --2- 172.21.15.3:0/1844828524 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc0b00b4130 0x7fc0b015b970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:13.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.850+0000 7fc0af59e640 1 -- 172.21.15.3:0/1844828524 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0b013ced0 con 0x7fc0b0144910 2024-05-09T06:36:13.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.851+0000 7febd7a7d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7febd00cc650 0x7febd01449f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:13.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.850+0000 7f924ffff640 1 --2- 172.21.15.3:0/3133696160 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f925406be30 0x7f925406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:13.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.851+0000 7f924ffff640 1 -- 172.21.15.3:0/3133696160 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f926008b260 con 0x7f92680cc2c0 2024-05-09T06:36:13.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.850+0000 7f597d968640 1 -- start start 2024-05-09T06:36:13.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.851+0000 7f910a87b640 1 -- 172.21.15.3:0/1774738169 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f91040b5a70 con 0x7f910415e6f0 2024-05-09T06:36:13.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.851+0000 7f91037fe640 1 --2- 172.21.15.3:0/1774738169 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f91040d2df0 0x7f91040c9180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:13.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.851+0000 7f9103fff640 1 -- 172.21.15.3:0/1774738169 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f910415e6f0 msgr2=0x7f91040d0c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:13.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.851+0000 7f9103fff640 1 --2- 172.21.15.3:0/1774738169 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f910415e6f0 0x7f91040d0c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:13.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.851+0000 7f9103fff640 1 -- 172.21.15.3:0/1774738169 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f91040d2df0 msgr2=0x7f91040c9180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:13.883 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.851+0000 7fe76759e640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe760006ef0 0x7fe7601804e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:13.883 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.851+0000 7fe76759e640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe76004ffd0 0x7fe7600468b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:13.883 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.851+0000 7fe76759e640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe7600ead00 0x7fe760046df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:13.883 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.851+0000 7fe76759e640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fe7600547c0 con 0x7fe76004ffd0 2024-05-09T06:36:13.883 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.851+0000 7fe765d9b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe76004ffd0 0x7fe7600468b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:13.883 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.851+0000 7fe765d9b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe76004ffd0 0x7fe7600468b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49646/0 (socket says 172.21.15.3:49646) 2024-05-09T06:36:13.883 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.851+0000 7fe765d9b640 1 -- 172.21.15.3:0/2097467231 learned_addr learned my addr 172.21.15.3:0/2097467231 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:36:13.883 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.851+0000 7fe76759e640 1 -- 172.21.15.3:0/2097467231 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fe760053fc0 con 0x7fe7600ead00 2024-05-09T06:36:13.883 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.851+0000 7fe766d9d640 1 --2- 172.21.15.3:0/2097467231 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe7600ead00 0x7fe760046df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:13.883 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.851+0000 7fe76759e640 1 -- 172.21.15.3:0/2097467231 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fe760054940 con 0x7fe760006ef0 2024-05-09T06:36:13.884 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.851+0000 7f9266ffd640 1 --2- 172.21.15.3:0/3133696160 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f925406be30 0x7f925406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:13.884 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.851+0000 7fe76659c640 1 --2- 172.21.15.3:0/2097467231 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe760006ef0 0x7fe7601804e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:13.884 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.851+0000 7f9103fff640 1 --2- 172.21.15.3:0/1774738169 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f91040d2df0 0x7f91040c9180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:13.884 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.851+0000 7f9103fff640 1 -- 172.21.15.3:0/1774738169 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91040c9b90 con 0x7f91040d2000 2024-05-09T06:36:13.884 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.852+0000 7f91037fe640 1 --2- 172.21.15.3:0/1774738169 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f91040d2df0 0x7f91040c9180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T06:36:13.885 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.851+0000 7f7b48ff9640 1 -- 172.21.15.3:0/533500004 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b44041020 con 0x7f7b4c06b410 2024-05-09T06:36:13.885 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.851+0000 7f7b48ff9640 1 -- 172.21.15.3:0/533500004 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b44037040 con 0x7f7b4c06b410 2024-05-09T06:36:13.885 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.851+0000 7f7b48ff9640 1 -- 172.21.15.3:0/533500004 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b44036070 con 0x7f7b4c06b410 2024-05-09T06:36:13.885 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.852+0000 7f24c331d640 1 -- 172.21.15.3:0/2144740818 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f249c005290 con 0x7f24bc070360 2024-05-09T06:36:13.885 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.852+0000 7f7b523fb640 1 -- 172.21.15.3:0/533500004 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b4c1f9030 con 0x7f7b4c06b410 2024-05-09T06:36:13.886 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.852+0000 7fe76659c640 1 -- 172.21.15.3:0/2097467231 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe7600ead00 msgr2=0x7fe760046df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:13.886 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.852+0000 7fe76659c640 1 --2- 172.21.15.3:0/2097467231 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe7600ead00 0x7fe760046df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:13.886 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.852+0000 7fe76659c640 1 -- 172.21.15.3:0/2097467231 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe76004ffd0 msgr2=0x7fe7600468b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:13.886 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.852+0000 7fe76659c640 1 --2- 172.21.15.3:0/2097467231 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe76004ffd0 0x7fe7600468b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:13.886 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.852+0000 7fe76659c640 1 -- 172.21.15.3:0/2097467231 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe76004d910 con 0x7fe760006ef0 2024-05-09T06:36:13.886 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.852+0000 7f7b523fb640 1 -- 172.21.15.3:0/533500004 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b4c1f9310 con 0x7f7b4c06b410 2024-05-09T06:36:13.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.852+0000 7fe765d9b640 1 --2- 172.21.15.3:0/2097467231 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe76004ffd0 0x7fe7600468b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T06:36:13.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.851+0000 7f597d968640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f59600051c0 0x7f59781567f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:13.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.851+0000 7f597d968640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5978149a00 0x7f59780ab0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:13.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.851+0000 7f597d968640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5978155270 0x7f59780ab620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:13.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.851+0000 7f597d968640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f597815dc80 con 0x7f5978149a00 2024-05-09T06:36:13.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.851+0000 7f597d968640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f5978073320 con 0x7f59600051c0 2024-05-09T06:36:13.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.851+0000 7f597d968640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f597806f110 con 0x7f5978155270 2024-05-09T06:36:13.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.851+0000 7f59777fe640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5978155270 0x7f59780ab620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:13.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.851+0000 7f59777fe640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5978155270 0x7f59780ab620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.3:45186/0 (socket says 172.21.15.3:45186) 2024-05-09T06:36:13.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.851+0000 7f59777fe640 1 -- 172.21.15.3:0/1469160415 learned_addr learned my addr 172.21.15.3:0/1469160415 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:36:13.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.852+0000 7f5976ffd640 1 --2- 172.21.15.3:0/1469160415 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f59600051c0 0x7f59781567f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:13.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.852+0000 7f59767fc640 1 --2- 172.21.15.3:0/1469160415 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5978149a00 0x7f59780ab0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:13.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.852+0000 7febd5819640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7febd00cc270 0x7febd0147a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:13.889 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.852+0000 7f9103fff640 1 --2- 172.21.15.3:0/1774738169 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f91040d2000 0x7f9104152630 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f90f4000c00 tx=0x7f90f40046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:13.889 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.852+0000 7f5976ffd640 1 -- 172.21.15.3:0/1469160415 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5978155270 msgr2=0x7f59780ab620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:13.889 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.852+0000 7f5976ffd640 1 --2- 172.21.15.3:0/1469160415 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5978155270 0x7f59780ab620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:13.889 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.852+0000 7f5976ffd640 1 -- 172.21.15.3:0/1469160415 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5978149a00 msgr2=0x7f59780ab0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:13.889 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.852+0000 7f5976ffd640 1 --2- 172.21.15.3:0/1469160415 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5978149a00 0x7f59780ab0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:13.889 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.852+0000 7f5976ffd640 1 -- 172.21.15.3:0/1469160415 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5978155750 con 0x7f59600051c0 2024-05-09T06:36:13.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.852+0000 7f9266ffd640 1 --2- 172.21.15.3:0/3133696160 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f925406be30 0x7f925406e2f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f925c034000 tx=0x7f925c02e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:13.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.852+0000 7f926cea7640 1 -- 172.21.15.3:0/3133696160 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9268148e60 con 0x7f92680cc2c0 2024-05-09T06:36:13.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.852+0000 7fc0af59e640 1 --2- 172.21.15.3:0/1844828524 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc0b0144910 0x7fc0b01594c0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fc0a800cb20 tx=0x7fc0a800cf20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:13.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.852+0000 7febd7a7d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7febd01639a0 0x7febd014b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:13.891 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.852+0000 7f59777fe640 1 --2- 172.21.15.3:0/1469160415 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5978155270 0x7f59780ab620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T06:36:13.891 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.853+0000 7f24b1ffb640 1 -- 172.21.15.3:0/2144740818 <== mon.2 v2:172.21.15.190:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f24b8047020 con 0x7f24bc070360 2024-05-09T06:36:13.891 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.852+0000 7febd7a7d640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7febd006e790 con 0x7febd01639a0 2024-05-09T06:36:13.891 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.852+0000 7febd7a7d640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7febd0166760 con 0x7febd00cc650 2024-05-09T06:36:13.891 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.852+0000 7febd7a7d640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7febd006e910 con 0x7febd00cc270 2024-05-09T06:36:13.892 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.852+0000 7f9100ff9640 1 -- 172.21.15.3:0/1774738169 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90f4041020 con 0x7f91040d2000 2024-05-09T06:36:13.892 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.853+0000 7fc0977fe640 1 -- 172.21.15.3:0/1844828524 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0a8017070 con 0x7fc0b0144910 2024-05-09T06:36:13.892 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.853+0000 7f9100ff9640 1 -- 172.21.15.3:0/1774738169 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90f40309b0 con 0x7f91040d2000 2024-05-09T06:36:13.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.853+0000 7febd5819640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7febd00cc270 0x7febd0147a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.3:45162/0 (socket says 172.21.15.3:45162) 2024-05-09T06:36:13.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.852+0000 7f59767fc640 1 --2- 172.21.15.3:0/1469160415 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5978149a00 0x7f59780ab0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:36:13.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.853+0000 7f5976ffd640 1 --2- 172.21.15.3:0/1469160415 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f59600051c0 0x7f59781567f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f596c015c80 tx=0x7f596c0134b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:13.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.853+0000 7febd5819640 1 -- 172.21.15.3:0/3506697645 learned_addr learned my addr 172.21.15.3:0/3506697645 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:36:13.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.853+0000 7fc0977fe640 1 -- 172.21.15.3:0/1844828524 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc0a800abd0 con 0x7fc0b0144910 2024-05-09T06:36:13.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.853+0000 7fe76659c640 1 --2- 172.21.15.3:0/2097467231 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe760006ef0 0x7fe7601804e0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fe758031000 tx=0x7fe7580046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:13.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.853+0000 7fc0b51a2640 1 -- 172.21.15.3:0/1844828524 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0b013d140 con 0x7fc0b0144910 2024-05-09T06:36:13.895 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.853+0000 7f9100ff9640 1 -- 172.21.15.3:0/1774738169 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90f4037040 con 0x7f91040d2000 2024-05-09T06:36:13.895 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.853+0000 7f910a87b640 1 -- 172.21.15.3:0/1774738169 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91040c9e70 con 0x7f91040d2000 2024-05-09T06:36:13.895 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.853+0000 7f910a87b640 1 -- 172.21.15.3:0/1774738169 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91040ca430 con 0x7f91040d2000 2024-05-09T06:36:13.895 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.853+0000 7febd5018640 1 --2- 172.21.15.3:0/3506697645 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7febd00cc650 0x7febd01449f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:13.895 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.853+0000 7febd601a640 1 --2- 172.21.15.3:0/3506697645 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7febd01639a0 0x7febd014b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:13.895 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.853+0000 7febd5018640 1 -- 172.21.15.3:0/3506697645 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7febd00cc270 msgr2=0x7febd0147a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:13.895 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.853+0000 7febd5018640 1 --2- 172.21.15.3:0/3506697645 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7febd00cc270 0x7febd0147a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:13.895 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.853+0000 7febd5018640 1 -- 172.21.15.3:0/3506697645 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7febd01639a0 msgr2=0x7febd014b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:13.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.853+0000 7f24b1ffb640 1 --2- 172.21.15.3:0/2144740818 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f24a006be30 0x7f24a006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:13.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.853+0000 7febd5018640 1 --2- 172.21.15.3:0/3506697645 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7febd01639a0 0x7febd014b1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:13.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.853+0000 7fe75effd640 1 -- 172.21.15.3:0/2097467231 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe758032e60 con 0x7fe760006ef0 2024-05-09T06:36:13.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.853+0000 7fe75effd640 1 -- 172.21.15.3:0/2097467231 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe758030770 con 0x7fe760006ef0 2024-05-09T06:36:13.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.853+0000 7fe76759e640 1 -- 172.21.15.3:0/2097467231 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe76004dbf0 con 0x7fe760006ef0 2024-05-09T06:36:13.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.853+0000 7fe76759e640 1 -- 172.21.15.3:0/2097467231 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fe76004e1b0 con 0x7fe760006ef0 2024-05-09T06:36:13.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.853+0000 7febd5018640 1 -- 172.21.15.3:0/3506697645 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febd014a150 con 0x7febd00cc650 2024-05-09T06:36:13.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.853+0000 7fc0b51a2640 1 -- 172.21.15.3:0/1844828524 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0b013d700 con 0x7fc0b0144910 2024-05-09T06:36:13.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.854+0000 7febd601a640 1 --2- 172.21.15.3:0/3506697645 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7febd01639a0 0x7febd014b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T06:36:13.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.854+0000 7febd5018640 1 --2- 172.21.15.3:0/3506697645 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7febd00cc650 0x7febd01449f0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7febc400c7c0 tx=0x7febc400cc80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:13.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.854+0000 7f7b523fb640 1 -- 172.21.15.3:0/533500004 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b4c04f5f0 con 0x7f7b4c06b410 2024-05-09T06:36:13.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.854+0000 7fc0977fe640 1 -- 172.21.15.3:0/1844828524 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0a8012400 con 0x7fc0b0144910 2024-05-09T06:36:13.899 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.854+0000 7f24b1ffb640 1 -- 172.21.15.3:0/2144740818 <== mon.2 v2:172.21.15.190:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f24b80afa10 con 0x7f24bc070360 2024-05-09T06:36:13.899 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.854+0000 7f24c08b8640 1 --2- 172.21.15.3:0/2144740818 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f24a006be30 0x7f24a006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:13.899 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.854+0000 7f24c08b8640 1 --2- 172.21.15.3:0/2144740818 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f24a006be30 0x7f24a006e2f0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f24bc0b80a0 tx=0x7f24ac007a40 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:13.899 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.854+0000 7fe75effd640 1 -- 172.21.15.3:0/2097467231 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe758037040 con 0x7fe760006ef0 2024-05-09T06:36:13.899 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.854+0000 7f7b48ff9640 1 -- 172.21.15.3:0/533500004 <== mon.2 v2:172.21.15.190:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7b44044020 con 0x7f7b4c06b410 2024-05-09T06:36:13.900 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.854+0000 7f910a87b640 1 -- 172.21.15.3:0/1774738169 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91040682e0 con 0x7f91040d2000 2024-05-09T06:36:13.900 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.855+0000 7fe76759e640 1 -- 172.21.15.3:0/2097467231 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe744005290 con 0x7fe760006ef0 2024-05-09T06:36:13.900 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.855+0000 7f7b48ff9640 1 --2- 172.21.15.3:0/533500004 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f7b3006be80 0x7f7b3006e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:13.900 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.855+0000 7fc0b51a2640 1 -- 172.21.15.3:0/1844828524 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc090005290 con 0x7fc0b0144910 2024-05-09T06:36:13.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.855+0000 7f7b48ff9640 1 -- 172.21.15.3:0/533500004 <== mon.2 v2:172.21.15.190:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f7b440ab9a0 con 0x7f7b4c06b410 2024-05-09T06:36:13.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.855+0000 7f7b4b7fe640 1 --2- 172.21.15.3:0/533500004 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f7b3006be80 0x7f7b3006e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:13.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.856+0000 7f7b4b7fe640 1 --2- 172.21.15.3:0/533500004 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f7b3006be80 0x7f7b3006e340 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f7b4c156580 tx=0x7f7b38009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:13.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.863+0000 7f595ffff640 1 -- 172.21.15.3:0/1469160415 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f596c020070 con 0x7f59600051c0 2024-05-09T06:36:13.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.863+0000 7febc27fc640 1 -- 172.21.15.3:0/3506697645 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febc4015070 con 0x7febd00cc650 2024-05-09T06:36:13.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.863+0000 7febd7a7d640 1 -- 172.21.15.3:0/3506697645 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7febd014a3f0 con 0x7febd00cc650 2024-05-09T06:36:13.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.863+0000 7f597d968640 1 -- 172.21.15.3:0/1469160415 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5978155a30 con 0x7f59600051c0 2024-05-09T06:36:13.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.863+0000 7febd7a7d640 1 -- 172.21.15.3:0/3506697645 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7febd014ac90 con 0x7febd00cc650 2024-05-09T06:36:13.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.863+0000 7f597d968640 1 -- 172.21.15.3:0/1469160415 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5978155ff0 con 0x7f59600051c0 2024-05-09T06:36:13.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.863+0000 7f924ffff640 1 -- 172.21.15.3:0/3133696160 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f9268148e60 con 0x7f92680cc2c0 2024-05-09T06:36:13.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.864+0000 7f595ffff640 1 -- 172.21.15.3:0/1469160415 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f596c013dd0 con 0x7f59600051c0 2024-05-09T06:36:13.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.864+0000 7febc27fc640 1 -- 172.21.15.3:0/3506697645 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7febc4007820 con 0x7febd00cc650 2024-05-09T06:36:13.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.864+0000 7febc27fc640 1 -- 172.21.15.3:0/3506697645 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febc401f060 con 0x7febd00cc650 2024-05-09T06:36:13.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.864+0000 7f24b1ffb640 1 -- 172.21.15.3:0/2144740818 <== mon.2 v2:172.21.15.190:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f24b804e030 con 0x7f24bc070360 2024-05-09T06:36:13.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.864+0000 7f595ffff640 1 -- 172.21.15.3:0/1469160415 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f596c02a060 con 0x7f59600051c0 2024-05-09T06:36:13.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.865+0000 7febd7a7d640 1 -- 172.21.15.3:0/3506697645 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7febb8005290 con 0x7febd00cc650 2024-05-09T06:36:13.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.865+0000 7f595ffff640 1 -- 172.21.15.3:0/1469160415 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f596c023070 con 0x7f59600051c0 2024-05-09T06:36:13.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.865+0000 7f597d968640 1 -- 172.21.15.3:0/1469160415 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5958005290 con 0x7f59600051c0 2024-05-09T06:36:13.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.865+0000 7f9100ff9640 1 -- 172.21.15.3:0/1774738169 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f90f4044020 con 0x7f91040d2000 2024-05-09T06:36:13.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.866+0000 7fe75effd640 1 -- 172.21.15.3:0/2097467231 <== mon.2 v2:172.21.15.190:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe758044020 con 0x7fe760006ef0 2024-05-09T06:36:13.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.866+0000 7febc27fc640 1 -- 172.21.15.3:0/3506697645 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7febc4018070 con 0x7febd00cc650 2024-05-09T06:36:13.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.866+0000 7f9100ff9640 1 --2- 172.21.15.3:0/1774738169 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f90f006be30 0x7f90f006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:13.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.866+0000 7f595ffff640 1 --2- 172.21.15.3:0/1469160415 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f596406c030 0x7f596406e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:13.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.866+0000 7fe75effd640 1 --2- 172.21.15.3:0/2097467231 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fe75006be80 0x7fe75006e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:13.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.866+0000 7f91037fe640 1 --2- 172.21.15.3:0/1774738169 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f90f006be30 0x7f90f006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:13.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.866+0000 7f9100ff9640 1 -- 172.21.15.3:0/1774738169 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f90f40abb90 con 0x7f91040d2000 2024-05-09T06:36:13.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.866+0000 7febc27fc640 1 --2- 172.21.15.3:0/3506697645 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7febb406be30 0x7febb406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:13.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.867+0000 7fe765d9b640 1 --2- 172.21.15.3:0/2097467231 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fe75006be80 0x7fe75006e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:13.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.867+0000 7f59767fc640 1 --2- 172.21.15.3:0/1469160415 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f596406c030 0x7f596406e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:13.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.867+0000 7febd5819640 1 --2- 172.21.15.3:0/3506697645 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7febb406be30 0x7febb406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:13.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.867+0000 7f91037fe640 1 --2- 172.21.15.3:0/1774738169 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f90f006be30 0x7f90f006e2f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f9104152b70 tx=0x7f90f8007d30 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:13.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.867+0000 7f59767fc640 1 --2- 172.21.15.3:0/1469160415 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f596406c030 0x7f596406e4f0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f59781f2f50 tx=0x7f5968013040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:13.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.867+0000 7fe765d9b640 1 --2- 172.21.15.3:0/2097467231 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fe75006be80 0x7fe75006e340 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fe754007b10 tx=0x7fe754007a60 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:13.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.867+0000 7febd5819640 1 --2- 172.21.15.3:0/3506697645 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7febb406be30 0x7febb406e2f0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7febcc009560 tx=0x7febcc02e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:13.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.874+0000 7f7b48ff9640 1 -- 172.21.15.3:0/533500004 <== mon.2 v2:172.21.15.190:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f7b4404b030 con 0x7f7b4c06b410 2024-05-09T06:36:13.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.875+0000 7f595ffff640 1 -- 172.21.15.3:0/1469160415 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f596c090c70 con 0x7f59600051c0 2024-05-09T06:36:13.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.875+0000 7febc27fc640 1 -- 172.21.15.3:0/3506697645 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7febc40572d0 con 0x7febd00cc650 2024-05-09T06:36:13.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.875+0000 7fe75effd640 1 -- 172.21.15.3:0/2097467231 <== mon.2 v2:172.21.15.190:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fe758077f90 con 0x7fe760006ef0 2024-05-09T06:36:13.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.875+0000 7f9100ff9640 1 -- 172.21.15.3:0/1774738169 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f90f404b030 con 0x7f91040d2000 2024-05-09T06:36:13.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.876+0000 7fc0977fe640 1 -- 172.21.15.3:0/1844828524 <== mon.2 v2:172.21.15.190:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc0a801a070 con 0x7fc0b0144910 2024-05-09T06:36:13.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.877+0000 7fc0977fe640 1 --2- 172.21.15.3:0/1844828524 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fc09c06e090 0x7fc09c070550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:13.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.877+0000 7fc0977fe640 1 -- 172.21.15.3:0/1844828524 <== mon.2 v2:172.21.15.190:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fc0a808afa0 con 0x7fc0b0144910 2024-05-09T06:36:13.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.877+0000 7fc0aed9d640 1 --2- 172.21.15.3:0/1844828524 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fc09c06e090 0x7fc09c070550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:13.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.877+0000 7fc0aed9d640 1 --2- 172.21.15.3:0/1844828524 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fc09c06e090 0x7fc09c070550 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fc0a002a170 tx=0x7fc0a002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:13.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.884+0000 7fe75effd640 1 -- 172.21.15.3:0/2097467231 <== mon.2 v2:172.21.15.190:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fe758079ef0 con 0x7fe760006ef0 2024-05-09T06:36:13.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.885+0000 7f595ffff640 1 -- 172.21.15.3:0/1469160415 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f596c060910 con 0x7f59600051c0 2024-05-09T06:36:13.913 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.890+0000 7febc27fc640 1 -- 172.21.15.3:0/3506697645 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7febc4059230 con 0x7febd00cc650 2024-05-09T06:36:13.913 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:13.890+0000 7fc0977fe640 1 -- 172.21.15.3:0/1844828524 <== mon.2 v2:172.21.15.190:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fc0a8021020 con 0x7fc0b0144910 2024-05-09T06:36:14.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:14.262+0000 7f926cea7640 1 -- 172.21.15.3:0/3133696160 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f9268069550 con 0x7f92680cc2c0 2024-05-09T06:36:14.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:14.262+0000 7f924ffff640 1 -- 172.21.15.3:0/3133696160 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f9260010040 con 0x7f92680cc2c0 2024-05-09T06:36:14.243 INFO:teuthology.orchestra.run.smithi003.stdout:55834574853 2024-05-09T06:36:14.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:14.266+0000 7f924dffb640 1 -- 172.21.15.3:0/3133696160 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f925406be30 msgr2=0x7f925406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:14.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:14.266+0000 7f924dffb640 1 --2- 172.21.15.3:0/3133696160 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f925406be30 0x7f925406e2f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f925c034000 tx=0x7f925c02e000 comp rx=0 tx=0).stop 2024-05-09T06:36:14.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:14.266+0000 7f924dffb640 1 -- 172.21.15.3:0/3133696160 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f92680cc2c0 msgr2=0x7f9268141fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:14.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:14.266+0000 7f924dffb640 1 --2- 172.21.15.3:0/3133696160 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f92680cc2c0 0x7f9268141fe0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f9260009f20 tx=0x7f926000c550 comp rx=0 tx=0).stop 2024-05-09T06:36:14.247 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:14.266+0000 7f924dffb640 1 -- 172.21.15.3:0/3133696160 shutdown_connections 2024-05-09T06:36:14.247 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:14.266+0000 7f924dffb640 1 --2- 172.21.15.3:0/3133696160 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f925406be30 0x7f925406e2f0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:14.247 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:14.266+0000 7f924dffb640 1 --2- 172.21.15.3:0/3133696160 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9268147ec0 0x7f9268160990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:14.247 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:14.266+0000 7f924dffb640 1 --2- 172.21.15.3:0/3133696160 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9268162ae0 0x7f92680cf350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:14.247 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:14.266+0000 7f924dffb640 1 --2- 172.21.15.3:0/3133696160 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f92680cc2c0 0x7f9268141fe0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:14.247 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:14.266+0000 7f924dffb640 1 -- 172.21.15.3:0/3133696160 >> 172.21.15.3:0/3133696160 conn(0x7f926806bb30 msgr2=0x7f926806d770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:14.247 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:14.266+0000 7f924dffb640 1 -- 172.21.15.3:0/3133696160 shutdown_connections 2024-05-09T06:36:14.247 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:14.266+0000 7f924dffb640 1 -- 172.21.15.3:0/3133696160 wait complete. 2024-05-09T06:36:14.264 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.2 2024-05-09T06:36:14.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:14.288+0000 7f7b523fb640 1 -- 172.21.15.3:0/533500004 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f7b4c0ade40 con 0x7f7b4c06b410 2024-05-09T06:36:14.270 INFO:teuthology.orchestra.run.smithi003.stdout:55834574854 2024-05-09T06:36:14.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:14.289+0000 7f7b48ff9640 1 -- 172.21.15.3:0/533500004 <== mon.2 v2:172.21.15.190:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f7b4407a430 con 0x7f7b4c06b410 2024-05-09T06:36:14.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:14.290+0000 7f24c331d640 1 -- 172.21.15.3:0/2144740818 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f249c005580 con 0x7f24bc070360 2024-05-09T06:36:14.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:14.290+0000 7f24b1ffb640 1 -- 172.21.15.3:0/2144740818 <== mon.2 v2:172.21.15.190:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f24b807e3b0 con 0x7f24bc070360 2024-05-09T06:36:14.272 INFO:teuthology.orchestra.run.smithi003.stdout:55834574853 2024-05-09T06:36:14.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:14.292+0000 7f7b523fb640 1 -- 172.21.15.3:0/533500004 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f7b3006be80 msgr2=0x7f7b3006e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:14.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:14.292+0000 7f7b523fb640 1 --2- 172.21.15.3:0/533500004 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f7b3006be80 0x7f7b3006e340 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f7b4c156580 tx=0x7f7b38009040 comp rx=0 tx=0).stop 2024-05-09T06:36:14.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:14.292+0000 7f7b523fb640 1 -- 172.21.15.3:0/533500004 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7b4c06b410 msgr2=0x7f7b4c1fc320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:14.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:14.292+0000 7f7b523fb640 1 --2- 172.21.15.3:0/533500004 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7b4c06b410 0x7f7b4c1fc320 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f7b44009e50 tx=0x7f7b44004a70 comp rx=0 tx=0).stop 2024-05-09T06:36:14.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:14.293+0000 7f7b523fb640 1 -- 172.21.15.3:0/533500004 shutdown_connections 2024-05-09T06:36:14.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:14.293+0000 7f7b523fb640 1 --2- 172.21.15.3:0/533500004 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f7b3006be80 0x7f7b3006e340 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:14.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:14.293+0000 7f7b523fb640 1 --2- 172.21.15.3:0/533500004 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7b4c0706b0 0x7f7b4c0b0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:14.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:14.293+0000 7f7b523fb640 1 --2- 172.21.15.3:0/533500004 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7b4c06e3f0 0x7f7b4c1f8490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:14.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:14.293+0000 7f7b523fb640 1 --2- 172.21.15.3:0/533500004 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7b4c06b410 0x7f7b4c1fc320 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:14.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:14.293+0000 7f7b523fb640 1 -- 172.21.15.3:0/533500004 >> 172.21.15.3:0/533500004 conn(0x7f7b4c06ce50 msgr2=0x7f7b4c15db40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:14.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:14.293+0000 7f7b523fb640 1 -- 172.21.15.3:0/533500004 shutdown_connections 2024-05-09T06:36:14.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:14.293+0000 7f7b523fb640 1 -- 172.21.15.3:0/533500004 wait complete. 2024-05-09T06:36:14.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:14.294+0000 7f24c331d640 1 -- 172.21.15.3:0/2144740818 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f24a006be30 msgr2=0x7f24a006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:14.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:14.294+0000 7f24c331d640 1 --2- 172.21.15.3:0/2144740818 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f24a006be30 0x7f24a006e2f0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f24bc0b80a0 tx=0x7f24ac007a40 comp rx=0 tx=0).stop 2024-05-09T06:36:14.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:14.294+0000 7f24c331d640 1 -- 172.21.15.3:0/2144740818 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f24bc070360 msgr2=0x7f24bc158390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:14.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:14.294+0000 7f24c331d640 1 --2- 172.21.15.3:0/2144740818 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f24bc070360 0x7f24bc158390 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f24b8035000 tx=0x7f24b80046c0 comp rx=0 tx=0).stop 2024-05-09T06:36:14.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:14.294+0000 7f24c331d640 1 -- 172.21.15.3:0/2144740818 shutdown_connections 2024-05-09T06:36:14.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:14.294+0000 7f24c331d640 1 --2- 172.21.15.3:0/2144740818 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f24a006be30 0x7f24a006e2f0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:14.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:14.294+0000 7f24c331d640 1 --2- 172.21.15.3:0/2144740818 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f24bc1603e0 0x7f24bc0c9790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:14.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:14.294+0000 7f24c331d640 1 --2- 172.21.15.3:0/2144740818 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f24bc160000 0x7f24bc156080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:14.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:14.294+0000 7f24c331d640 1 --2- 172.21.15.3:0/2144740818 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f24bc070360 0x7f24bc158390 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:14.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:14.294+0000 7f24c331d640 1 -- 172.21.15.3:0/2144740818 >> 172.21.15.3:0/2144740818 conn(0x7f24bc0b3ae0 msgr2=0x7f24bc06d370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:14.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:14.294+0000 7f24c331d640 1 -- 172.21.15.3:0/2144740818 shutdown_connections 2024-05-09T06:36:14.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:14.294+0000 7f24c331d640 1 -- 172.21.15.3:0/2144740818 wait complete. 2024-05-09T06:36:14.281 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:14.300+0000 7f910a87b640 1 -- 172.21.15.3:0/1774738169 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f9104152ba0 con 0x7f91040d2000 2024-05-09T06:36:14.281 INFO:teuthology.orchestra.run.smithi003.stdout:55834574853 2024-05-09T06:36:14.282 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:14.300+0000 7f9100ff9640 1 -- 172.21.15.3:0/1774738169 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f90f40ab030 con 0x7f91040d2000 2024-05-09T06:36:14.285 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:14.304+0000 7f910a87b640 1 -- 172.21.15.3:0/1774738169 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f90f006be30 msgr2=0x7f90f006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:14.285 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:14.304+0000 7f910a87b640 1 --2- 172.21.15.3:0/1774738169 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f90f006be30 0x7f90f006e2f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f9104152b70 tx=0x7f90f8007d30 comp rx=0 tx=0).stop 2024-05-09T06:36:14.285 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:14.304+0000 7f910a87b640 1 -- 172.21.15.3:0/1774738169 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f91040d2000 msgr2=0x7f9104152630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:14.285 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:14.304+0000 7f910a87b640 1 --2- 172.21.15.3:0/1774738169 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f91040d2000 0x7f9104152630 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f90f4000c00 tx=0x7f90f40046c0 comp rx=0 tx=0).stop 2024-05-09T06:36:14.285 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:14.304+0000 7f910a87b640 1 -- 172.21.15.3:0/1774738169 shutdown_connections 2024-05-09T06:36:14.285 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:14.304+0000 7f910a87b640 1 --2- 172.21.15.3:0/1774738169 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f90f006be30 0x7f90f006e2f0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:14.285 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:14.304+0000 7f910a87b640 1 --2- 172.21.15.3:0/1774738169 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f910415e6f0 0x7f91040d0c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:14.285 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:14.304+0000 7f910a87b640 1 --2- 172.21.15.3:0/1774738169 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f91040d2df0 0x7f91040c9180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:14.285 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:14.304+0000 7f910a87b640 1 --2- 172.21.15.3:0/1774738169 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f91040d2000 0x7f9104152630 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:14.286 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:14.304+0000 7f910a87b640 1 -- 172.21.15.3:0/1774738169 >> 172.21.15.3:0/1774738169 conn(0x7f910409d500 msgr2=0x7f91040b2870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:14.286 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:14.305+0000 7fc0b51a2640 1 -- 172.21.15.3:0/1844828524 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7fc090005850 con 0x7fc0b0144910 2024-05-09T06:36:14.286 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:14.304+0000 7f910a87b640 1 -- 172.21.15.3:0/1774738169 shutdown_connections 2024-05-09T06:36:14.286 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:14.305+0000 7f910a87b640 1 -- 172.21.15.3:0/1774738169 wait complete. 2024-05-09T06:36:14.287 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:14.305+0000 7fc0977fe640 1 -- 172.21.15.3:0/1844828524 <== mon.2 v2:172.21.15.190:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fc0a80592e0 con 0x7fc0b0144910 2024-05-09T06:36:14.287 INFO:teuthology.orchestra.run.smithi003.stdout:55834574854 2024-05-09T06:36:14.287 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:14.306+0000 7febd7a7d640 1 -- 172.21.15.3:0/3506697645 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7febb8005850 con 0x7febd00cc650 2024-05-09T06:36:14.288 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:14.307+0000 7febc27fc640 1 -- 172.21.15.3:0/3506697645 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7febc405d530 con 0x7febd00cc650 2024-05-09T06:36:14.288 INFO:teuthology.orchestra.run.smithi003.stdout:55834574854 2024-05-09T06:36:14.290 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:14.308+0000 7fc0b51a2640 1 -- 172.21.15.3:0/1844828524 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fc09c06e090 msgr2=0x7fc09c070550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:14.290 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:14.308+0000 7fc0b51a2640 1 --2- 172.21.15.3:0/1844828524 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fc09c06e090 0x7fc09c070550 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fc0a002a170 tx=0x7fc0a002e000 comp rx=0 tx=0).stop 2024-05-09T06:36:14.290 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:14.309+0000 7fc0b51a2640 1 -- 172.21.15.3:0/1844828524 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc0b0144910 msgr2=0x7fc0b01594c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:14.290 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:14.309+0000 7fc0b51a2640 1 --2- 172.21.15.3:0/1844828524 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc0b0144910 0x7fc0b01594c0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fc0a800cb20 tx=0x7fc0a800cf20 comp rx=0 tx=0).stop 2024-05-09T06:36:14.290 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:14.309+0000 7fc0b51a2640 1 -- 172.21.15.3:0/1844828524 shutdown_connections 2024-05-09T06:36:14.290 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:14.309+0000 7fc0b51a2640 1 --2- 172.21.15.3:0/1844828524 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fc09c06e090 0x7fc09c070550 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:14.290 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:14.309+0000 7fc0b51a2640 1 --2- 172.21.15.3:0/1844828524 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc0b0144910 0x7fc0b01594c0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:14.290 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:14.309+0000 7fc0b51a2640 1 --2- 172.21.15.3:0/1844828524 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc0b00b4130 0x7fc0b015b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:14.290 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:14.309+0000 7fc0b51a2640 1 --2- 172.21.15.3:0/1844828524 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc0b0072970 0x7fc0b013e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:14.290 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:14.309+0000 7fc0b51a2640 1 -- 172.21.15.3:0/1844828524 >> 172.21.15.3:0/1844828524 conn(0x7fc0b00b1c20 msgr2=0x7fc0b006e9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:14.290 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:14.309+0000 7fc0b51a2640 1 -- 172.21.15.3:0/1844828524 shutdown_connections 2024-05-09T06:36:14.290 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:14.309+0000 7fc0b51a2640 1 -- 172.21.15.3:0/1844828524 wait complete. 2024-05-09T06:36:14.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:14.310+0000 7febd7a7d640 1 -- 172.21.15.3:0/3506697645 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7febb406be30 msgr2=0x7febb406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:14.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:14.310+0000 7febd7a7d640 1 --2- 172.21.15.3:0/3506697645 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7febb406be30 0x7febb406e2f0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7febcc009560 tx=0x7febcc02e000 comp rx=0 tx=0).stop 2024-05-09T06:36:14.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:14.311+0000 7febd7a7d640 1 -- 172.21.15.3:0/3506697645 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7febd00cc650 msgr2=0x7febd01449f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:14.292 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.6 2024-05-09T06:36:14.293 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:14.311+0000 7febd7a7d640 1 --2- 172.21.15.3:0/3506697645 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7febd00cc650 0x7febd01449f0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7febc400c7c0 tx=0x7febc400cc80 comp rx=0 tx=0).stop 2024-05-09T06:36:14.294 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:14.311+0000 7febd7a7d640 1 -- 172.21.15.3:0/3506697645 shutdown_connections 2024-05-09T06:36:14.294 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:14.311+0000 7febd7a7d640 1 --2- 172.21.15.3:0/3506697645 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7febb406be30 0x7febb406e2f0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:14.294 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:14.311+0000 7febd7a7d640 1 --2- 172.21.15.3:0/3506697645 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7febd01639a0 0x7febd014b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:14.294 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:14.311+0000 7febd7a7d640 1 --2- 172.21.15.3:0/3506697645 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7febd00cc650 0x7febd01449f0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:14.294 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:14.311+0000 7febd7a7d640 1 --2- 172.21.15.3:0/3506697645 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7febd00cc270 0x7febd0147a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:14.294 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:14.311+0000 7febd7a7d640 1 -- 172.21.15.3:0/3506697645 >> 172.21.15.3:0/3506697645 conn(0x7febd00b1c20 msgr2=0x7febd0145b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:14.294 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:14.311+0000 7febd7a7d640 1 -- 172.21.15.3:0/3506697645 shutdown_connections 2024-05-09T06:36:14.294 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:14.311+0000 7febd7a7d640 1 -- 172.21.15.3:0/3506697645 wait complete. 2024-05-09T06:36:14.295 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.4 2024-05-09T06:36:14.302 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:14.320+0000 7f597d968640 1 -- 172.21.15.3:0/1469160415 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f5958005580 con 0x7f59600051c0 2024-05-09T06:36:14.302 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:14.320+0000 7fe76759e640 1 -- 172.21.15.3:0/2097467231 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7fe744005850 con 0x7fe760006ef0 2024-05-09T06:36:14.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:14.321+0000 7f595ffff640 1 -- 172.21.15.3:0/1469160415 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f596c064c10 con 0x7f59600051c0 2024-05-09T06:36:14.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:14.321+0000 7fe75effd640 1 -- 172.21.15.3:0/2097467231 <== mon.2 v2:172.21.15.190:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fe75807e1f0 con 0x7fe760006ef0 2024-05-09T06:36:14.304 INFO:teuthology.orchestra.run.smithi003.stdout:55834574853 2024-05-09T06:36:14.304 INFO:teuthology.orchestra.run.smithi003.stdout:55834574854 2024-05-09T06:36:14.305 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.7 2024-05-09T06:36:14.305 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:14.324+0000 7f597d968640 1 -- 172.21.15.3:0/1469160415 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f596406c030 msgr2=0x7f596406e4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:14.305 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:14.324+0000 7f597d968640 1 --2- 172.21.15.3:0/1469160415 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f596406c030 0x7f596406e4f0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f59781f2f50 tx=0x7f5968013040 comp rx=0 tx=0).stop 2024-05-09T06:36:14.305 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:14.324+0000 7f597d968640 1 -- 172.21.15.3:0/1469160415 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f59600051c0 msgr2=0x7f59781567f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:14.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:14.324+0000 7f597d968640 1 --2- 172.21.15.3:0/1469160415 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f59600051c0 0x7f59781567f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f596c015c80 tx=0x7f596c0134b0 comp rx=0 tx=0).stop 2024-05-09T06:36:14.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:14.324+0000 7fe76759e640 1 -- 172.21.15.3:0/2097467231 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fe75006be80 msgr2=0x7fe75006e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:14.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:14.324+0000 7fe76759e640 1 --2- 172.21.15.3:0/2097467231 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fe75006be80 0x7fe75006e340 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fe754007b10 tx=0x7fe754007a60 comp rx=0 tx=0).stop 2024-05-09T06:36:14.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:14.324+0000 7fe76759e640 1 -- 172.21.15.3:0/2097467231 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe760006ef0 msgr2=0x7fe7601804e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:14.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:14.324+0000 7fe76759e640 1 --2- 172.21.15.3:0/2097467231 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe760006ef0 0x7fe7601804e0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fe758031000 tx=0x7fe7580046c0 comp rx=0 tx=0).stop 2024-05-09T06:36:14.307 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.1 2024-05-09T06:36:14.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:14.324+0000 7f597d968640 1 -- 172.21.15.3:0/1469160415 shutdown_connections 2024-05-09T06:36:14.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:14.324+0000 7f597d968640 1 --2- 172.21.15.3:0/1469160415 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f596406c030 0x7f596406e4f0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:14.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:14.324+0000 7f597d968640 1 --2- 172.21.15.3:0/1469160415 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5978155270 0x7f59780ab620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:14.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:14.324+0000 7f597d968640 1 --2- 172.21.15.3:0/1469160415 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5978149a00 0x7f59780ab0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:14.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:14.324+0000 7f597d968640 1 --2- 172.21.15.3:0/1469160415 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f59600051c0 0x7f59781567f0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:14.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:14.324+0000 7f597d968640 1 -- 172.21.15.3:0/1469160415 >> 172.21.15.3:0/1469160415 conn(0x7f59780b1c20 msgr2=0x7f59780b2000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:14.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:14.324+0000 7fe76759e640 1 -- 172.21.15.3:0/2097467231 shutdown_connections 2024-05-09T06:36:14.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:14.324+0000 7fe76759e640 1 --2- 172.21.15.3:0/2097467231 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fe75006be80 0x7fe75006e340 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:14.309 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:14.324+0000 7fe76759e640 1 --2- 172.21.15.3:0/2097467231 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe7600ead00 0x7fe760046df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:14.309 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:14.324+0000 7fe76759e640 1 --2- 172.21.15.3:0/2097467231 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe76004ffd0 0x7fe7600468b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:14.309 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:14.324+0000 7fe76759e640 1 --2- 172.21.15.3:0/2097467231 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe760006ef0 0x7fe7601804e0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:14.309 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:14.324+0000 7fe76759e640 1 -- 172.21.15.3:0/2097467231 >> 172.21.15.3:0/2097467231 conn(0x7fe760009fb0 msgr2=0x7fe76003a3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:14.309 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:14.324+0000 7fe76759e640 1 -- 172.21.15.3:0/2097467231 shutdown_connections 2024-05-09T06:36:14.309 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:14.325+0000 7f597d968640 1 -- 172.21.15.3:0/1469160415 shutdown_connections 2024-05-09T06:36:14.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:14.325+0000 7fe76759e640 1 -- 172.21.15.3:0/2097467231 wait complete. 2024-05-09T06:36:14.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:14.325+0000 7f597d968640 1 -- 172.21.15.3:0/1469160415 wait complete. 2024-05-09T06:36:14.311 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.0 2024-05-09T06:36:14.317 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.3 2024-05-09T06:36:14.318 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.5 2024-05-09T06:36:15.265 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-05-09T06:36:15.293 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-05-09T06:36:15.296 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-05-09T06:36:15.306 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-05-09T06:36:15.308 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-05-09T06:36:15.312 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-05-09T06:36:15.318 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-05-09T06:36:15.318 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-05-09T06:36:15.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.447+0000 7fed6e8f5640 1 Processor -- start 2024-05-09T06:36:15.432 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.447+0000 7fed6e8f5640 1 -- start start 2024-05-09T06:36:15.432 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.449+0000 7fed6e8f5640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fed6816adb0 0x7fed6808b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:15.432 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.449+0000 7fed6e8f5640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fed6816b190 0x7fed6808b800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:15.432 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.449+0000 7fed6e8f5640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fed6806f380 con 0x7fed680b4030 2024-05-09T06:36:15.432 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.449+0000 7fed6e8f5640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fed6808be10 con 0x7fed6816b190 2024-05-09T06:36:15.432 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.449+0000 7fed6e8f5640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fed6816fa10 con 0x7fed6816adb0 2024-05-09T06:36:15.432 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.449+0000 7fed6ce92640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fed680b4030 0x7fed68071080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:49258/0 (socket says 172.21.15.3:49258) 2024-05-09T06:36:15.432 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.449+0000 7fed6ce92640 1 -- 172.21.15.3:0/524822842 learned_addr learned my addr 172.21.15.3:0/524822842 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-05-09T06:36:15.432 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.449+0000 7fed677fe640 1 --2- 172.21.15.3:0/524822842 >> v2:172.21.15.3:3300/0 conn(0x7fed6816b190 0x7fed6808b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:15.432 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.450+0000 7fed67fff640 1 --2- 172.21.15.3:0/524822842 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fed6816adb0 0x7fed6808b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:15.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.450+0000 7fed66ffd640 1 -- 172.21.15.3:0/524822842 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2602430711 0 0) 0x7fed6806f380 con 0x7fed680b4030 2024-05-09T06:36:15.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.450+0000 7fed66ffd640 1 -- 172.21.15.3:0/524822842 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fed50007930 con 0x7fed680b4030 2024-05-09T06:36:15.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.451+0000 7fed67fff640 1 -- 172.21.15.3:0/524822842 >> v2:172.21.15.3:3300/0 conn(0x7fed6816b190 msgr2=0x7fed6808b800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:15.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.451+0000 7fed67fff640 1 --2- 172.21.15.3:0/524822842 >> v2:172.21.15.3:3300/0 conn(0x7fed6816b190 0x7fed6808b800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:15.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.451+0000 7fed67fff640 1 -- 172.21.15.3:0/524822842 >> v1:172.21.15.3:6789/0 conn(0x7fed680b4030 legacy=0x7fed68071080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:15.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.451+0000 7fed67fff640 1 -- 172.21.15.3:0/524822842 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed6816ec50 con 0x7fed6816adb0 2024-05-09T06:36:15.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.453+0000 7fed67fff640 1 --2- 172.21.15.3:0/524822842 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fed6816adb0 0x7fed6808b2c0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fed6808bf90 tx=0x7fed5c032ac0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=42a789f98009d997 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:15.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.453+0000 7fed66ffd640 1 -- 172.21.15.3:0/524822842 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed5c034820 con 0x7fed6816adb0 2024-05-09T06:36:15.436 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.453+0000 7fed66ffd640 1 -- 172.21.15.3:0/524822842 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed5c0349c0 con 0x7fed6816adb0 2024-05-09T06:36:15.436 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.453+0000 7fed66ffd640 1 -- 172.21.15.3:0/524822842 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed5c034db0 con 0x7fed6816adb0 2024-05-09T06:36:15.436 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.454+0000 7fed6e8f5640 1 -- 172.21.15.3:0/524822842 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fed6816adb0 msgr2=0x7fed6808b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:15.436 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.454+0000 7fed6e8f5640 1 --2- 172.21.15.3:0/524822842 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fed6816adb0 0x7fed6808b2c0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fed6808bf90 tx=0x7fed5c032ac0 comp rx=0 tx=0).stop 2024-05-09T06:36:15.436 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.454+0000 7fed6e8f5640 1 -- 172.21.15.3:0/524822842 shutdown_connections 2024-05-09T06:36:15.436 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.454+0000 7fed6e8f5640 1 --2- 172.21.15.3:0/524822842 >> v2:172.21.15.3:3300/0 conn(0x7fed6816b190 0x7fed6808b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:15.436 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.455+0000 7fed6e8f5640 1 --2- 172.21.15.3:0/524822842 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fed6816adb0 0x7fed6808b2c0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:15.436 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.455+0000 7fed6e8f5640 1 -- 172.21.15.3:0/524822842 >> 172.21.15.3:0/524822842 conn(0x7fed680b1c20 msgr2=0x7fed680b2000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:15.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.455+0000 7fed6e8f5640 1 -- 172.21.15.3:0/524822842 shutdown_connections 2024-05-09T06:36:15.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.455+0000 7fed6e8f5640 1 -- 172.21.15.3:0/524822842 wait complete. 2024-05-09T06:36:15.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.457+0000 7fed6e8f5640 1 Processor -- start 2024-05-09T06:36:15.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.457+0000 7fed6e8f5640 1 -- start start 2024-05-09T06:36:15.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.458+0000 7fed6e8f5640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fed680b4030 0x7fed680b2f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:15.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.459+0000 7fed6e8f5640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fed6816adb0 0x7fed681f5960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:15.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.459+0000 7fed677fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fed6816adb0 0x7fed681f5960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:15.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.459+0000 7fed677fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fed6816adb0 0x7fed681f5960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49674/0 (socket says 172.21.15.3:49674) 2024-05-09T06:36:15.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.459+0000 7fed67fff640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fed680b4030 0x7fed680b2f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:15.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.459+0000 7fed677fe640 1 -- 172.21.15.3:0/2567913175 learned_addr learned my addr 172.21.15.3:0/2567913175 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:36:15.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.459+0000 7fed6e8f5640 1 --2- 172.21.15.3:0/2567913175 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fed6816b190 0x7fed681f5ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:15.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.459+0000 7fed6e8f5640 1 -- 172.21.15.3:0/2567913175 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fed6816f4f0 con 0x7fed6816adb0 2024-05-09T06:36:15.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.459+0000 7fed6e8f5640 1 -- 172.21.15.3:0/2567913175 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fed6816f210 con 0x7fed6816b190 2024-05-09T06:36:15.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.459+0000 7fed6e8f5640 1 -- 172.21.15.3:0/2567913175 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fed6806e090 con 0x7fed680b4030 2024-05-09T06:36:15.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.460+0000 7fed6ce92640 1 --2- 172.21.15.3:0/2567913175 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fed6816b190 0x7fed681f5ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:15.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.460+0000 7fed677fe640 1 -- 172.21.15.3:0/2567913175 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fed680b4030 msgr2=0x7fed680b2f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:15.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.460+0000 7fed677fe640 1 --2- 172.21.15.3:0/2567913175 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fed680b4030 0x7fed680b2f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:15.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.460+0000 7fed677fe640 1 -- 172.21.15.3:0/2567913175 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fed6816b190 msgr2=0x7fed681f5ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:15.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.460+0000 7fed677fe640 1 --2- 172.21.15.3:0/2567913175 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fed6816b190 0x7fed681f5ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:15.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.460+0000 7fed677fe640 1 -- 172.21.15.3:0/2567913175 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed68168080 con 0x7fed6816adb0 2024-05-09T06:36:15.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.460+0000 7fed677fe640 1 --2- 172.21.15.3:0/2567913175 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fed6816adb0 0x7fed681f5960 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fed58007b00 tx=0x7fed5800ac80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:15.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.461+0000 7fed67fff640 1 --2- 172.21.15.3:0/2567913175 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fed680b4030 0x7fed680b2f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:36:15.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.462+0000 7fed64ff9640 1 -- 172.21.15.3:0/2567913175 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed58004230 con 0x7fed6816adb0 2024-05-09T06:36:15.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.462+0000 7fed64ff9640 1 -- 172.21.15.3:0/2567913175 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed58018070 con 0x7fed6816adb0 2024-05-09T06:36:15.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.462+0000 7fed6e8f5640 1 -- 172.21.15.3:0/2567913175 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed68168340 con 0x7fed6816adb0 2024-05-09T06:36:15.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.464+0000 7fed6e8f5640 1 -- 172.21.15.3:0/2567913175 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed68168900 con 0x7fed6816adb0 2024-05-09T06:36:15.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.467+0000 7fed64ff9640 1 -- 172.21.15.3:0/2567913175 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed5800d760 con 0x7fed6816adb0 2024-05-09T06:36:15.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.467+0000 7fed64ff9640 1 -- 172.21.15.3:0/2567913175 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fed58012070 con 0x7fed6816adb0 2024-05-09T06:36:15.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.467+0000 7fed64ff9640 1 --2- 172.21.15.3:0/2567913175 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fed4c06bf60 0x7fed4c06e420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:15.450 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.468+0000 7fed64ff9640 1 -- 172.21.15.3:0/2567913175 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fed5808f060 con 0x7fed6816adb0 2024-05-09T06:36:15.450 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.468+0000 7fed67fff640 1 --2- 172.21.15.3:0/2567913175 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fed4c06bf60 0x7fed4c06e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:15.450 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.469+0000 7fed567fc640 1 -- 172.21.15.3:0/2567913175 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed44005290 con 0x7fed6816adb0 2024-05-09T06:36:15.460 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.479+0000 7fed64ff9640 1 -- 172.21.15.3:0/2567913175 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fed58058db0 con 0x7fed6816adb0 2024-05-09T06:36:15.460 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.479+0000 7fed67fff640 1 --2- 172.21.15.3:0/2567913175 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fed4c06bf60 0x7fed4c06e420 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fed5c03d000 tx=0x7fed5c038000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:15.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.501+0000 7f8b35763640 1 Processor -- start 2024-05-09T06:36:15.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.501+0000 7f8b35763640 1 -- start start 2024-05-09T06:36:15.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.502+0000 7f8b35763640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f8b280df100 0x7f8b280e0100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:15.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.502+0000 7f8b35763640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f8b280530a0 0x7f8b2804fa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:15.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.502+0000 7f8b35763640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8b280503c0 con 0x7f8b2804ffb0 2024-05-09T06:36:15.486 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.499+0000 7fd7fa858640 1 Processor -- start 2024-05-09T06:36:15.486 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.499+0000 7fd7fa858640 1 -- start start 2024-05-09T06:36:15.486 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.500+0000 7fd7fa858640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fd7f415a4b0 0x7fd7f40afcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:15.486 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.500+0000 7fd7fa858640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd7f40af7a0 0x7fd7f40cd300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:15.486 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.501+0000 7fd7fa858640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fd7f40cd840 0x7fd7f406e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:15.486 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.501+0000 7fd7fa858640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fd7f40b3670 con 0x7fd7f415a4b0 2024-05-09T06:36:15.486 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.501+0000 7fd7fa858640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7fd7f40b38c0 con 0x7fd7f40cd840 2024-05-09T06:36:15.486 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.501+0000 7fd7fa858640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fd7f406df20 con 0x7fd7f40af7a0 2024-05-09T06:36:15.486 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.501+0000 7fd7f9055640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd7f40af7a0 0x7fd7f40cd300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:15.486 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.501+0000 7fd7f9055640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd7f40af7a0 0x7fd7f40cd300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.3:45218/0 (socket says 172.21.15.3:45218) 2024-05-09T06:36:15.487 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.501+0000 7fd7f9055640 1 -- 172.21.15.3:0/1323262832 learned_addr learned my addr 172.21.15.3:0/1323262832 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:36:15.487 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.501+0000 7fd7fa057640 1 --2- 172.21.15.3:0/1323262832 >> v2:172.21.15.190:3300/0 conn(0x7fd7f40cd840 0x7fd7f406e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:15.487 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.501+0000 7fd7f9856640 1 --2- 172.21.15.3:0/1323262832 >> v2:172.21.15.3:3300/0 conn(0x7fd7f415a4b0 0x7fd7f40afcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:15.487 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.501+0000 7fd7f9055640 1 -- 172.21.15.3:0/1323262832 >> v2:172.21.15.190:3300/0 conn(0x7fd7f40cd840 msgr2=0x7fd7f406e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:15.487 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.502+0000 7f8b35763640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f8b280df4e0 con 0x7f8b280df100 2024-05-09T06:36:15.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.502+0000 7f8b35763640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f8b280535f0 con 0x7f8b280530a0 2024-05-09T06:36:15.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.502+0000 7f8b2e7fc640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f8b280530a0 0x7f8b2804fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:15.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.503+0000 7f8b2effd640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f8b280df100 0x7f8b280e0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:15.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.503+0000 7f8b2effd640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f8b280df100 0x7f8b280e0100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49690/0 (socket says 172.21.15.3:49690) 2024-05-09T06:36:15.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.503+0000 7f8b2effd640 1 -- 172.21.15.3:0/1111817909 learned_addr learned my addr 172.21.15.3:0/1111817909 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:36:15.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.503+0000 7f8b2e7fc640 1 -- 172.21.15.3:0/1111817909 >> v2:172.21.15.3:3300/0 conn(0x7f8b280df100 msgr2=0x7f8b280e0100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:15.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.503+0000 7f8b2e7fc640 1 --2- 172.21.15.3:0/1111817909 >> v2:172.21.15.3:3300/0 conn(0x7f8b280df100 0x7f8b280e0100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:15.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.503+0000 7f8b2e7fc640 1 -- 172.21.15.3:0/1111817909 >> v1:172.21.15.190:6789/0 conn(0x7f8b2804ffb0 legacy=0x7f8b280e2c70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T06:36:15.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.503+0000 7f8b2e7fc640 1 -- 172.21.15.3:0/1111817909 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b280538d0 con 0x7f8b280530a0 2024-05-09T06:36:15.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.504+0000 7f8b2e7fc640 1 --2- 172.21.15.3:0/1111817909 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8b280530a0 0x7f8b2804fa70 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f8b3004fb10 tx=0x7f8b300a9f50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8202c584ffe27859 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:15.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.501+0000 7fd7f9055640 1 --2- 172.21.15.3:0/1323262832 >> v2:172.21.15.190:3300/0 conn(0x7fd7f40cd840 0x7fd7f406e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:15.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.501+0000 7fd7f9055640 1 -- 172.21.15.3:0/1323262832 >> v2:172.21.15.3:3300/0 conn(0x7fd7f415a4b0 msgr2=0x7fd7f40afcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:15.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.501+0000 7fd7f9055640 1 --2- 172.21.15.3:0/1323262832 >> v2:172.21.15.3:3300/0 conn(0x7fd7f415a4b0 0x7fd7f40afcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:15.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.502+0000 7fd7f9055640 1 -- 172.21.15.3:0/1323262832 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7f406e200 con 0x7fd7f40af7a0 2024-05-09T06:36:15.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.503+0000 7fd7f9856640 1 --2- 172.21.15.3:0/1323262832 >> v2:172.21.15.3:3300/0 conn(0x7fd7f415a4b0 0x7fd7f40afcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:36:15.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.503+0000 7fd7f9055640 1 --2- 172.21.15.3:0/1323262832 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd7f40af7a0 0x7fd7f40cd300 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fd7f406f8e0 tx=0x7fd7f004c210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=10e86fc2d4616a5d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:15.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.504+0000 7fd7f8854640 1 -- 172.21.15.3:0/1323262832 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7f0059070 con 0x7fd7f40af7a0 2024-05-09T06:36:15.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.504+0000 7fd7f8854640 1 -- 172.21.15.3:0/1323262832 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd7f0054440 con 0x7fd7f40af7a0 2024-05-09T06:36:15.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.505+0000 7fd7fa858640 1 -- 172.21.15.3:0/1323262832 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd7f40af7a0 msgr2=0x7fd7f40cd300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:15.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.505+0000 7fd7fa858640 1 --2- 172.21.15.3:0/1323262832 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd7f40af7a0 0x7fd7f40cd300 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fd7f406f8e0 tx=0x7fd7f004c210 comp rx=0 tx=0).stop 2024-05-09T06:36:15.490 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.505+0000 7f8b2dffb640 1 -- 172.21.15.3:0/1111817909 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b300b7070 con 0x7f8b280530a0 2024-05-09T06:36:15.490 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.505+0000 7fd7fa858640 1 -- 172.21.15.3:0/1323262832 shutdown_connections 2024-05-09T06:36:15.491 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.505+0000 7fd7fa858640 1 --2- 172.21.15.3:0/1323262832 >> v2:172.21.15.190:3300/0 conn(0x7fd7f40cd840 0x7fd7f406e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:15.491 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.505+0000 7fd7fa858640 1 --2- 172.21.15.3:0/1323262832 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd7f40af7a0 0x7fd7f40cd300 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:15.491 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.506+0000 7f8b2dffb640 1 -- 172.21.15.3:0/1111817909 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b300aad60 con 0x7f8b280530a0 2024-05-09T06:36:15.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.505+0000 7fd7fa858640 1 --2- 172.21.15.3:0/1323262832 >> v2:172.21.15.3:3300/0 conn(0x7fd7f415a4b0 0x7fd7f40afcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:15.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.505+0000 7fd7fa858640 1 -- 172.21.15.3:0/1323262832 >> 172.21.15.3:0/1323262832 conn(0x7fd7f40b24e0 msgr2=0x7fd7f40b28f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:15.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.506+0000 7fd7fa858640 1 -- 172.21.15.3:0/1323262832 shutdown_connections 2024-05-09T06:36:15.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.506+0000 7fd7fa858640 1 -- 172.21.15.3:0/1323262832 wait complete. 2024-05-09T06:36:15.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.506+0000 7f8b2dffb640 1 -- 172.21.15.3:0/1111817909 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b300b2640 con 0x7f8b280530a0 2024-05-09T06:36:15.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.506+0000 7f8b35763640 1 -- 172.21.15.3:0/1111817909 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8b280530a0 msgr2=0x7f8b2804fa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:15.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.506+0000 7f8b35763640 1 --2- 172.21.15.3:0/1111817909 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8b280530a0 0x7f8b2804fa70 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f8b3004fb10 tx=0x7f8b300a9f50 comp rx=0 tx=0).stop 2024-05-09T06:36:15.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.507+0000 7f8b35763640 1 -- 172.21.15.3:0/1111817909 shutdown_connections 2024-05-09T06:36:15.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.507+0000 7f8b35763640 1 --2- 172.21.15.3:0/1111817909 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8b280530a0 0x7f8b2804fa70 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:15.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.507+0000 7f8b35763640 1 --2- 172.21.15.3:0/1111817909 >> v2:172.21.15.3:3300/0 conn(0x7f8b280df100 0x7f8b280e0100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:15.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.507+0000 7f8b35763640 1 -- 172.21.15.3:0/1111817909 >> 172.21.15.3:0/1111817909 conn(0x7f8b28009fb0 msgr2=0x7f8b2800a3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:15.494 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.507+0000 7fec5e4b2640 1 Processor -- start 2024-05-09T06:36:15.494 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.507+0000 7fec5e4b2640 1 -- start start 2024-05-09T06:36:15.494 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.508+0000 7fd7fa858640 1 Processor -- start 2024-05-09T06:36:15.494 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.507+0000 7f8b35763640 1 -- 172.21.15.3:0/1111817909 shutdown_connections 2024-05-09T06:36:15.494 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.508+0000 7f8b35763640 1 -- 172.21.15.3:0/1111817909 wait complete. 2024-05-09T06:36:15.495 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.508+0000 7fd7fa858640 1 -- start start 2024-05-09T06:36:15.495 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.508+0000 7fa3a081a640 1 Processor -- start 2024-05-09T06:36:15.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.508+0000 7fec5e4b2640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fec5813df10 0x7fec5815e970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:15.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.508+0000 7fec5e4b2640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fec5815eeb0 0x7fec580cf660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:15.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.508+0000 7fec5e4b2640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fec580d0450 con 0x7fec580cfba0 2024-05-09T06:36:15.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.508+0000 7fec5e4b2640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fec580d0cf0 con 0x7fec5813df10 2024-05-09T06:36:15.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.508+0000 7fec5e4b2640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fec5815f310 con 0x7fec5815eeb0 2024-05-09T06:36:15.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.509+0000 7fd7fa858640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd7f40af7a0 0x7fd7f415bd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:15.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.509+0000 7fec5dcb1640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fec580cfba0 0x7fec5815b350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:49268/0 (socket says 172.21.15.3:49268) 2024-05-09T06:36:15.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.509+0000 7fec5dcb1640 1 -- 172.21.15.3:0/1518313396 learned_addr learned my addr 172.21.15.3:0/1518313396 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-05-09T06:36:15.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.509+0000 7fec5d4b0640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fec5813df10 0x7fec5815e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:15.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.509+0000 7fd7fa858640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd7f40cd840 0x7fd7f40a9920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:15.498 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.510+0000 7fd7fa858640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd7f415a4b0 0x7fd7f41398a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:15.498 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.508+0000 7fa3a081a640 1 -- start start 2024-05-09T06:36:15.498 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.510+0000 7fd7fa858640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fd7f415a890 con 0x7fd7f40af7a0 2024-05-09T06:36:15.498 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.510+0000 7f8b35763640 1 Processor -- start 2024-05-09T06:36:15.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.510+0000 7fd7f9055640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd7f40cd840 0x7fd7f40a9920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:15.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.510+0000 7fd7fa057640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd7f415a4b0 0x7fd7f41398a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:15.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.510+0000 7fd7fa057640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd7f415a4b0 0x7fd7f41398a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.3:47904/0 (socket says 172.21.15.3:47904) 2024-05-09T06:36:15.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.510+0000 7fd7fa057640 1 -- 172.21.15.3:0/2811719499 learned_addr learned my addr 172.21.15.3:0/2811719499 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:36:15.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.510+0000 7fa3a081a640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa3940e3300 0x7fa3940572f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:15.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.510+0000 7fa3a081a640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fa394053750 0x7fa394053c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:15.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.510+0000 7fa3a081a640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa3940e36e0 con 0x7fa394054140 2024-05-09T06:36:15.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.510+0000 7fa3a081a640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fa394057b40 con 0x7fa3940e3300 2024-05-09T06:36:15.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.510+0000 7fa3a081a640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7fa394057e20 con 0x7fa394053750 2024-05-09T06:36:15.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.510+0000 7fa39ad9d640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa3940e3300 0x7fa3940572f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:15.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.510+0000 7fa39ad9d640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa3940e3300 0x7fa3940572f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49720/0 (socket says 172.21.15.3:49720) 2024-05-09T06:36:15.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.510+0000 7fa39ad9d640 1 -- 172.21.15.3:0/245202545 learned_addr learned my addr 172.21.15.3:0/245202545 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:36:15.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.510+0000 7fa39a59c640 1 --2- 172.21.15.3:0/245202545 >> v2:172.21.15.190:3300/0 conn(0x7fa394053750 0x7fa394053c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:15.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.510+0000 7fd7fa858640 1 -- 172.21.15.3:0/2811719499 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fd7f40b3ba0 con 0x7fd7f415a4b0 2024-05-09T06:36:15.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.510+0000 7fec5d4b0640 1 -- 172.21.15.3:0/1518313396 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fec5815eeb0 msgr2=0x7fec580cf660 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T06:36:15.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.510+0000 7fec5d4b0640 1 --2- 172.21.15.3:0/1518313396 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fec5815eeb0 0x7fec580cf660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:15.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.510+0000 7fec4ffff640 1 -- 172.21.15.3:0/1518313396 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4211764320 0 0) 0x7fec580d0450 con 0x7fec580cfba0 2024-05-09T06:36:15.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.510+0000 7fec5d4b0640 1 -- 172.21.15.3:0/1518313396 >> v1:172.21.15.3:6789/0 conn(0x7fec580cfba0 legacy=0x7fec5815b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:15.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.511+0000 7fd7fa858640 1 -- 172.21.15.3:0/2811719499 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fd7f415aa10 con 0x7fd7f40cd840 2024-05-09T06:36:15.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.511+0000 7fec5d4b0640 1 -- 172.21.15.3:0/1518313396 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec580d0730 con 0x7fec5813df10 2024-05-09T06:36:15.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.511+0000 7fd7fa057640 1 -- 172.21.15.3:0/2811719499 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd7f40cd840 msgr2=0x7fd7f40a9920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:15.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.511+0000 7fd7fa057640 1 --2- 172.21.15.3:0/2811719499 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd7f40cd840 0x7fd7f40a9920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:15.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.511+0000 7fd7fa057640 1 -- 172.21.15.3:0/2811719499 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd7f40af7a0 msgr2=0x7fd7f415bd70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T06:36:15.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.511+0000 7fd7f9856640 1 --2- 172.21.15.3:0/2811719499 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd7f40af7a0 0x7fd7f415bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:15.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.511+0000 7fa39ad9d640 1 -- 172.21.15.3:0/245202545 >> v2:172.21.15.190:3300/0 conn(0x7fa394053750 msgr2=0x7fa394053c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:15.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.511+0000 7fd7fa057640 1 --2- 172.21.15.3:0/2811719499 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd7f40af7a0 0x7fd7f415bd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:15.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.511+0000 7fd7fa057640 1 -- 172.21.15.3:0/2811719499 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7f415c350 con 0x7fd7f415a4b0 2024-05-09T06:36:15.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.511+0000 7fa39ad9d640 1 --2- 172.21.15.3:0/245202545 >> v2:172.21.15.190:3300/0 conn(0x7fa394053750 0x7fa394053c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:15.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.512+0000 7fd7f9055640 1 --2- 172.21.15.3:0/2811719499 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd7f40cd840 0x7fd7f40a9920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:36:15.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.511+0000 7fa39ad9d640 1 -- 172.21.15.3:0/245202545 >> v1:172.21.15.3:6789/0 conn(0x7fa394054140 legacy=0x7fa3940061f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:15.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.512+0000 7fd7fa057640 1 --2- 172.21.15.3:0/2811719499 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd7f415a4b0 0x7fd7f41398a0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fd7ec00eed0 tx=0x7fd7ec00b480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:15.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.512+0000 7fd7f9856640 1 --2- 172.21.15.3:0/2811719499 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd7f40af7a0 0x7fd7f415bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:36:15.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.511+0000 7fa39ad9d640 1 -- 172.21.15.3:0/245202545 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa394058100 con 0x7fa3940e3300 2024-05-09T06:36:15.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.511+0000 7f8b35763640 1 -- start start 2024-05-09T06:36:15.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.512+0000 7fa39ad9d640 1 --2- 172.21.15.3:0/245202545 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa3940e3300 0x7fa3940572f0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fa39403bef0 tx=0x7fa38c02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b58b37db37d63f5c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:15.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.512+0000 7f8b35763640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8b2804ffb0 0x7f8b280d4300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:15.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.512+0000 7fec5d4b0640 1 --2- 172.21.15.3:0/1518313396 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fec5813df10 0x7fec5815e970 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fec50009dc0 tx=0x7fec5002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=532c6a4ec53e83c0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:15.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.512+0000 7f8b35763640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8b280530a0 0x7f8b280dccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:15.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.512+0000 7f8b35763640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8b280df100 0x7f8b280dd1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:15.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.512+0000 7fd7ea7fc640 1 -- 172.21.15.3:0/2811719499 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7ec018020 con 0x7fd7f415a4b0 2024-05-09T06:36:15.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.513+0000 7fec4ffff640 1 -- 172.21.15.3:0/1518313396 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec50007740 con 0x7fec5813df10 2024-05-09T06:36:15.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.513+0000 7f8b35763640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f8b28054170 con 0x7f8b280df100 2024-05-09T06:36:15.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.513+0000 7fec4ffff640 1 -- 172.21.15.3:0/1518313396 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec500078e0 con 0x7fec5813df10 2024-05-09T06:36:15.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.513+0000 7fd7fa858640 1 -- 172.21.15.3:0/2811719499 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7f415c690 con 0x7fd7f415a4b0 2024-05-09T06:36:15.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.513+0000 7fec4ffff640 1 -- 172.21.15.3:0/1518313396 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec50007740 con 0x7fec5813df10 2024-05-09T06:36:15.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.513+0000 7f8b2f7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8b280df100 0x7f8b280dd1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:15.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.513+0000 7f8b2f7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8b280df100 0x7f8b280dd1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49738/0 (socket says 172.21.15.3:49738) 2024-05-09T06:36:15.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.513+0000 7f8b2f7fe640 1 -- 172.21.15.3:0/3332261616 learned_addr learned my addr 172.21.15.3:0/3332261616 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:36:15.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.513+0000 7f8b2effd640 1 --2- 172.21.15.3:0/3332261616 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8b2804ffb0 0x7f8b280d4300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:15.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.513+0000 7f8b2e7fc640 1 --2- 172.21.15.3:0/3332261616 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8b280530a0 0x7f8b280dccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:15.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.513+0000 7f8b35763640 1 -- 172.21.15.3:0/3332261616 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f8b28053e90 con 0x7f8b2804ffb0 2024-05-09T06:36:15.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.513+0000 7f8b35763640 1 -- 172.21.15.3:0/3332261616 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f8b2800aad0 con 0x7f8b280530a0 2024-05-09T06:36:15.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.513+0000 7fd7fa858640 1 -- 172.21.15.3:0/2811719499 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7f415ced0 con 0x7fd7f415a4b0 2024-05-09T06:36:15.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.514+0000 7fd7ea7fc640 1 -- 172.21.15.3:0/2811719499 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd7ec014400 con 0x7fd7f415a4b0 2024-05-09T06:36:15.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.514+0000 7fd7ea7fc640 1 -- 172.21.15.3:0/2811719499 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7ec0147d0 con 0x7fd7f415a4b0 2024-05-09T06:36:15.512 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.514+0000 7fa399d9b640 1 -- 172.21.15.3:0/245202545 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa38c037070 con 0x7fa3940e3300 2024-05-09T06:36:15.512 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.514+0000 7fa399d9b640 1 -- 172.21.15.3:0/245202545 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa38c032440 con 0x7fa3940e3300 2024-05-09T06:36:15.512 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.514+0000 7fa399d9b640 1 -- 172.21.15.3:0/245202545 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa38c032810 con 0x7fa3940e3300 2024-05-09T06:36:15.512 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.514+0000 7f8b2f7fe640 1 -- 172.21.15.3:0/3332261616 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8b280530a0 msgr2=0x7f8b280dccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:15.512 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.514+0000 7f8b2f7fe640 1 --2- 172.21.15.3:0/3332261616 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8b280530a0 0x7f8b280dccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:15.512 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.514+0000 7f8b2f7fe640 1 -- 172.21.15.3:0/3332261616 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8b2804ffb0 msgr2=0x7f8b280d4300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:15.512 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.514+0000 7f8b2f7fe640 1 --2- 172.21.15.3:0/3332261616 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8b2804ffb0 0x7f8b280d4300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:15.513 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.514+0000 7f8b2f7fe640 1 -- 172.21.15.3:0/3332261616 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b28033ff0 con 0x7f8b280df100 2024-05-09T06:36:15.513 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.514+0000 7f8b2f7fe640 1 --2- 172.21.15.3:0/3332261616 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8b280df100 0x7f8b280dd1f0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f8b2400b420 tx=0x7f8b2400b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:15.513 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.514+0000 7f8b177fe640 1 -- 172.21.15.3:0/3332261616 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b24011830 con 0x7f8b280df100 2024-05-09T06:36:15.513 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.515+0000 7f8b177fe640 1 -- 172.21.15.3:0/3332261616 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b24016070 con 0x7f8b280df100 2024-05-09T06:36:15.513 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.515+0000 7f8b177fe640 1 -- 172.21.15.3:0/3332261616 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b24011c10 con 0x7f8b280df100 2024-05-09T06:36:15.513 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.515+0000 7fec5e4b2640 1 -- 172.21.15.3:0/1518313396 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fec5813df10 msgr2=0x7fec5815e970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:15.513 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.515+0000 7fec5e4b2640 1 --2- 172.21.15.3:0/1518313396 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fec5813df10 0x7fec5815e970 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fec50009dc0 tx=0x7fec5002a6a0 comp rx=0 tx=0).stop 2024-05-09T06:36:15.513 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.515+0000 7fd7fa858640 1 -- 172.21.15.3:0/2811719499 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7dc005290 con 0x7fd7f415a4b0 2024-05-09T06:36:15.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.515+0000 7f8b35763640 1 -- 172.21.15.3:0/3332261616 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b280342d0 con 0x7f8b280df100 2024-05-09T06:36:15.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.515+0000 7f8b35763640 1 -- 172.21.15.3:0/3332261616 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b28034890 con 0x7f8b280df100 2024-05-09T06:36:15.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.516+0000 7fec5e4b2640 1 -- 172.21.15.3:0/1518313396 shutdown_connections 2024-05-09T06:36:15.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.516+0000 7fec5e4b2640 1 --2- 172.21.15.3:0/1518313396 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fec5815eeb0 0x7fec580cf660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:15.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.516+0000 7fec5e4b2640 1 --2- 172.21.15.3:0/1518313396 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fec5813df10 0x7fec5815e970 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:15.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.516+0000 7fec5e4b2640 1 -- 172.21.15.3:0/1518313396 >> 172.21.15.3:0/1518313396 conn(0x7fec580b2c30 msgr2=0x7fec580b3010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:15.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.515+0000 7fa3a081a640 1 -- 172.21.15.3:0/245202545 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa3940e3300 msgr2=0x7fa3940572f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:15.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.515+0000 7fa3a081a640 1 --2- 172.21.15.3:0/245202545 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa3940e3300 0x7fa3940572f0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fa39403bef0 tx=0x7fa38c02a210 comp rx=0 tx=0).stop 2024-05-09T06:36:15.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.516+0000 7fa3a081a640 1 -- 172.21.15.3:0/245202545 shutdown_connections 2024-05-09T06:36:15.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.516+0000 7fa3a081a640 1 --2- 172.21.15.3:0/245202545 >> v2:172.21.15.190:3300/0 conn(0x7fa394053750 0x7fa394053c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:15.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.516+0000 7fa3a081a640 1 --2- 172.21.15.3:0/245202545 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa3940e3300 0x7fa3940572f0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:15.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.516+0000 7fa3a081a640 1 -- 172.21.15.3:0/245202545 >> 172.21.15.3:0/245202545 conn(0x7fa394058ef0 msgr2=0x7fa394059300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:15.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.516+0000 7fd7ea7fc640 1 -- 172.21.15.3:0/2811719499 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd7ec005d10 con 0x7fd7f415a4b0 2024-05-09T06:36:15.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.517+0000 7fd7ea7fc640 1 --2- 172.21.15.3:0/2811719499 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fd7d806bf00 0x7fd7d806e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:15.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.517+0000 7fec5e4b2640 1 -- 172.21.15.3:0/1518313396 shutdown_connections 2024-05-09T06:36:15.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.516+0000 7fa3a081a640 1 -- 172.21.15.3:0/245202545 shutdown_connections 2024-05-09T06:36:15.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.516+0000 7fa3a081a640 1 -- 172.21.15.3:0/245202545 wait complete. 2024-05-09T06:36:15.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.517+0000 7fec5e4b2640 1 -- 172.21.15.3:0/1518313396 wait complete. 2024-05-09T06:36:15.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.518+0000 7fd7ea7fc640 1 -- 172.21.15.3:0/2811719499 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fd7ec08b550 con 0x7fd7f415a4b0 2024-05-09T06:36:15.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.519+0000 7fa3a081a640 1 Processor -- start 2024-05-09T06:36:15.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.519+0000 7fa3a081a640 1 -- start start 2024-05-09T06:36:15.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.519+0000 7fd7f9856640 1 --2- 172.21.15.3:0/2811719499 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fd7d806bf00 0x7fd7d806e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:15.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.520+0000 7fec5e4b2640 1 Processor -- start 2024-05-09T06:36:15.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.520+0000 7fec5e4b2640 1 -- start start 2024-05-09T06:36:15.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.519+0000 7f8b35763640 1 -- 172.21.15.3:0/3332261616 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b280052c0 con 0x7f8b280df100 2024-05-09T06:36:15.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.519+0000 7f8b177fe640 1 -- 172.21.15.3:0/3332261616 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8b2401b460 con 0x7f8b280df100 2024-05-09T06:36:15.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.520+0000 7f8b177fe640 1 --2- 172.21.15.3:0/3332261616 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f8b1c06be30 0x7f8b1c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:15.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.520+0000 7f8b177fe640 1 -- 172.21.15.3:0/3332261616 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f8b2408aa20 con 0x7f8b280df100 2024-05-09T06:36:15.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.520+0000 7f8b2effd640 1 --2- 172.21.15.3:0/3332261616 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f8b1c06be30 0x7f8b1c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:15.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.521+0000 7f8b2effd640 1 --2- 172.21.15.3:0/3332261616 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f8b1c06be30 0x7f8b1c06e2f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f8b20009a10 tx=0x7f8b20008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:15.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.520+0000 7fa3a081a640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa394053750 0x7fa3940cb5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:15.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.520+0000 7fa39ad9d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa394053750 0x7fa3940cb5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:15.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.520+0000 7fa39ad9d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa394053750 0x7fa3940cb5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49744/0 (socket says 172.21.15.3:49744) 2024-05-09T06:36:15.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.520+0000 7fa39ad9d640 1 -- 172.21.15.3:0/3663840303 learned_addr learned my addr 172.21.15.3:0/3663840303 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:36:15.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.520+0000 7fa3a081a640 1 --2- 172.21.15.3:0/3663840303 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa394054140 0x7fa3940cbae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:15.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.521+0000 7fec5e4b2640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fec580cfba0 0x7fec58157590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:15.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.520+0000 7fa3a081a640 1 --2- 172.21.15.3:0/3663840303 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa3940e3300 0x7fa39404ee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:15.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.520+0000 7fa3a081a640 1 -- 172.21.15.3:0/3663840303 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fa394057830 con 0x7fa394053750 2024-05-09T06:36:15.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.520+0000 7fa3a081a640 1 -- 172.21.15.3:0/3663840303 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fa394054520 con 0x7fa394054140 2024-05-09T06:36:15.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.520+0000 7fa3a081a640 1 -- 172.21.15.3:0/3663840303 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fa39403d130 con 0x7fa3940e3300 2024-05-09T06:36:15.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.521+0000 7fa39a59c640 1 --2- 172.21.15.3:0/3663840303 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa394054140 0x7fa3940cbae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:15.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.521+0000 7fa39ad9d640 1 -- 172.21.15.3:0/3663840303 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa3940e3300 msgr2=0x7fa39404ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:15.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.521+0000 7fa39ad9d640 1 --2- 172.21.15.3:0/3663840303 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa3940e3300 0x7fa39404ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:15.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.521+0000 7fa39ad9d640 1 -- 172.21.15.3:0/3663840303 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa394054140 msgr2=0x7fa3940cbae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:15.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.521+0000 7fa39ad9d640 1 --2- 172.21.15.3:0/3663840303 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa394054140 0x7fa3940cbae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:15.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.521+0000 7fa39ad9d640 1 -- 172.21.15.3:0/3663840303 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa394051e60 con 0x7fa394053750 2024-05-09T06:36:15.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.521+0000 7fec5e4b2640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fec5813df10 0x7fec5812bff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:15.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.522+0000 7fec5e4b2640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fec5815eeb0 0x7fec5812c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:15.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.522+0000 7fd7ea7fc640 1 -- 172.21.15.3:0/2811719499 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fd7ec022070 con 0x7fd7f415a4b0 2024-05-09T06:36:15.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.522+0000 7fec5dcb1640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fec5815eeb0 0x7fec5812c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:15.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.522+0000 7fec5dcb1640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fec5815eeb0 0x7fec5812c530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49746/0 (socket says 172.21.15.3:49746) 2024-05-09T06:36:15.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.522+0000 7fec5dcb1640 1 -- 172.21.15.3:0/2575134751 learned_addr learned my addr 172.21.15.3:0/2575134751 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:36:15.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.522+0000 7fec5e4b2640 1 -- 172.21.15.3:0/2575134751 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fec580d0030 con 0x7fec5815eeb0 2024-05-09T06:36:15.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.522+0000 7fa39ad9d640 1 --2- 172.21.15.3:0/3663840303 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa394053750 0x7fa3940cb5a0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fa38c009960 tx=0x7fa38c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:15.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.522+0000 7fec5e4b2640 1 -- 172.21.15.3:0/2575134751 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fec580b4160 con 0x7fec5813df10 2024-05-09T06:36:15.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.522+0000 7fec5e4b2640 1 -- 172.21.15.3:0/2575134751 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fec580d01b0 con 0x7fec580cfba0 2024-05-09T06:36:15.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.522+0000 7fec5d4b0640 1 --2- 172.21.15.3:0/2575134751 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fec580cfba0 0x7fec58157590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:15.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.523+0000 7fec5ccaf640 1 --2- 172.21.15.3:0/2575134751 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fec5813df10 0x7fec5812bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:15.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.523+0000 7fec5dcb1640 1 -- 172.21.15.3:0/2575134751 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fec580cfba0 msgr2=0x7fec58157590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:15.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.523+0000 7fec5dcb1640 1 --2- 172.21.15.3:0/2575134751 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fec580cfba0 0x7fec58157590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:15.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.523+0000 7fec5dcb1640 1 -- 172.21.15.3:0/2575134751 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fec5813df10 msgr2=0x7fec5812bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:15.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.523+0000 7fec5dcb1640 1 --2- 172.21.15.3:0/2575134751 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fec5813df10 0x7fec5812bff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:15.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.523+0000 7fec5dcb1640 1 -- 172.21.15.3:0/2575134751 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec581e5f30 con 0x7fec5815eeb0 2024-05-09T06:36:15.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.522+0000 7fd7f9856640 1 --2- 172.21.15.3:0/2811719499 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fd7d806bf00 0x7fd7d806e3c0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fd7e4006fd0 tx=0x7fd7e4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:15.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.523+0000 7fec5dcb1640 1 --2- 172.21.15.3:0/2575134751 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fec5815eeb0 0x7fec5812c530 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fec48001eb0 tx=0x7fec4800c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:15.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.526+0000 7fec4dffb640 1 -- 172.21.15.3:0/2575134751 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec480074e0 con 0x7fec5815eeb0 2024-05-09T06:36:15.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.526+0000 7fec4dffb640 1 -- 172.21.15.3:0/2575134751 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec48018070 con 0x7fec5815eeb0 2024-05-09T06:36:15.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.526+0000 7fec4dffb640 1 -- 172.21.15.3:0/2575134751 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec480049e0 con 0x7fec5815eeb0 2024-05-09T06:36:15.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.526+0000 7f8b177fe640 1 -- 172.21.15.3:0/3332261616 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f8b24058d70 con 0x7f8b280df100 2024-05-09T06:36:15.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.526+0000 7fa3837fe640 1 -- 172.21.15.3:0/3663840303 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa38c040020 con 0x7fa394053750 2024-05-09T06:36:15.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.526+0000 7fa3a081a640 1 -- 172.21.15.3:0/3663840303 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa394052140 con 0x7fa394053750 2024-05-09T06:36:15.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.529+0000 7fa3837fe640 1 -- 172.21.15.3:0/3663840303 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa38c038040 con 0x7fa394053750 2024-05-09T06:36:15.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.530+0000 7fa3a081a640 1 -- 172.21.15.3:0/3663840303 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa394052700 con 0x7fa394053750 2024-05-09T06:36:15.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.530+0000 7fa3837fe640 1 -- 172.21.15.3:0/3663840303 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa38c037070 con 0x7fa394053750 2024-05-09T06:36:15.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.532+0000 7fa3837fe640 1 -- 172.21.15.3:0/3663840303 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa38c043020 con 0x7fa394053750 2024-05-09T06:36:15.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.533+0000 7fa3837fe640 1 --2- 172.21.15.3:0/3663840303 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fa38806bf00 0x7fa38806e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:15.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.533+0000 7fa3837fe640 1 -- 172.21.15.3:0/3663840303 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fa38c0ab090 con 0x7fa394053750 2024-05-09T06:36:15.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.535+0000 7fa3a081a640 1 -- 172.21.15.3:0/3663840303 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa37c005290 con 0x7fa394053750 2024-05-09T06:36:15.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.535+0000 7fa39a59c640 1 --2- 172.21.15.3:0/3663840303 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fa38806bf00 0x7fa38806e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:15.529 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.527+0000 7fec5e4b2640 1 -- 172.21.15.3:0/2575134751 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec581e6210 con 0x7fec5815eeb0 2024-05-09T06:36:15.529 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.527+0000 7fec5e4b2640 1 -- 172.21.15.3:0/2575134751 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec5813ad90 con 0x7fec5815eeb0 2024-05-09T06:36:15.529 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.529+0000 7fec3b7fe640 1 -- 172.21.15.3:0/2575134751 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec580682e0 con 0x7fec5815eeb0 2024-05-09T06:36:15.529 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.530+0000 7fec4dffb640 1 -- 172.21.15.3:0/2575134751 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fec48004b80 con 0x7fec5815eeb0 2024-05-09T06:36:15.529 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.531+0000 7fec4dffb640 1 --2- 172.21.15.3:0/2575134751 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fec3c06beb0 0x7fec3c06e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:15.529 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.535+0000 7fec4dffb640 1 -- 172.21.15.3:0/2575134751 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fec4808e690 con 0x7fec5815eeb0 2024-05-09T06:36:15.529 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.536+0000 7fec5d4b0640 1 --2- 172.21.15.3:0/2575134751 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fec3c06beb0 0x7fec3c06e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:15.530 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.536+0000 7fa39a59c640 1 --2- 172.21.15.3:0/3663840303 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fa38806bf00 0x7fa38806e3c0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fa39c000b60 tx=0x7fa39c09f040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:15.530 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.537+0000 7fec5d4b0640 1 --2- 172.21.15.3:0/2575134751 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fec3c06beb0 0x7fec3c06e370 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fec50034000 tx=0x7fec5002f000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:15.530 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.542+0000 7fec4dffb640 1 -- 172.21.15.3:0/2575134751 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fec4805d020 con 0x7fec5815eeb0 2024-05-09T06:36:15.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.552+0000 7fa3837fe640 1 -- 172.21.15.3:0/3663840303 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fa38c04a030 con 0x7fa394053750 2024-05-09T06:36:15.538 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.556+0000 7fe500d6a640 1 Processor -- start 2024-05-09T06:36:15.539 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.556+0000 7fefdfba2640 1 Processor -- start 2024-05-09T06:36:15.539 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.556+0000 7fe500d6a640 1 -- start start 2024-05-09T06:36:15.539 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.558+0000 7fe500d6a640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe4fc15e5c0 0x7fe4fc0d2800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:15.540 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.556+0000 7fefdfba2640 1 -- start start 2024-05-09T06:36:15.540 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.556+0000 7fdf6f29e640 1 Processor -- start 2024-05-09T06:36:15.540 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.556+0000 7fdf6f29e640 1 -- start start 2024-05-09T06:36:15.540 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.558+0000 7fdf6f29e640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdf6814be70 0x7fdf6806eea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:15.540 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.558+0000 7fdf6f29e640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fdf6814c250 0x7fdf6806d240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:15.540 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.558+0000 7fdf6f29e640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdf681664c0 con 0x7fdf6806d780 2024-05-09T06:36:15.540 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.558+0000 7fdf6f29e640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fdf6806f3e0 con 0x7fdf6814c250 2024-05-09T06:36:15.540 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.558+0000 7fdf6f29e640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fdf6806ea10 con 0x7fdf6814be70 2024-05-09T06:36:15.541 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.558+0000 7fefdfba2640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fefd80cc270 0x7fefd8166220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:15.541 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.558+0000 7fefdfba2640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fefd806e250 0x7fefd806e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:15.541 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.558+0000 7fefdfba2640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fefd8166a70 con 0x7fefd80cc650 2024-05-09T06:36:15.541 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.558+0000 7fefdfba2640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fefd8166d50 con 0x7fefd806e250 2024-05-09T06:36:15.541 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.558+0000 7fefdfba2640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fefd8167030 con 0x7fefd80cc270 2024-05-09T06:36:15.542 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.558+0000 7fe4fa59c640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe4fc15e5c0 0x7fe4fc0d2800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:15.542 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.558+0000 7fe4fa59c640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe4fc15e5c0 0x7fe4fc0d2800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.3:45258/0 (socket says 172.21.15.3:45258) 2024-05-09T06:36:15.542 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.558+0000 7fe4f9d9b640 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7fe4fc0d4c40 0x7fe4fc0d0bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.3:58486/0 (socket says 172.21.15.3:58486) 2024-05-09T06:36:15.543 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.558+0000 7fdf6c839640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fdf6814c250 0x7fdf6806d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:15.543 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.558+0000 7fdf6c839640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fdf6814c250 0x7fdf6806d240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49760/0 (socket says 172.21.15.3:49760) 2024-05-09T06:36:15.543 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.558+0000 7fdf6c839640 1 -- 172.21.15.3:0/2938211474 learned_addr learned my addr 172.21.15.3:0/2938211474 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:36:15.543 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.559+0000 7fdf6d03a640 1 --2- 172.21.15.3:0/2938211474 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdf6814be70 0x7fdf6806eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:15.543 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.559+0000 7fe500d6a640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fe4fc0d1300 0x7fe4fc0b47e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:15.543 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.559+0000 7fefdd93e640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fefd80cc270 0x7fefd8166220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:15.544 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.559+0000 7fefdd93e640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fefd80cc270 0x7fefd8166220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.3:45280/0 (socket says 172.21.15.3:45280) 2024-05-09T06:36:15.544 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.559+0000 7fe4fa59c640 1 -- 172.21.15.3:0/3139262144 learned_addr learned my addr 172.21.15.3:0/3139262144 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:36:15.544 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.559+0000 7fe4fad9d640 1 --2- 172.21.15.3:0/3139262144 >> v2:172.21.15.190:3300/0 conn(0x7fe4fc0d1300 0x7fe4fc0b47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:15.544 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.559+0000 7fe500d6a640 1 -- 172.21.15.3:0/3139262144 --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe4fc15e9a0 con 0x7fe4fc0d4c40 2024-05-09T06:36:15.544 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.559+0000 7fefde13f640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fefd806e250 0x7fefd806e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:15.544 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.559+0000 7fefde13f640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fefd806e250 0x7fefd806e6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49774/0 (socket says 172.21.15.3:49774) 2024-05-09T06:36:15.545 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.559+0000 7fefde13f640 1 -- 172.21.15.3:0/564500201 learned_addr learned my addr 172.21.15.3:0/564500201 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:36:15.545 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.559+0000 7fdf5ffff640 1 -- 172.21.15.3:0/2938211474 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1658070877 0 0) 0x7fdf681664c0 con 0x7fdf6806d780 2024-05-09T06:36:15.545 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.559+0000 7fdf5ffff640 1 -- 172.21.15.3:0/2938211474 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdf54003520 con 0x7fdf6806d780 2024-05-09T06:36:15.545 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.559+0000 7fe500d6a640 1 -- 172.21.15.3:0/3139262144 --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7fe4fc0d31c0 con 0x7fe4fc0d1300 2024-05-09T06:36:15.545 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.559+0000 7fe500d6a640 1 -- 172.21.15.3:0/3139262144 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fe4fc0d50a0 con 0x7fe4fc15e5c0 2024-05-09T06:36:15.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.560+0000 7fe4fa59c640 1 -- 172.21.15.3:0/3139262144 >> v2:172.21.15.190:3300/0 conn(0x7fe4fc0d1300 msgr2=0x7fe4fc0b47e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:15.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.559+0000 7fdf6c839640 1 -- 172.21.15.3:0/2938211474 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdf6814be70 msgr2=0x7fdf6806eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:15.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.559+0000 7fdf6c839640 1 --2- 172.21.15.3:0/2938211474 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdf6814be70 0x7fdf6806eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:15.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.559+0000 7fdf6c839640 1 -- 172.21.15.3:0/2938211474 >> v1:172.21.15.190:6789/0 conn(0x7fdf6806d780 legacy=0x7fdf680b4210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:15.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.559+0000 7fdf6c839640 1 -- 172.21.15.3:0/2938211474 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf6806dd90 con 0x7fdf6814c250 2024-05-09T06:36:15.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.560+0000 7fefde13f640 1 -- 172.21.15.3:0/564500201 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fefd80cc270 msgr2=0x7fefd8166220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:15.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.560+0000 7fefde13f640 1 --2- 172.21.15.3:0/564500201 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fefd80cc270 0x7fefd8166220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:15.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.560+0000 7fefde13f640 1 -- 172.21.15.3:0/564500201 >> v1:172.21.15.190:6789/0 conn(0x7fefd80cc650 legacy=0x7fefd8163460 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T06:36:15.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.560+0000 7fefde13f640 1 -- 172.21.15.3:0/564500201 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefd8162f40 con 0x7fefd806e250 2024-05-09T06:36:15.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.560+0000 7fe4fa59c640 1 --2- 172.21.15.3:0/3139262144 >> v2:172.21.15.190:3300/0 conn(0x7fe4fc0d1300 0x7fe4fc0b47e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:15.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.560+0000 7fe4fa59c640 1 -- 172.21.15.3:0/3139262144 >> v1:172.21.15.190:6789/0 conn(0x7fe4fc0d4c40 legacy=0x7fe4fc0d0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:15.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.560+0000 7fe4fa59c640 1 -- 172.21.15.3:0/3139262144 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4fc0d2ee0 con 0x7fe4fc15e5c0 2024-05-09T06:36:15.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.561+0000 7fdf6c839640 1 --2- 172.21.15.3:0/2938211474 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdf6814c250 0x7fdf6806d240 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fdf60009e50 tx=0x7fdf6002a750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd190b63ba20b7 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:15.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.561+0000 7fe4fa59c640 1 --2- 172.21.15.3:0/3139262144 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe4fc15e5c0 0x7fe4fc0d2800 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fe4f4009e50 tx=0x7fe4f402a630 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cf9a2f43e8c4db18 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:15.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.561+0000 7fe4f959a640 1 -- 172.21.15.3:0/3139262144 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4f4036070 con 0x7fe4fc15e5c0 2024-05-09T06:36:15.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.561+0000 7fe4f959a640 1 -- 172.21.15.3:0/3139262144 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4f4031600 con 0x7fe4fc15e5c0 2024-05-09T06:36:15.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.561+0000 7fe4f959a640 1 -- 172.21.15.3:0/3139262144 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4f40319b0 con 0x7fe4fc15e5c0 2024-05-09T06:36:15.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.561+0000 7fdf5ffff640 1 -- 172.21.15.3:0/2938211474 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf60036070 con 0x7fdf6814c250 2024-05-09T06:36:15.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.561+0000 7fdf5ffff640 1 -- 172.21.15.3:0/2938211474 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf60031600 con 0x7fdf6814c250 2024-05-09T06:36:15.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.562+0000 7fdf6f29e640 1 -- 172.21.15.3:0/2938211474 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdf6814c250 msgr2=0x7fdf6806d240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:15.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.562+0000 7fdf6f29e640 1 --2- 172.21.15.3:0/2938211474 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdf6814c250 0x7fdf6806d240 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fdf60009e50 tx=0x7fdf6002a750 comp rx=0 tx=0).stop 2024-05-09T06:36:15.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.562+0000 7fdf6f29e640 1 -- 172.21.15.3:0/2938211474 shutdown_connections 2024-05-09T06:36:15.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.562+0000 7fdf6f29e640 1 --2- 172.21.15.3:0/2938211474 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdf6814c250 0x7fdf6806d240 secure :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fdf60009e50 tx=0x7fdf6002a750 comp rx=0 tx=0).stop 2024-05-09T06:36:15.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.562+0000 7fdf6f29e640 1 --2- 172.21.15.3:0/2938211474 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdf6814be70 0x7fdf6806eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:15.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.562+0000 7fdf6f29e640 1 -- 172.21.15.3:0/2938211474 >> 172.21.15.3:0/2938211474 conn(0x7fdf680b1c20 msgr2=0x7fdf680b2000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:15.550 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.562+0000 7fe500d6a640 1 -- 172.21.15.3:0/3139262144 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe4fc15e5c0 msgr2=0x7fe4fc0d2800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:15.550 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.562+0000 7fe500d6a640 1 --2- 172.21.15.3:0/3139262144 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe4fc15e5c0 0x7fe4fc0d2800 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fe4f4009e50 tx=0x7fe4f402a630 comp rx=0 tx=0).stop 2024-05-09T06:36:15.550 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.563+0000 7fdf6f29e640 1 -- 172.21.15.3:0/2938211474 shutdown_connections 2024-05-09T06:36:15.550 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.563+0000 7fdf6f29e640 1 -- 172.21.15.3:0/2938211474 wait complete. 2024-05-09T06:36:15.550 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.563+0000 7fe500d6a640 1 -- 172.21.15.3:0/3139262144 shutdown_connections 2024-05-09T06:36:15.551 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.563+0000 7fefde13f640 1 --2- 172.21.15.3:0/564500201 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fefd806e250 0x7fefd806e6e0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fefd0009e50 tx=0x7fefd002a6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90f226714fad9188 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:15.551 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.563+0000 7fe500d6a640 1 --2- 172.21.15.3:0/3139262144 >> v2:172.21.15.190:3300/0 conn(0x7fe4fc0d1300 0x7fe4fc0b47e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:15.551 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.563+0000 7fe500d6a640 1 --2- 172.21.15.3:0/3139262144 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe4fc15e5c0 0x7fe4fc0d2800 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:15.551 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.563+0000 7fe500d6a640 1 -- 172.21.15.3:0/3139262144 >> 172.21.15.3:0/3139262144 conn(0x7fe4fc09d500 msgr2=0x7fe4fc0d4220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:15.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.563+0000 7fefdc93c640 1 -- 172.21.15.3:0/564500201 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefd0036070 con 0x7fefd806e250 2024-05-09T06:36:15.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.563+0000 7fefdc93c640 1 -- 172.21.15.3:0/564500201 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fefd0031600 con 0x7fefd806e250 2024-05-09T06:36:15.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.564+0000 7fefdfba2640 1 -- 172.21.15.3:0/564500201 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fefd806e250 msgr2=0x7fefd806e6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:15.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.564+0000 7fefdfba2640 1 --2- 172.21.15.3:0/564500201 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fefd806e250 0x7fefd806e6e0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fefd0009e50 tx=0x7fefd002a6d0 comp rx=0 tx=0).stop 2024-05-09T06:36:15.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.564+0000 7fefdfba2640 1 -- 172.21.15.3:0/564500201 shutdown_connections 2024-05-09T06:36:15.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.564+0000 7fefdfba2640 1 --2- 172.21.15.3:0/564500201 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fefd806e250 0x7fefd806e6e0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:15.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.564+0000 7fefdfba2640 1 --2- 172.21.15.3:0/564500201 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fefd80cc270 0x7fefd8166220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:15.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.564+0000 7fefdfba2640 1 -- 172.21.15.3:0/564500201 >> 172.21.15.3:0/564500201 conn(0x7fefd80b1c20 msgr2=0x7fefd80b2030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:15.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.565+0000 7fe500d6a640 1 -- 172.21.15.3:0/3139262144 shutdown_connections 2024-05-09T06:36:15.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.565+0000 7fe500d6a640 1 -- 172.21.15.3:0/3139262144 wait complete. 2024-05-09T06:36:15.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.565+0000 7fefdfba2640 1 -- 172.21.15.3:0/564500201 shutdown_connections 2024-05-09T06:36:15.554 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.565+0000 7fdf6f29e640 1 Processor -- start 2024-05-09T06:36:15.554 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.567+0000 7fe500d6a640 1 Processor -- start 2024-05-09T06:36:15.554 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.567+0000 7fe500d6a640 1 -- start start 2024-05-09T06:36:15.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.565+0000 7fefdfba2640 1 -- 172.21.15.3:0/564500201 wait complete. 2024-05-09T06:36:15.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.567+0000 7fefdfba2640 1 Processor -- start 2024-05-09T06:36:15.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.567+0000 7fefdfba2640 1 -- start start 2024-05-09T06:36:15.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.565+0000 7fdf6f29e640 1 -- start start 2024-05-09T06:36:15.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.566+0000 7fdf6f29e640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdf6806d780 0x7fdf68161f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:15.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.566+0000 7fdf6f29e640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdf6814be70 0x7fdf68164f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:15.556 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.569+0000 7fe500d6a640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe4fc0d1300 0x7fe4fc15cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:15.556 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.566+0000 7fdf6f29e640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdf68147210 0x7fdf68071080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:15.556 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.569+0000 7fefdfba2640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fefd806e250 0x7fefd8139b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:15.557 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.569+0000 7fe500d6a640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe4fc0d4c40 0x7fe4fc0cd090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:15.557 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.569+0000 7fefdfba2640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fefd80cc270 0x7fefd8165ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:15.557 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.569+0000 7fe500d6a640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe4fc15e5c0 0x7fe4fc0cd5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:15.557 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.569+0000 7fefdfba2640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fefd80cc650 0x7fefd8136e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:15.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.569+0000 7fefdfba2640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fefd8166760 con 0x7fefd806e250 2024-05-09T06:36:15.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.569+0000 7fe4fad9d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe4fc15e5c0 0x7fe4fc0cd5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:15.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.569+0000 7fefdfba2640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fefd8162c30 con 0x7fefd80cc270 2024-05-09T06:36:15.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.569+0000 7fe4fad9d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe4fc15e5c0 0x7fe4fc0cd5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.3:47990/0 (socket says 172.21.15.3:47990) 2024-05-09T06:36:15.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.569+0000 7fefdfba2640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fefd8145cc0 con 0x7fefd80cc650 2024-05-09T06:36:15.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.569+0000 7fe4fad9d640 1 -- 172.21.15.3:0/1664041991 learned_addr learned my addr 172.21.15.3:0/1664041991 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:36:15.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.569+0000 7fe500d6a640 1 -- 172.21.15.3:0/1664041991 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fe4fc0b42c0 con 0x7fe4fc0d4c40 2024-05-09T06:36:15.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.569+0000 7fe500d6a640 1 -- 172.21.15.3:0/1664041991 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fe4fc0b3fe0 con 0x7fe4fc15e5c0 2024-05-09T06:36:15.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.569+0000 7fe500d6a640 1 -- 172.21.15.3:0/1664041991 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fe4fc0b4d20 con 0x7fe4fc0d1300 2024-05-09T06:36:15.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.570+0000 7fe4f9d9b640 1 --2- 172.21.15.3:0/1664041991 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe4fc0d4c40 0x7fe4fc0cd090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:15.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.570+0000 7fe4fa59c640 1 --2- 172.21.15.3:0/1664041991 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe4fc0d1300 0x7fe4fc15cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:15.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.570+0000 7fefdd93e640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fefd806e250 0x7fefd8139b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:15.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.570+0000 7fefdd93e640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fefd806e250 0x7fefd8139b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49788/0 (socket says 172.21.15.3:49788) 2024-05-09T06:36:15.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.570+0000 7fefdd93e640 1 -- 172.21.15.3:0/977788228 learned_addr learned my addr 172.21.15.3:0/977788228 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:36:15.562 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.566+0000 7fdf6f29e640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fdf6806e6a0 con 0x7fdf6806d780 2024-05-09T06:36:15.562 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.566+0000 7fdf6c839640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdf6814be70 0x7fdf68164f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:15.562 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.566+0000 7fdf6c839640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdf6814be70 0x7fdf68164f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.3:47970/0 (socket says 172.21.15.3:47970) 2024-05-09T06:36:15.562 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.566+0000 7fdf6c839640 1 -- 172.21.15.3:0/2027673630 learned_addr learned my addr 172.21.15.3:0/2027673630 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:36:15.562 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.566+0000 7fdf6d83b640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdf68147210 0x7fdf68071080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:15.562 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.567+0000 7fdf6f29e640 1 -- 172.21.15.3:0/2027673630 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fdf6806e3c0 con 0x7fdf6814be70 2024-05-09T06:36:15.562 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.567+0000 7fdf6f29e640 1 -- 172.21.15.3:0/2027673630 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fdf680b4920 con 0x7fdf68147210 2024-05-09T06:36:15.562 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.567+0000 7fdf6d03a640 1 --2- 172.21.15.3:0/2027673630 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdf6806d780 0x7fdf68161f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:15.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.568+0000 7fdf6d83b640 1 -- 172.21.15.3:0/2027673630 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdf6814be70 msgr2=0x7fdf68164f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:15.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.568+0000 7fdf6d83b640 1 --2- 172.21.15.3:0/2027673630 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdf6814be70 0x7fdf68164f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:15.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.568+0000 7fdf6d83b640 1 -- 172.21.15.3:0/2027673630 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdf6806d780 msgr2=0x7fdf68161f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:15.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.568+0000 7fdf6d83b640 1 --2- 172.21.15.3:0/2027673630 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdf6806d780 0x7fdf68161f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:15.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.568+0000 7fdf6d83b640 1 -- 172.21.15.3:0/2027673630 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf68164be0 con 0x7fdf68147210 2024-05-09T06:36:15.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.571+0000 7fefde13f640 1 --2- 172.21.15.3:0/977788228 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fefd80cc650 0x7fefd8136e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:15.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.571+0000 7fefdd93e640 1 -- 172.21.15.3:0/977788228 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fefd80cc650 msgr2=0x7fefd8136e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:15.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.568+0000 7fdf6d83b640 1 --2- 172.21.15.3:0/2027673630 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdf68147210 0x7fdf68071080 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fdf6400c3c0 tx=0x7fdf6400c880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:15.564 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.571+0000 7fefdd93e640 1 --2- 172.21.15.3:0/977788228 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fefd80cc650 0x7fefd8136e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:15.564 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.571+0000 7fefdd93e640 1 -- 172.21.15.3:0/977788228 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fefd80cc270 msgr2=0x7fefd8165ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:15.564 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.569+0000 7fdf5dffb640 1 -- 172.21.15.3:0/2027673630 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf64039070 con 0x7fdf68147210 2024-05-09T06:36:15.564 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.569+0000 7fdf5dffb640 1 -- 172.21.15.3:0/2027673630 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf640050d0 con 0x7fdf68147210 2024-05-09T06:36:15.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.570+0000 7fdf5dffb640 1 -- 172.21.15.3:0/2027673630 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf64033660 con 0x7fdf68147210 2024-05-09T06:36:15.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.570+0000 7fdf6f29e640 1 -- 172.21.15.3:0/2027673630 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf68143460 con 0x7fdf68147210 2024-05-09T06:36:15.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.571+0000 7fe4f9d9b640 1 -- 172.21.15.3:0/1664041991 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe4fc0d1300 msgr2=0x7fe4fc15cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:15.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.571+0000 7fe4f9d9b640 1 --2- 172.21.15.3:0/1664041991 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe4fc0d1300 0x7fe4fc15cb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:15.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.571+0000 7fe4f9d9b640 1 -- 172.21.15.3:0/1664041991 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe4fc15e5c0 msgr2=0x7fe4fc0cd5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:15.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.571+0000 7fe4f9d9b640 1 --2- 172.21.15.3:0/1664041991 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe4fc15e5c0 0x7fe4fc0cd5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:15.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.571+0000 7fe4f9d9b640 1 -- 172.21.15.3:0/1664041991 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4fc0cc8c0 con 0x7fe4fc0d4c40 2024-05-09T06:36:15.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.571+0000 7fdf6f29e640 1 -- 172.21.15.3:0/2027673630 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf681642b0 con 0x7fdf68147210 2024-05-09T06:36:15.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.571+0000 7fefdd13d640 1 --2- 172.21.15.3:0/977788228 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fefd80cc270 0x7fefd8165ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:15.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.571+0000 7fefdd93e640 1 --2- 172.21.15.3:0/977788228 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fefd80cc270 0x7fefd8165ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:15.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.571+0000 7fefdd93e640 1 -- 172.21.15.3:0/977788228 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefd81376a0 con 0x7fefd806e250 2024-05-09T06:36:15.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.571+0000 7fefdd13d640 1 --2- 172.21.15.3:0/977788228 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fefd80cc270 0x7fefd8165ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:36:15.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.572+0000 7fefdd93e640 1 --2- 172.21.15.3:0/977788228 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fefd806e250 0x7fefd8139b60 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fefc800e9b0 tx=0x7fefc800ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:15.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.572+0000 7fdf6f29e640 1 -- 172.21.15.3:0/2027673630 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf68069550 con 0x7fdf68147210 2024-05-09T06:36:15.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.572+0000 7fe4f9d9b640 1 --2- 172.21.15.3:0/1664041991 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe4fc0d4c40 0x7fe4fc0cd090 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fe4ec00c780 tx=0x7fe4ec00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:15.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.572+0000 7fe4fad9d640 1 --2- 172.21.15.3:0/1664041991 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe4fc15e5c0 0x7fe4fc0cd5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:36:15.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.573+0000 7fe4eaffd640 1 -- 172.21.15.3:0/1664041991 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4ec018070 con 0x7fe4fc0d4c40 2024-05-09T06:36:15.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.573+0000 7fe4eaffd640 1 -- 172.21.15.3:0/1664041991 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4ec0072d0 con 0x7fe4fc0d4c40 2024-05-09T06:36:15.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.573+0000 7fe4eaffd640 1 -- 172.21.15.3:0/1664041991 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4ec0047e0 con 0x7fe4fc0d4c40 2024-05-09T06:36:15.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.573+0000 7fefce7fc640 1 -- 172.21.15.3:0/977788228 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefc8017070 con 0x7fefd806e250 2024-05-09T06:36:15.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.573+0000 7fefdfba2640 1 -- 172.21.15.3:0/977788228 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fefd8137980 con 0x7fefd806e250 2024-05-09T06:36:15.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.573+0000 7fefdfba2640 1 -- 172.21.15.3:0/977788228 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fefd8137f40 con 0x7fefd806e250 2024-05-09T06:36:15.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.573+0000 7fefce7fc640 1 -- 172.21.15.3:0/977788228 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fefc800c9a0 con 0x7fefd806e250 2024-05-09T06:36:15.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.573+0000 7fdf5dffb640 1 -- 172.21.15.3:0/2027673630 <== mon.2 v2:172.21.15.190:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fdf6403c070 con 0x7fdf68147210 2024-05-09T06:36:15.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.574+0000 7fdf5dffb640 1 --2- 172.21.15.3:0/2027673630 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fdf4c06bf00 0x7fdf4c06e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:15.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.574+0000 7fdf5dffb640 1 -- 172.21.15.3:0/2027673630 <== mon.2 v2:172.21.15.190:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fdf640ac010 con 0x7fdf68147210 2024-05-09T06:36:15.570 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.574+0000 7fe500d6a640 1 -- 172.21.15.3:0/1664041991 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4fc0cc020 con 0x7fe4fc0d4c40 2024-05-09T06:36:15.570 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.574+0000 7fe500d6a640 1 -- 172.21.15.3:0/1664041991 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4fc0cc5e0 con 0x7fe4fc0d4c40 2024-05-09T06:36:15.570 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.573+0000 7fefce7fc640 1 -- 172.21.15.3:0/977788228 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefc800cd90 con 0x7fefd806e250 2024-05-09T06:36:15.570 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.576+0000 7fefdfba2640 1 -- 172.21.15.3:0/977788228 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fefc0005290 con 0x7fefd806e250 2024-05-09T06:36:15.570 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.576+0000 7fe500d6a640 1 -- 172.21.15.3:0/1664041991 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4fc069550 con 0x7fe4fc0d4c40 2024-05-09T06:36:15.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.576+0000 7fdf6d03a640 1 --2- 172.21.15.3:0/2027673630 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fdf4c06bf00 0x7fdf4c06e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:15.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.577+0000 7fdf6d03a640 1 --2- 172.21.15.3:0/2027673630 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fdf4c06bf00 0x7fdf4c06e3c0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fdf58009b40 tx=0x7fdf58009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:15.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.578+0000 7fe4eaffd640 1 -- 172.21.15.3:0/1664041991 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe4ec01b070 con 0x7fe4fc0d4c40 2024-05-09T06:36:15.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.578+0000 7fe4eaffd640 1 --2- 172.21.15.3:0/1664041991 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fe4dc06bf00 0x7fe4dc06e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:15.572 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.579+0000 7fe4eaffd640 1 -- 172.21.15.3:0/1664041991 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fe4ec08a1d0 con 0x7fe4fc0d4c40 2024-05-09T06:36:15.572 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.578+0000 7fefce7fc640 1 -- 172.21.15.3:0/977788228 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fefc801a070 con 0x7fefd806e250 2024-05-09T06:36:15.572 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.579+0000 7fdf5dffb640 1 -- 172.21.15.3:0/2027673630 <== mon.2 v2:172.21.15.190:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fdf6407a350 con 0x7fdf68147210 2024-05-09T06:36:15.572 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.579+0000 7fefce7fc640 1 --2- 172.21.15.3:0/977788228 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fefbc06bf00 0x7fefbc06e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:15.573 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.579+0000 7fefce7fc640 1 -- 172.21.15.3:0/977788228 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fefc808b6e0 con 0x7fefd806e250 2024-05-09T06:36:15.573 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.580+0000 7fefdd13d640 1 --2- 172.21.15.3:0/977788228 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fefbc06bf00 0x7fefbc06e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:15.573 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.580+0000 7fe4fa59c640 1 --2- 172.21.15.3:0/1664041991 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fe4dc06bf00 0x7fe4dc06e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:15.573 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.580+0000 7fefdd13d640 1 --2- 172.21.15.3:0/977788228 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fefbc06bf00 0x7fefbc06e3c0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fefd8145300 tx=0x7fefd4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:15.573 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.581+0000 7fe4fa59c640 1 --2- 172.21.15.3:0/1664041991 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fe4dc06bf00 0x7fe4dc06e3c0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fe4f4033000 tx=0x7fe4f4009000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:15.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.588+0000 7fefce7fc640 1 -- 172.21.15.3:0/977788228 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fefc805a070 con 0x7fefd806e250 2024-05-09T06:36:15.578 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.597+0000 7fe4eaffd640 1 -- 172.21.15.3:0/1664041991 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fe4ec0585a0 con 0x7fe4fc0d4c40 2024-05-09T06:36:15.891 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.909+0000 7fed567fc640 1 -- 172.21.15.3:0/2567913175 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7fed44005850 con 0x7fed6816adb0 2024-05-09T06:36:15.892 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.910+0000 7fed64ff9640 1 -- 172.21.15.3:0/2567913175 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fed580586b0 con 0x7fed6816adb0 2024-05-09T06:36:15.892 INFO:teuthology.orchestra.run.smithi003.stdout:55834574855 2024-05-09T06:36:15.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.913+0000 7fed6e8f5640 1 -- 172.21.15.3:0/2567913175 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fed4c06bf60 msgr2=0x7fed4c06e420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:15.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.913+0000 7fed6e8f5640 1 --2- 172.21.15.3:0/2567913175 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fed4c06bf60 0x7fed4c06e420 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fed5c03d000 tx=0x7fed5c038000 comp rx=0 tx=0).stop 2024-05-09T06:36:15.895 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.913+0000 7fed6e8f5640 1 -- 172.21.15.3:0/2567913175 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fed6816adb0 msgr2=0x7fed681f5960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:15.895 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.913+0000 7fed6e8f5640 1 --2- 172.21.15.3:0/2567913175 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fed6816adb0 0x7fed681f5960 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fed58007b00 tx=0x7fed5800ac80 comp rx=0 tx=0).stop 2024-05-09T06:36:15.895 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.914+0000 7fed6e8f5640 1 -- 172.21.15.3:0/2567913175 shutdown_connections 2024-05-09T06:36:15.895 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.914+0000 7fed6e8f5640 1 --2- 172.21.15.3:0/2567913175 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fed4c06bf60 0x7fed4c06e420 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:15.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.914+0000 7fed6e8f5640 1 --2- 172.21.15.3:0/2567913175 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fed6816b190 0x7fed681f5ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:15.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.914+0000 7fed6e8f5640 1 --2- 172.21.15.3:0/2567913175 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fed6816adb0 0x7fed681f5960 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:15.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.914+0000 7fed6e8f5640 1 --2- 172.21.15.3:0/2567913175 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fed680b4030 0x7fed680b2f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:15.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.914+0000 7fed6e8f5640 1 -- 172.21.15.3:0/2567913175 >> 172.21.15.3:0/2567913175 conn(0x7fed680b1c20 msgr2=0x7fed680afb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:15.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.915+0000 7fed6e8f5640 1 -- 172.21.15.3:0/2567913175 shutdown_connections 2024-05-09T06:36:15.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.915+0000 7fed6e8f5640 1 -- 172.21.15.3:0/2567913175 wait complete. 2024-05-09T06:36:15.913 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574855 for osd.2 2024-05-09T06:36:15.913 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.933+0000 7fd7fa858640 1 -- 172.21.15.3:0/2811719499 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7fd7dc005580 con 0x7fd7f415a4b0 2024-05-09T06:36:15.914 DEBUG:teuthology.parallel:result is None 2024-05-09T06:36:15.914 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.933+0000 7fd7ea7fc640 1 -- 172.21.15.3:0/2811719499 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fd7f415c690 con 0x7fd7f415a4b0 2024-05-09T06:36:15.914 INFO:teuthology.orchestra.run.smithi003.stdout:55834574856 2024-05-09T06:36:15.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.936+0000 7fd7fa858640 1 -- 172.21.15.3:0/2811719499 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fd7d806bf00 msgr2=0x7fd7d806e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:15.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.937+0000 7fd7fa858640 1 --2- 172.21.15.3:0/2811719499 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fd7d806bf00 0x7fd7d806e3c0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fd7e4006fd0 tx=0x7fd7e4008040 comp rx=0 tx=0).stop 2024-05-09T06:36:15.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.937+0000 7fd7fa858640 1 -- 172.21.15.3:0/2811719499 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd7f415a4b0 msgr2=0x7fd7f41398a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:15.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.937+0000 7fd7fa858640 1 --2- 172.21.15.3:0/2811719499 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd7f415a4b0 0x7fd7f41398a0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fd7ec00eed0 tx=0x7fd7ec00b480 comp rx=0 tx=0).stop 2024-05-09T06:36:15.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.937+0000 7fd7fa858640 1 -- 172.21.15.3:0/2811719499 shutdown_connections 2024-05-09T06:36:15.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.937+0000 7fd7fa858640 1 --2- 172.21.15.3:0/2811719499 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fd7d806bf00 0x7fd7d806e3c0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:15.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.937+0000 7fd7fa858640 1 --2- 172.21.15.3:0/2811719499 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd7f415a4b0 0x7fd7f41398a0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:15.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.937+0000 7fd7fa858640 1 --2- 172.21.15.3:0/2811719499 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd7f40cd840 0x7fd7f40a9920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:15.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.937+0000 7fd7fa858640 1 --2- 172.21.15.3:0/2811719499 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd7f40af7a0 0x7fd7f415bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:15.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.937+0000 7fd7fa858640 1 -- 172.21.15.3:0/2811719499 >> 172.21.15.3:0/2811719499 conn(0x7fd7f40b24e0 msgr2=0x7fd7f41596f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:15.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.937+0000 7fd7fa858640 1 -- 172.21.15.3:0/2811719499 shutdown_connections 2024-05-09T06:36:15.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.938+0000 7fd7fa858640 1 -- 172.21.15.3:0/2811719499 wait complete. 2024-05-09T06:36:15.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.949+0000 7f8b35763640 1 -- 172.21.15.3:0/3332261616 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f8b28052c50 con 0x7f8b280df100 2024-05-09T06:36:15.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.950+0000 7f8b177fe640 1 -- 172.21.15.3:0/3332261616 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f8b2405d070 con 0x7f8b280df100 2024-05-09T06:36:15.931 INFO:teuthology.orchestra.run.smithi003.stdout:55834574856 2024-05-09T06:36:15.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.950+0000 7fec3b7fe640 1 -- 172.21.15.3:0/2575134751 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7fec580b6380 con 0x7fec5815eeb0 2024-05-09T06:36:15.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.951+0000 7fec4dffb640 1 -- 172.21.15.3:0/2575134751 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fec4805c920 con 0x7fec5815eeb0 2024-05-09T06:36:15.932 INFO:teuthology.orchestra.run.smithi003.stdout:55834574855 2024-05-09T06:36:15.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.953+0000 7f8b35763640 1 -- 172.21.15.3:0/3332261616 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f8b1c06be30 msgr2=0x7f8b1c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:15.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.953+0000 7f8b35763640 1 --2- 172.21.15.3:0/3332261616 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f8b1c06be30 0x7f8b1c06e2f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f8b20009a10 tx=0x7f8b20008040 comp rx=0 tx=0).stop 2024-05-09T06:36:15.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.953+0000 7f8b35763640 1 -- 172.21.15.3:0/3332261616 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8b280df100 msgr2=0x7f8b280dd1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:15.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.953+0000 7f8b35763640 1 --2- 172.21.15.3:0/3332261616 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8b280df100 0x7f8b280dd1f0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f8b2400b420 tx=0x7f8b2400b8e0 comp rx=0 tx=0).stop 2024-05-09T06:36:15.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.953+0000 7fec3b7fe640 1 -- 172.21.15.3:0/2575134751 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fec3c06beb0 msgr2=0x7fec3c06e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:15.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.953+0000 7f8b35763640 1 -- 172.21.15.3:0/3332261616 shutdown_connections 2024-05-09T06:36:15.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.953+0000 7f8b35763640 1 --2- 172.21.15.3:0/3332261616 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f8b1c06be30 0x7f8b1c06e2f0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:15.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.953+0000 7f8b35763640 1 --2- 172.21.15.3:0/3332261616 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8b280df100 0x7f8b280dd1f0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:15.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.953+0000 7f8b35763640 1 --2- 172.21.15.3:0/3332261616 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8b280530a0 0x7f8b280dccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:15.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.953+0000 7f8b35763640 1 --2- 172.21.15.3:0/3332261616 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8b2804ffb0 0x7f8b280d4300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:15.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.953+0000 7f8b35763640 1 -- 172.21.15.3:0/3332261616 >> 172.21.15.3:0/3332261616 conn(0x7f8b28009fb0 msgr2=0x7f8b28038b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:15.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.954+0000 7fec3b7fe640 1 --2- 172.21.15.3:0/2575134751 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fec3c06beb0 0x7fec3c06e370 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fec50034000 tx=0x7fec5002f000 comp rx=0 tx=0).stop 2024-05-09T06:36:15.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.954+0000 7fec3b7fe640 1 -- 172.21.15.3:0/2575134751 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fec5815eeb0 msgr2=0x7fec5812c530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:15.936 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574856 for osd.7 2024-05-09T06:36:15.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.954+0000 7fec3b7fe640 1 --2- 172.21.15.3:0/2575134751 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fec5815eeb0 0x7fec5812c530 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fec48001eb0 tx=0x7fec4800c870 comp rx=0 tx=0).stop 2024-05-09T06:36:15.937 DEBUG:teuthology.parallel:result is None 2024-05-09T06:36:15.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.954+0000 7f8b35763640 1 -- 172.21.15.3:0/3332261616 shutdown_connections 2024-05-09T06:36:15.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.954+0000 7f8b35763640 1 -- 172.21.15.3:0/3332261616 wait complete. 2024-05-09T06:36:15.938 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.954+0000 7fec3b7fe640 1 -- 172.21.15.3:0/2575134751 shutdown_connections 2024-05-09T06:36:15.938 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.954+0000 7fec3b7fe640 1 --2- 172.21.15.3:0/2575134751 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fec3c06beb0 0x7fec3c06e370 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:15.938 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.954+0000 7fec3b7fe640 1 --2- 172.21.15.3:0/2575134751 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fec5815eeb0 0x7fec5812c530 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:15.938 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.954+0000 7fec3b7fe640 1 --2- 172.21.15.3:0/2575134751 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fec5813df10 0x7fec5812bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:15.938 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.954+0000 7fec3b7fe640 1 --2- 172.21.15.3:0/2575134751 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fec580cfba0 0x7fec58157590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:15.938 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.954+0000 7fec3b7fe640 1 -- 172.21.15.3:0/2575134751 >> 172.21.15.3:0/2575134751 conn(0x7fec580b2c30 msgr2=0x7fec580b2310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:15.938 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.954+0000 7fec3b7fe640 1 -- 172.21.15.3:0/2575134751 shutdown_connections 2024-05-09T06:36:15.938 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.954+0000 7fec3b7fe640 1 -- 172.21.15.3:0/2575134751 wait complete. 2024-05-09T06:36:15.947 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574855 for osd.4 2024-05-09T06:36:15.948 DEBUG:teuthology.parallel:result is None 2024-05-09T06:36:15.952 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574856 for osd.6 2024-05-09T06:36:15.952 DEBUG:teuthology.parallel:result is None 2024-05-09T06:36:15.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.981+0000 7fa3a081a640 1 -- 172.21.15.3:0/3663840303 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7fa37c005580 con 0x7fa394053750 2024-05-09T06:36:15.963 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.982+0000 7fa3837fe640 1 -- 172.21.15.3:0/3663840303 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fa38c079ab0 con 0x7fa394053750 2024-05-09T06:36:15.963 INFO:teuthology.orchestra.run.smithi003.stdout:55834574855 2024-05-09T06:36:15.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.985+0000 7fa3a081a640 1 -- 172.21.15.3:0/3663840303 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fa38806bf00 msgr2=0x7fa38806e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:15.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.985+0000 7fa3a081a640 1 --2- 172.21.15.3:0/3663840303 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fa38806bf00 0x7fa38806e3c0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fa39c000b60 tx=0x7fa39c09f040 comp rx=0 tx=0).stop 2024-05-09T06:36:15.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.985+0000 7fa3a081a640 1 -- 172.21.15.3:0/3663840303 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa394053750 msgr2=0x7fa3940cb5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:15.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.985+0000 7fa3a081a640 1 --2- 172.21.15.3:0/3663840303 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa394053750 0x7fa3940cb5a0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fa38c009960 tx=0x7fa38c0046c0 comp rx=0 tx=0).stop 2024-05-09T06:36:15.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.985+0000 7fa3a081a640 1 -- 172.21.15.3:0/3663840303 shutdown_connections 2024-05-09T06:36:15.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.985+0000 7fa3a081a640 1 --2- 172.21.15.3:0/3663840303 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fa38806bf00 0x7fa38806e3c0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:15.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.985+0000 7fa3a081a640 1 --2- 172.21.15.3:0/3663840303 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa3940e3300 0x7fa39404ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:15.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.985+0000 7fa3a081a640 1 --2- 172.21.15.3:0/3663840303 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa394054140 0x7fa3940cbae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:15.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.985+0000 7fa3a081a640 1 --2- 172.21.15.3:0/3663840303 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa394053750 0x7fa3940cb5a0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:15.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.985+0000 7fa3a081a640 1 -- 172.21.15.3:0/3663840303 >> 172.21.15.3:0/3663840303 conn(0x7fa394058ef0 msgr2=0x7fa3940e39d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:15.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.985+0000 7fa3a081a640 1 -- 172.21.15.3:0/3663840303 shutdown_connections 2024-05-09T06:36:15.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.985+0000 7fa3a081a640 1 -- 172.21.15.3:0/3663840303 wait complete. 2024-05-09T06:36:15.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.985+0000 7fefdfba2640 1 -- 172.21.15.3:0/977788228 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7fefc0005850 con 0x7fefd806e250 2024-05-09T06:36:15.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.986+0000 7fefce7fc640 1 -- 172.21.15.3:0/977788228 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fefc8059970 con 0x7fefd806e250 2024-05-09T06:36:15.968 INFO:teuthology.orchestra.run.smithi003.stdout:55834574856 2024-05-09T06:36:15.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.987+0000 7fe500d6a640 1 -- 172.21.15.3:0/1664041991 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7fe4fc06db60 con 0x7fe4fc0d4c40 2024-05-09T06:36:15.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.987+0000 7fe4eaffd640 1 -- 172.21.15.3:0/1664041991 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fe4ec05c8a0 con 0x7fe4fc0d4c40 2024-05-09T06:36:15.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.988+0000 7fefdfba2640 1 -- 172.21.15.3:0/977788228 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fefbc06bf00 msgr2=0x7fefbc06e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:15.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.988+0000 7fefdfba2640 1 --2- 172.21.15.3:0/977788228 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fefbc06bf00 0x7fefbc06e3c0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fefd8145300 tx=0x7fefd4008040 comp rx=0 tx=0).stop 2024-05-09T06:36:15.969 INFO:teuthology.orchestra.run.smithi003.stdout:55834574856 2024-05-09T06:36:15.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.988+0000 7fefdfba2640 1 -- 172.21.15.3:0/977788228 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fefd806e250 msgr2=0x7fefd8139b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:15.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.988+0000 7fefdfba2640 1 --2- 172.21.15.3:0/977788228 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fefd806e250 0x7fefd8139b60 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fefc800e9b0 tx=0x7fefc800ee70 comp rx=0 tx=0).stop 2024-05-09T06:36:15.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.988+0000 7fefdfba2640 1 -- 172.21.15.3:0/977788228 shutdown_connections 2024-05-09T06:36:15.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.988+0000 7fefdfba2640 1 --2- 172.21.15.3:0/977788228 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fefbc06bf00 0x7fefbc06e3c0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:15.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.988+0000 7fefdfba2640 1 --2- 172.21.15.3:0/977788228 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fefd80cc650 0x7fefd8136e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:15.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.988+0000 7fefdfba2640 1 --2- 172.21.15.3:0/977788228 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fefd80cc270 0x7fefd8165ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:15.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.988+0000 7fefdfba2640 1 --2- 172.21.15.3:0/977788228 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fefd806e250 0x7fefd8139b60 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:15.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.988+0000 7fefdfba2640 1 -- 172.21.15.3:0/977788228 >> 172.21.15.3:0/977788228 conn(0x7fefd80b1c20 msgr2=0x7fefd8071080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:15.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.988+0000 7fefdfba2640 1 -- 172.21.15.3:0/977788228 shutdown_connections 2024-05-09T06:36:15.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.988+0000 7fefdfba2640 1 -- 172.21.15.3:0/977788228 wait complete. 2024-05-09T06:36:15.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.990+0000 7fe500d6a640 1 -- 172.21.15.3:0/1664041991 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fe4dc06bf00 msgr2=0x7fe4dc06e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:15.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.990+0000 7fe500d6a640 1 --2- 172.21.15.3:0/1664041991 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fe4dc06bf00 0x7fe4dc06e3c0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fe4f4033000 tx=0x7fe4f4009000 comp rx=0 tx=0).stop 2024-05-09T06:36:15.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.990+0000 7fe500d6a640 1 -- 172.21.15.3:0/1664041991 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe4fc0d4c40 msgr2=0x7fe4fc0cd090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:15.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.990+0000 7fe500d6a640 1 --2- 172.21.15.3:0/1664041991 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe4fc0d4c40 0x7fe4fc0cd090 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fe4ec00c780 tx=0x7fe4ec00cc40 comp rx=0 tx=0).stop 2024-05-09T06:36:15.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.990+0000 7fe500d6a640 1 -- 172.21.15.3:0/1664041991 shutdown_connections 2024-05-09T06:36:15.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.990+0000 7fe500d6a640 1 --2- 172.21.15.3:0/1664041991 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fe4dc06bf00 0x7fe4dc06e3c0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:15.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.990+0000 7fe500d6a640 1 --2- 172.21.15.3:0/1664041991 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe4fc15e5c0 0x7fe4fc0cd5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:15.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.990+0000 7fe500d6a640 1 --2- 172.21.15.3:0/1664041991 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe4fc0d4c40 0x7fe4fc0cd090 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:15.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.990+0000 7fe500d6a640 1 --2- 172.21.15.3:0/1664041991 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe4fc0d1300 0x7fe4fc15cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:15.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.990+0000 7fe500d6a640 1 -- 172.21.15.3:0/1664041991 >> 172.21.15.3:0/1664041991 conn(0x7fe4fc09d500 msgr2=0x7fe4fc0cf220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:15.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.990+0000 7fe500d6a640 1 -- 172.21.15.3:0/1664041991 shutdown_connections 2024-05-09T06:36:15.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:15.990+0000 7fe500d6a640 1 -- 172.21.15.3:0/1664041991 wait complete. 2024-05-09T06:36:15.982 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574856 for osd.3 2024-05-09T06:36:15.982 DEBUG:teuthology.parallel:result is None 2024-05-09T06:36:15.984 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574855 for osd.1 2024-05-09T06:36:15.984 DEBUG:teuthology.parallel:result is None 2024-05-09T06:36:15.984 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574856 for osd.0 2024-05-09T06:36:15.984 DEBUG:teuthology.parallel:result is None 2024-05-09T06:36:15.985 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:16.004+0000 7fdf6f29e640 1 -- 172.21.15.3:0/2027673630 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7fdf680723a0 con 0x7fdf68147210 2024-05-09T06:36:15.985 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:16.005+0000 7fdf5dffb640 1 -- 172.21.15.3:0/2027673630 <== mon.2 v2:172.21.15.190:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fdf6407e650 con 0x7fdf68147210 2024-05-09T06:36:15.986 INFO:teuthology.orchestra.run.smithi003.stdout:55834574855 2024-05-09T06:36:15.988 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:16.007+0000 7fdf6f29e640 1 -- 172.21.15.3:0/2027673630 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fdf4c06bf00 msgr2=0x7fdf4c06e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:15.988 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:16.007+0000 7fdf6f29e640 1 --2- 172.21.15.3:0/2027673630 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fdf4c06bf00 0x7fdf4c06e3c0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fdf58009b40 tx=0x7fdf58009190 comp rx=0 tx=0).stop 2024-05-09T06:36:15.988 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:16.007+0000 7fdf6f29e640 1 -- 172.21.15.3:0/2027673630 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdf68147210 msgr2=0x7fdf68071080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:15.988 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:16.007+0000 7fdf6f29e640 1 --2- 172.21.15.3:0/2027673630 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdf68147210 0x7fdf68071080 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fdf6400c3c0 tx=0x7fdf6400c880 comp rx=0 tx=0).stop 2024-05-09T06:36:15.988 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:16.007+0000 7fdf6f29e640 1 -- 172.21.15.3:0/2027673630 shutdown_connections 2024-05-09T06:36:15.988 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:16.007+0000 7fdf6f29e640 1 --2- 172.21.15.3:0/2027673630 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fdf4c06bf00 0x7fdf4c06e3c0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:15.988 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:16.007+0000 7fdf6f29e640 1 --2- 172.21.15.3:0/2027673630 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdf68147210 0x7fdf68071080 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:15.988 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:16.007+0000 7fdf6f29e640 1 --2- 172.21.15.3:0/2027673630 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdf6814be70 0x7fdf68164f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:15.988 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:16.007+0000 7fdf6f29e640 1 --2- 172.21.15.3:0/2027673630 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdf6806d780 0x7fdf68161f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:15.988 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:16.007+0000 7fdf6f29e640 1 -- 172.21.15.3:0/2027673630 >> 172.21.15.3:0/2027673630 conn(0x7fdf680b1c20 msgr2=0x7fdf680b2000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:15.989 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:16.007+0000 7fdf6f29e640 1 -- 172.21.15.3:0/2027673630 shutdown_connections 2024-05-09T06:36:15.989 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:16.007+0000 7fdf6f29e640 1 -- 172.21.15.3:0/2027673630 wait complete. 2024-05-09T06:36:15.998 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574855 for osd.5 2024-05-09T06:36:15.999 DEBUG:teuthology.parallel:result is None 2024-05-09T06:36:15.999 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-05-09T06:36:15.999 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-09T06:36:16.142 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:16.161+0000 7fd42f46b640 1 Processor -- start 2024-05-09T06:36:16.142 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:16.162+0000 7fd42f46b640 1 -- start start 2024-05-09T06:36:16.144 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:16.162+0000 7fd42f46b640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fd42815e480 0x7fd4280d1dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:16.144 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:16.162+0000 7fd42f46b640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fd4280d2ed0 0x7fd4281683a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:16.144 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:16.162+0000 7fd42f46b640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd4280d4e50 con 0x7fd4280d2310 2024-05-09T06:36:16.144 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:16.162+0000 7fd42f46b640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fd4280d0270 con 0x7fd42815e480 2024-05-09T06:36:16.144 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:16.162+0000 7fd42f46b640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7fd4280cf1d0 con 0x7fd4280d2ed0 2024-05-09T06:36:16.144 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:16.162+0000 7fd42d207640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fd42815e480 0x7fd4280d1dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:16.144 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:16.162+0000 7fd42d207640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fd42815e480 0x7fd4280d1dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49794/0 (socket says 172.21.15.3:49794) 2024-05-09T06:36:16.144 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:16.162+0000 7fd42d207640 1 -- 172.21.15.3:0/2853938158 learned_addr learned my addr 172.21.15.3:0/2853938158 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:36:16.144 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:16.163+0000 7fd42da08640 1 --2- 172.21.15.3:0/2853938158 >> v2:172.21.15.190:3300/0 conn(0x7fd4280d2ed0 0x7fd4281683a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:16.144 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:16.163+0000 7fd42d207640 1 -- 172.21.15.3:0/2853938158 >> v2:172.21.15.190:3300/0 conn(0x7fd4280d2ed0 msgr2=0x7fd4281683a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:16.144 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:16.163+0000 7fd42d207640 1 --2- 172.21.15.3:0/2853938158 >> v2:172.21.15.190:3300/0 conn(0x7fd4280d2ed0 0x7fd4281683a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:16.144 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:16.163+0000 7fd42d207640 1 -- 172.21.15.3:0/2853938158 >> v1:172.21.15.190:6789/0 conn(0x7fd4280d2310 legacy=0x7fd4280d27c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:16.144 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:16.163+0000 7fd42d207640 1 -- 172.21.15.3:0/2853938158 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4280cf4b0 con 0x7fd42815e480 2024-05-09T06:36:16.146 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:16.163+0000 7fd42d207640 1 --2- 172.21.15.3:0/2853938158 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd42815e480 0x7fd4280d1dd0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fd42808e050 tx=0x7fd41802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa05c9909bd3f0dc server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:16.146 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:16.164+0000 7fd427fff640 1 -- 172.21.15.3:0/2853938158 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd418037070 con 0x7fd42815e480 2024-05-09T06:36:16.146 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:16.164+0000 7fd427fff640 1 -- 172.21.15.3:0/2853938158 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd418032440 con 0x7fd42815e480 2024-05-09T06:36:16.146 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:16.164+0000 7fd427fff640 1 -- 172.21.15.3:0/2853938158 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd418032810 con 0x7fd42815e480 2024-05-09T06:36:16.146 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:16.164+0000 7fd42f46b640 1 -- 172.21.15.3:0/2853938158 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd42815e480 msgr2=0x7fd4280d1dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:16.146 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:16.164+0000 7fd42f46b640 1 --2- 172.21.15.3:0/2853938158 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd42815e480 0x7fd4280d1dd0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fd42808e050 tx=0x7fd41802a230 comp rx=0 tx=0).stop 2024-05-09T06:36:16.146 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:16.164+0000 7fd42f46b640 1 -- 172.21.15.3:0/2853938158 shutdown_connections 2024-05-09T06:36:16.146 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:16.164+0000 7fd42f46b640 1 --2- 172.21.15.3:0/2853938158 >> v2:172.21.15.190:3300/0 conn(0x7fd4280d2ed0 0x7fd4281683a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:16.146 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:16.164+0000 7fd42f46b640 1 --2- 172.21.15.3:0/2853938158 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd42815e480 0x7fd4280d1dd0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:16.146 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:16.164+0000 7fd42f46b640 1 -- 172.21.15.3:0/2853938158 >> 172.21.15.3:0/2853938158 conn(0x7fd42809d4d0 msgr2=0x7fd4280d4200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:16.146 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:16.164+0000 7fd42f46b640 1 -- 172.21.15.3:0/2853938158 shutdown_connections 2024-05-09T06:36:16.146 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:16.164+0000 7fd42f46b640 1 -- 172.21.15.3:0/2853938158 wait complete. 2024-05-09T06:36:16.149 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:16.165+0000 7fd42f46b640 1 Processor -- start 2024-05-09T06:36:16.149 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:16.166+0000 7fd42f46b640 1 -- start start 2024-05-09T06:36:16.149 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:16.166+0000 7fd42f46b640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd4280d2310 0x7fd4280c9120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:16.150 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:16.166+0000 7fd42f46b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd4280d2ed0 0x7fd4281e5c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:16.150 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:16.166+0000 7fd42f46b640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd42815e480 0x7fd428155520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:16.150 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:16.166+0000 7fd42f46b640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fd4280cfd50 con 0x7fd4280d2ed0 2024-05-09T06:36:16.150 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:16.166+0000 7fd42f46b640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fd4280cfa70 con 0x7fd4280d2310 2024-05-09T06:36:16.150 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:16.166+0000 7fd42f46b640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fd4280b5ab0 con 0x7fd42815e480 2024-05-09T06:36:16.150 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:16.166+0000 7fd42ca06640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd4280d2ed0 0x7fd4281e5c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:16.150 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:16.166+0000 7fd42ca06640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd4280d2ed0 0x7fd4281e5c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49798/0 (socket says 172.21.15.3:49798) 2024-05-09T06:36:16.150 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:16.166+0000 7fd42ca06640 1 -- 172.21.15.3:0/238483205 learned_addr learned my addr 172.21.15.3:0/238483205 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:36:16.150 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:16.167+0000 7fd42d207640 1 --2- 172.21.15.3:0/238483205 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd4280d2310 0x7fd4280c9120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:16.151 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:16.167+0000 7fd42da08640 1 --2- 172.21.15.3:0/238483205 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd42815e480 0x7fd428155520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:16.151 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:16.167+0000 7fd42ca06640 1 -- 172.21.15.3:0/238483205 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd42815e480 msgr2=0x7fd428155520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:16.151 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:16.167+0000 7fd42ca06640 1 --2- 172.21.15.3:0/238483205 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd42815e480 0x7fd428155520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:16.151 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:16.167+0000 7fd42ca06640 1 -- 172.21.15.3:0/238483205 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd4280d2310 msgr2=0x7fd4280c9120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:16.151 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:16.167+0000 7fd42ca06640 1 --2- 172.21.15.3:0/238483205 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd4280d2310 0x7fd4280c9120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:16.151 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:16.167+0000 7fd42ca06640 1 -- 172.21.15.3:0/238483205 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4281f82d0 con 0x7fd4280d2ed0 2024-05-09T06:36:16.151 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:16.167+0000 7fd42d207640 1 --2- 172.21.15.3:0/238483205 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd4280d2310 0x7fd4280c9120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:36:16.151 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:16.167+0000 7fd42ca06640 1 --2- 172.21.15.3:0/238483205 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd4280d2ed0 0x7fd4281e5c60 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fd41c00b9c0 tx=0x7fd41c00be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:16.151 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:16.167+0000 7fd425ffb640 1 -- 172.21.15.3:0/238483205 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd41c00cba0 con 0x7fd4280d2ed0 2024-05-09T06:36:16.151 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:16.167+0000 7fd425ffb640 1 -- 172.21.15.3:0/238483205 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd41c00cd40 con 0x7fd4280d2ed0 2024-05-09T06:36:16.151 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:16.167+0000 7fd42f46b640 1 -- 172.21.15.3:0/238483205 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4281f85b0 con 0x7fd4280d2ed0 2024-05-09T06:36:16.151 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:16.167+0000 7fd425ffb640 1 -- 172.21.15.3:0/238483205 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd41c003bb0 con 0x7fd4280d2ed0 2024-05-09T06:36:16.151 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:16.167+0000 7fd42f46b640 1 -- 172.21.15.3:0/238483205 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4281f8b70 con 0x7fd4280d2ed0 2024-05-09T06:36:16.151 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:16.168+0000 7fd42f46b640 1 -- 172.21.15.3:0/238483205 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd42804f660 con 0x7fd4280d2ed0 2024-05-09T06:36:16.151 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:16.169+0000 7fd425ffb640 1 -- 172.21.15.3:0/238483205 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd41c014440 con 0x7fd4280d2ed0 2024-05-09T06:36:16.152 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:16.169+0000 7fd425ffb640 1 --2- 172.21.15.3:0/238483205 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fd41406be30 0x7fd41406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:16.152 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:16.169+0000 7fd425ffb640 1 -- 172.21.15.3:0/238483205 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fd41c08ade0 con 0x7fd4280d2ed0 2024-05-09T06:36:16.152 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:16.170+0000 7fd42d207640 1 --2- 172.21.15.3:0/238483205 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fd41406be30 0x7fd41406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:16.152 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:16.170+0000 7fd42d207640 1 --2- 172.21.15.3:0/238483205 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fd41406be30 0x7fd41406e2f0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fd418034000 tx=0x7fd41802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:16.154 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:16.174+0000 7fd425ffb640 1 -- 172.21.15.3:0/238483205 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fd41c0596f0 con 0x7fd4280d2ed0 2024-05-09T06:36:16.336 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:16.355+0000 7fd42f46b640 1 -- 172.21.15.3:0/238483205 --> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd4280d1f40 con 0x7fd41406be30 2024-05-09T06:36:16.342 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:16.362+0000 7fd425ffb640 1 -- 172.21.15.3:0/238483205 <== mgr.4105 v2:172.21.15.190:6832/975893321 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251836 (secure 0 0 0) 0x7fd41807f7d0 con 0x7fd41406be30 2024-05-09T06:36:16.345 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-09T06:36:16.347 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2024-05-09T06:36:16.348 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:16.365+0000 7fd42f46b640 1 -- 172.21.15.3:0/238483205 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fd41406be30 msgr2=0x7fd41406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:16.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:16.365+0000 7fd42f46b640 1 --2- 172.21.15.3:0/238483205 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fd41406be30 0x7fd41406e2f0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fd418034000 tx=0x7fd41802e000 comp rx=0 tx=0).stop 2024-05-09T06:36:16.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:16.365+0000 7fd42f46b640 1 -- 172.21.15.3:0/238483205 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd4280d2ed0 msgr2=0x7fd4281e5c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:16.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:16.365+0000 7fd42f46b640 1 --2- 172.21.15.3:0/238483205 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd4280d2ed0 0x7fd4281e5c60 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fd41c00b9c0 tx=0x7fd41c00be80 comp rx=0 tx=0).stop 2024-05-09T06:36:16.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:16.365+0000 7fd42f46b640 1 -- 172.21.15.3:0/238483205 shutdown_connections 2024-05-09T06:36:16.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:16.365+0000 7fd42f46b640 1 --2- 172.21.15.3:0/238483205 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fd41406be30 0x7fd41406e2f0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:16.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:16.365+0000 7fd42f46b640 1 --2- 172.21.15.3:0/238483205 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd42815e480 0x7fd428155520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:16.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:16.365+0000 7fd42f46b640 1 --2- 172.21.15.3:0/238483205 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd4280d2ed0 0x7fd4281e5c60 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:16.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:16.365+0000 7fd42f46b640 1 --2- 172.21.15.3:0/238483205 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd4280d2310 0x7fd4280c9120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:16.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:16.365+0000 7fd42f46b640 1 -- 172.21.15.3:0/238483205 >> 172.21.15.3:0/238483205 conn(0x7fd42809d4d0 msgr2=0x7fd42806bf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:16.350 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:16.365+0000 7fd42f46b640 1 -- 172.21.15.3:0/238483205 shutdown_connections 2024-05-09T06:36:16.350 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:16.365+0000 7fd42f46b640 1 -- 172.21.15.3:0/238483205 wait complete. 2024-05-09T06:36:16.357 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2024-05-09T06:36:14.777499+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218996,"kb_used_data":2824,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755724,"statfs":{"total":773094113280,"available":772869861376,"internally_reserved":0,"allocated":2891776,"data_stored":1753028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.933717"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T06:36:10.066849+0000","last_change":"2024-05-09T06:36:07.870322+0000","last_active":"2024-05-09T06:36:10.066849+0000","last_peered":"2024-05-09T06:36:10.066849+0000","last_clean":"2024-05-09T06:36:10.066849+0000","last_became_active":"2024-05-09T06:36:07.870000+0000","last_became_peered":"2024-05-09T06:36:07.870000+0000","last_unstale":"2024-05-09T06:36:10.066849+0000","last_undegraded":"2024-05-09T06:36:10.066849+0000","last_fullsized":"2024-05-09T06:36:10.066849+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:45:01.680949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T06:36:08.862440+0000","last_change":"2024-05-09T06:36:07.865401+0000","last_active":"2024-05-09T06:36:08.862440+0000","last_peered":"2024-05-09T06:36:08.862440+0000","last_clean":"2024-05-09T06:36:08.862440+0000","last_became_active":"2024-05-09T06:36:07.865119+0000","last_became_peered":"2024-05-09T06:36:07.865119+0000","last_unstale":"2024-05-09T06:36:08.862440+0000","last_undegraded":"2024-05-09T06:36:08.862440+0000","last_fullsized":"2024-05-09T06:36:08.862440+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:19:58.370458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T06:36:11.063203+0000","last_change":"2024-05-09T06:36:07.865504+0000","last_active":"2024-05-09T06:36:11.063203+0000","last_peered":"2024-05-09T06:36:11.063203+0000","last_clean":"2024-05-09T06:36:11.063203+0000","last_became_active":"2024-05-09T06:36:07.864109+0000","last_became_peered":"2024-05-09T06:36:07.864109+0000","last_unstale":"2024-05-09T06:36:11.063203+0000","last_undegraded":"2024-05-09T06:36:11.063203+0000","last_fullsized":"2024-05-09T06:36:11.063203+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:41:33.459622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T06:36:11.063245+0000","last_change":"2024-05-09T06:36:07.879922+0000","last_active":"2024-05-09T06:36:11.063245+0000","last_peered":"2024-05-09T06:36:11.063245+0000","last_clean":"2024-05-09T06:36:11.063245+0000","last_became_active":"2024-05-09T06:36:07.879559+0000","last_became_peered":"2024-05-09T06:36:07.879559+0000","last_unstale":"2024-05-09T06:36:11.063245+0000","last_undegraded":"2024-05-09T06:36:11.063245+0000","last_fullsized":"2024-05-09T06:36:11.063245+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:47:30.595382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T06:36:10.066399+0000","last_change":"2024-05-09T06:36:07.867057+0000","last_active":"2024-05-09T06:36:10.066399+0000","last_peered":"2024-05-09T06:36:10.066399+0000","last_clean":"2024-05-09T06:36:10.066399+0000","last_became_active":"2024-05-09T06:36:07.866657+0000","last_became_peered":"2024-05-09T06:36:07.866657+0000","last_unstale":"2024-05-09T06:36:10.066399+0000","last_undegraded":"2024-05-09T06:36:10.066399+0000","last_fullsized":"2024-05-09T06:36:10.066399+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:25:55.622101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T06:36:11.063093+0000","last_change":"2024-05-09T06:36:07.883141+0000","last_active":"2024-05-09T06:36:11.063093+0000","last_peered":"2024-05-09T06:36:11.063093+0000","last_clean":"2024-05-09T06:36:11.063093+0000","last_became_active":"2024-05-09T06:36:07.882826+0000","last_became_peered":"2024-05-09T06:36:07.882826+0000","last_unstale":"2024-05-09T06:36:11.063093+0000","last_undegraded":"2024-05-09T06:36:11.063093+0000","last_fullsized":"2024-05-09T06:36:11.063093+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:03:16.134498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T06:36:10.066319+0000","last_change":"2024-05-09T06:36:07.876998+0000","last_active":"2024-05-09T06:36:10.066319+0000","last_peered":"2024-05-09T06:36:10.066319+0000","last_clean":"2024-05-09T06:36:10.066319+0000","last_became_active":"2024-05-09T06:36:07.876672+0000","last_became_peered":"2024-05-09T06:36:07.876672+0000","last_unstale":"2024-05-09T06:36:10.066319+0000","last_undegraded":"2024-05-09T06:36:10.066319+0000","last_fullsized":"2024-05-09T06:36:10.066319+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:57:30.164280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T06:36:11.064025+0000","last_change":"2024-05-09T06:36:07.878020+0000","last_active":"2024-05-09T06:36:11.064025+0000","last_peered":"2024-05-09T06:36:11.064025+0000","last_clean":"2024-05-09T06:36:11.064025+0000","last_became_active":"2024-05-09T06:36:07.877767+0000","last_became_peered":"2024-05-09T06:36:07.877767+0000","last_unstale":"2024-05-09T06:36:11.064025+0000","last_undegraded":"2024-05-09T06:36:11.064025+0000","last_fullsized":"2024-05-09T06:36:11.064025+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:47:30.595382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T06:36:10.066848+0000","last_change":"2024-05-09T06:36:07.877214+0000","last_active":"2024-05-09T06:36:10.066848+0000","last_peered":"2024-05-09T06:36:10.066848+0000","last_clean":"2024-05-09T06:36:10.066848+0000","last_became_active":"2024-05-09T06:36:07.876924+0000","last_became_peered":"2024-05-09T06:36:07.876924+0000","last_unstale":"2024-05-09T06:36:10.066848+0000","last_undegraded":"2024-05-09T06:36:10.066848+0000","last_fullsized":"2024-05-09T06:36:10.066848+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:40:20.267663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T06:36:09.855529+0000","last_change":"2024-05-09T06:36:07.879229+0000","last_active":"2024-05-09T06:36:09.855529+0000","last_peered":"2024-05-09T06:36:09.855529+0000","last_clean":"2024-05-09T06:36:09.855529+0000","last_became_active":"2024-05-09T06:36:07.879000+0000","last_became_peered":"2024-05-09T06:36:07.879000+0000","last_unstale":"2024-05-09T06:36:09.855529+0000","last_undegraded":"2024-05-09T06:36:09.855529+0000","last_fullsized":"2024-05-09T06:36:09.855529+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:19:58.370458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T06:36:09.855365+0000","last_change":"2024-05-09T06:36:07.878671+0000","last_active":"2024-05-09T06:36:09.855365+0000","last_peered":"2024-05-09T06:36:09.855365+0000","last_clean":"2024-05-09T06:36:09.855365+0000","last_became_active":"2024-05-09T06:36:07.878389+0000","last_became_peered":"2024-05-09T06:36:07.878389+0000","last_unstale":"2024-05-09T06:36:09.855365+0000","last_undegraded":"2024-05-09T06:36:09.855365+0000","last_fullsized":"2024-05-09T06:36:09.855365+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:34:13.971998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T06:36:08.864278+0000","last_change":"2024-05-09T06:36:07.859955+0000","last_active":"2024-05-09T06:36:08.864278+0000","last_peered":"2024-05-09T06:36:08.864278+0000","last_clean":"2024-05-09T06:36:08.864278+0000","last_became_active":"2024-05-09T06:36:07.859649+0000","last_became_peered":"2024-05-09T06:36:07.859649+0000","last_unstale":"2024-05-09T06:36:08.864278+0000","last_undegraded":"2024-05-09T06:36:08.864278+0000","last_fullsized":"2024-05-09T06:36:08.864278+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:58:19.164284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T06:36:11.063503+0000","last_change":"2024-05-09T06:36:07.870408+0000","last_active":"2024-05-09T06:36:11.063503+0000","last_peered":"2024-05-09T06:36:11.063503+0000","last_clean":"2024-05-09T06:36:11.063503+0000","last_became_active":"2024-05-09T06:36:07.868679+0000","last_became_peered":"2024-05-09T06:36:07.868679+0000","last_unstale":"2024-05-09T06:36:11.063503+0000","last_undegraded":"2024-05-09T06:36:11.063503+0000","last_fullsized":"2024-05-09T06:36:11.063503+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:03:16.134498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T06:36:11.063203+0000","last_change":"2024-05-09T06:36:07.872146+0000","last_active":"2024-05-09T06:36:11.063203+0000","last_peered":"2024-05-09T06:36:11.063203+0000","last_clean":"2024-05-09T06:36:11.063203+0000","last_became_active":"2024-05-09T06:36:07.871858+0000","last_became_peered":"2024-05-09T06:36:07.871858+0000","last_unstale":"2024-05-09T06:36:11.063203+0000","last_undegraded":"2024-05-09T06:36:11.063203+0000","last_fullsized":"2024-05-09T06:36:11.063203+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:44:00.455366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T06:36:08.862179+0000","last_change":"2024-05-09T06:36:07.864771+0000","last_active":"2024-05-09T06:36:08.862179+0000","last_peered":"2024-05-09T06:36:08.862179+0000","last_clean":"2024-05-09T06:36:08.862179+0000","last_became_active":"2024-05-09T06:36:07.864352+0000","last_became_peered":"2024-05-09T06:36:07.864352+0000","last_unstale":"2024-05-09T06:36:08.862179+0000","last_undegraded":"2024-05-09T06:36:08.862179+0000","last_fullsized":"2024-05-09T06:36:08.862179+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:34:13.971998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T06:36:11.062927+0000","last_change":"2024-05-09T06:36:07.883189+0000","last_active":"2024-05-09T06:36:11.062927+0000","last_peered":"2024-05-09T06:36:11.062927+0000","last_clean":"2024-05-09T06:36:11.062927+0000","last_became_active":"2024-05-09T06:36:07.882906+0000","last_became_peered":"2024-05-09T06:36:07.882906+0000","last_unstale":"2024-05-09T06:36:11.062927+0000","last_undegraded":"2024-05-09T06:36:11.062927+0000","last_fullsized":"2024-05-09T06:36:11.062927+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:19:58.370458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T06:36:11.063637+0000","last_change":"2024-05-09T06:36:07.878966+0000","last_active":"2024-05-09T06:36:11.063637+0000","last_peered":"2024-05-09T06:36:11.063637+0000","last_clean":"2024-05-09T06:36:11.063637+0000","last_became_active":"2024-05-09T06:36:07.878678+0000","last_became_peered":"2024-05-09T06:36:07.878678+0000","last_unstale":"2024-05-09T06:36:11.063637+0000","last_undegraded":"2024-05-09T06:36:11.063637+0000","last_fullsized":"2024-05-09T06:36:11.063637+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:40:20.267663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T06:36:11.063413+0000","last_change":"2024-05-09T06:36:07.866493+0000","last_active":"2024-05-09T06:36:11.063413+0000","last_peered":"2024-05-09T06:36:11.063413+0000","last_clean":"2024-05-09T06:36:11.063413+0000","last_became_active":"2024-05-09T06:36:07.866157+0000","last_became_peered":"2024-05-09T06:36:07.866157+0000","last_unstale":"2024-05-09T06:36:11.063413+0000","last_undegraded":"2024-05-09T06:36:11.063413+0000","last_fullsized":"2024-05-09T06:36:11.063413+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:03:39.564328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T06:36:10.066784+0000","last_change":"2024-05-09T06:36:07.866352+0000","last_active":"2024-05-09T06:36:10.066784+0000","last_peered":"2024-05-09T06:36:10.066784+0000","last_clean":"2024-05-09T06:36:10.066784+0000","last_became_active":"2024-05-09T06:36:07.866066+0000","last_became_peered":"2024-05-09T06:36:07.866066+0000","last_unstale":"2024-05-09T06:36:10.066784+0000","last_undegraded":"2024-05-09T06:36:10.066784+0000","last_fullsized":"2024-05-09T06:36:10.066784+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:17:19.091125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T06:36:09.855483+0000","last_change":"2024-05-09T06:36:07.858555+0000","last_active":"2024-05-09T06:36:09.855483+0000","last_peered":"2024-05-09T06:36:09.855483+0000","last_clean":"2024-05-09T06:36:09.855483+0000","last_became_active":"2024-05-09T06:36:07.858259+0000","last_became_peered":"2024-05-09T06:36:07.858259+0000","last_unstale":"2024-05-09T06:36:09.855483+0000","last_undegraded":"2024-05-09T06:36:09.855483+0000","last_fullsized":"2024-05-09T06:36:09.855483+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:50:02.245694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T06:36:11.063981+0000","last_change":"2024-05-09T06:36:07.875125+0000","last_active":"2024-05-09T06:36:11.063981+0000","last_peered":"2024-05-09T06:36:11.063981+0000","last_clean":"2024-05-09T06:36:11.063981+0000","last_became_active":"2024-05-09T06:36:07.874432+0000","last_became_peered":"2024-05-09T06:36:07.874432+0000","last_unstale":"2024-05-09T06:36:11.063981+0000","last_undegraded":"2024-05-09T06:36:11.063981+0000","last_fullsized":"2024-05-09T06:36:11.063981+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:24:50.227387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T06:36:11.063112+0000","last_change":"2024-05-09T06:36:07.876701+0000","last_active":"2024-05-09T06:36:11.063112+0000","last_peered":"2024-05-09T06:36:11.063112+0000","last_clean":"2024-05-09T06:36:11.063112+0000","last_became_active":"2024-05-09T06:36:07.876416+0000","last_became_peered":"2024-05-09T06:36:07.876416+0000","last_unstale":"2024-05-09T06:36:11.063112+0000","last_undegraded":"2024-05-09T06:36:11.063112+0000","last_fullsized":"2024-05-09T06:36:11.063112+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:57:30.164280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T06:36:11.063138+0000","last_change":"2024-05-09T06:36:07.868404+0000","last_active":"2024-05-09T06:36:11.063138+0000","last_peered":"2024-05-09T06:36:11.063138+0000","last_clean":"2024-05-09T06:36:11.063138+0000","last_became_active":"2024-05-09T06:36:07.867909+0000","last_became_peered":"2024-05-09T06:36:07.867909+0000","last_unstale":"2024-05-09T06:36:11.063138+0000","last_undegraded":"2024-05-09T06:36:11.063138+0000","last_fullsized":"2024-05-09T06:36:11.063138+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:24:51.534066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T06:36:11.063002+0000","last_change":"2024-05-09T06:36:07.880411+0000","last_active":"2024-05-09T06:36:11.063002+0000","last_peered":"2024-05-09T06:36:11.063002+0000","last_clean":"2024-05-09T06:36:11.063002+0000","last_became_active":"2024-05-09T06:36:07.879887+0000","last_became_peered":"2024-05-09T06:36:07.879887+0000","last_unstale":"2024-05-09T06:36:11.063002+0000","last_undegraded":"2024-05-09T06:36:11.063002+0000","last_fullsized":"2024-05-09T06:36:11.063002+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:24:50.227387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T06:36:11.063293+0000","last_change":"2024-05-09T06:36:07.868320+0000","last_active":"2024-05-09T06:36:11.063293+0000","last_peered":"2024-05-09T06:36:11.063293+0000","last_clean":"2024-05-09T06:36:11.063293+0000","last_became_active":"2024-05-09T06:36:07.867969+0000","last_became_peered":"2024-05-09T06:36:07.867969+0000","last_unstale":"2024-05-09T06:36:11.063293+0000","last_undegraded":"2024-05-09T06:36:11.063293+0000","last_fullsized":"2024-05-09T06:36:11.063293+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:50:02.245694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T06:36:09.855614+0000","last_change":"2024-05-09T06:36:07.869956+0000","last_active":"2024-05-09T06:36:09.855614+0000","last_peered":"2024-05-09T06:36:09.855614+0000","last_clean":"2024-05-09T06:36:09.855614+0000","last_became_active":"2024-05-09T06:36:07.869695+0000","last_became_peered":"2024-05-09T06:36:07.869695+0000","last_unstale":"2024-05-09T06:36:09.855614+0000","last_undegraded":"2024-05-09T06:36:09.855614+0000","last_fullsized":"2024-05-09T06:36:09.855614+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:24:50.227387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T06:36:08.862523+0000","last_change":"2024-05-09T06:36:07.867102+0000","last_active":"2024-05-09T06:36:08.862523+0000","last_peered":"2024-05-09T06:36:08.862523+0000","last_clean":"2024-05-09T06:36:08.862523+0000","last_became_active":"2024-05-09T06:36:07.866785+0000","last_became_peered":"2024-05-09T06:36:07.866785+0000","last_unstale":"2024-05-09T06:36:08.862523+0000","last_undegraded":"2024-05-09T06:36:08.862523+0000","last_fullsized":"2024-05-09T06:36:08.862523+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:24:50.227387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T06:36:11.063773+0000","last_change":"2024-05-09T06:36:07.871788+0000","last_active":"2024-05-09T06:36:11.063773+0000","last_peered":"2024-05-09T06:36:11.063773+0000","last_clean":"2024-05-09T06:36:11.063773+0000","last_became_active":"2024-05-09T06:36:07.871500+0000","last_became_peered":"2024-05-09T06:36:07.871500+0000","last_unstale":"2024-05-09T06:36:11.063773+0000","last_undegraded":"2024-05-09T06:36:11.063773+0000","last_fullsized":"2024-05-09T06:36:11.063773+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:57:30.164280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T06:36:11.063659+0000","last_change":"2024-05-09T06:36:07.876623+0000","last_active":"2024-05-09T06:36:11.063659+0000","last_peered":"2024-05-09T06:36:11.063659+0000","last_clean":"2024-05-09T06:36:11.063659+0000","last_became_active":"2024-05-09T06:36:07.876358+0000","last_became_peered":"2024-05-09T06:36:07.876358+0000","last_unstale":"2024-05-09T06:36:11.063659+0000","last_undegraded":"2024-05-09T06:36:11.063659+0000","last_fullsized":"2024-05-09T06:36:11.063659+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:34:13.971998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T06:36:11.063353+0000","last_change":"2024-05-09T06:36:07.879149+0000","last_active":"2024-05-09T06:36:11.063353+0000","last_peered":"2024-05-09T06:36:11.063353+0000","last_clean":"2024-05-09T06:36:11.063353+0000","last_became_active":"2024-05-09T06:36:07.878868+0000","last_became_peered":"2024-05-09T06:36:07.878868+0000","last_unstale":"2024-05-09T06:36:11.063353+0000","last_undegraded":"2024-05-09T06:36:11.063353+0000","last_fullsized":"2024-05-09T06:36:11.063353+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:56:41.722475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T06:36:11.062828+0000","last_change":"2024-05-09T06:36:07.876443+0000","last_active":"2024-05-09T06:36:11.062828+0000","last_peered":"2024-05-09T06:36:11.062828+0000","last_clean":"2024-05-09T06:36:11.062828+0000","last_became_active":"2024-05-09T06:36:07.876122+0000","last_became_peered":"2024-05-09T06:36:07.876122+0000","last_unstale":"2024-05-09T06:36:11.062828+0000","last_undegraded":"2024-05-09T06:36:11.062828+0000","last_fullsized":"2024-05-09T06:36:11.062828+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:58:19.164284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T06:36:09.855285+0000","last_change":"2024-05-09T06:36:07.867899+0000","last_active":"2024-05-09T06:36:09.855285+0000","last_peered":"2024-05-09T06:36:09.855285+0000","last_clean":"2024-05-09T06:36:09.855285+0000","last_became_active":"2024-05-09T06:36:07.867371+0000","last_became_peered":"2024-05-09T06:36:07.867371+0000","last_unstale":"2024-05-09T06:36:09.855285+0000","last_undegraded":"2024-05-09T06:36:09.855285+0000","last_fullsized":"2024-05-09T06:36:09.855285+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:44:00.455366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T06:36:08.863886+0000","last_change":"2024-05-09T06:36:07.871355+0000","last_active":"2024-05-09T06:36:08.863886+0000","last_peered":"2024-05-09T06:36:08.863886+0000","last_clean":"2024-05-09T06:36:08.863886+0000","last_became_active":"2024-05-09T06:36:07.871025+0000","last_became_peered":"2024-05-09T06:36:07.871025+0000","last_unstale":"2024-05-09T06:36:08.863886+0000","last_undegraded":"2024-05-09T06:36:08.863886+0000","last_fullsized":"2024-05-09T06:36:08.863886+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:40:42.090918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T06:36:11.064068+0000","last_change":"2024-05-09T06:36:06.865248+0000","last_active":"2024-05-09T06:36:11.064068+0000","last_peered":"2024-05-09T06:36:11.064068+0000","last_clean":"2024-05-09T06:36:11.064068+0000","last_became_active":"2024-05-09T06:36:06.864940+0000","last_became_peered":"2024-05-09T06:36:06.864940+0000","last_unstale":"2024-05-09T06:36:11.064068+0000","last_undegraded":"2024-05-09T06:36:11.064068+0000","last_fullsized":"2024-05-09T06:36:11.064068+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_clean_scrub_stamp":"2024-05-09T06:36:05.833229+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:08:20.004456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T06:36:11.063797+0000","last_change":"2024-05-09T06:36:07.874223+0000","last_active":"2024-05-09T06:36:11.063797+0000","last_peered":"2024-05-09T06:36:11.063797+0000","last_clean":"2024-05-09T06:36:11.063797+0000","last_became_active":"2024-05-09T06:36:07.873787+0000","last_became_peered":"2024-05-09T06:36:07.873787+0000","last_unstale":"2024-05-09T06:36:11.063797+0000","last_undegraded":"2024-05-09T06:36:11.063797+0000","last_fullsized":"2024-05-09T06:36:11.063797+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:51:24.938206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T06:36:09.855173+0000","last_change":"2024-05-09T06:36:06.862586+0000","last_active":"2024-05-09T06:36:09.855173+0000","last_peered":"2024-05-09T06:36:09.855173+0000","last_clean":"2024-05-09T06:36:09.855173+0000","last_became_active":"2024-05-09T06:36:06.861545+0000","last_became_peered":"2024-05-09T06:36:06.861545+0000","last_unstale":"2024-05-09T06:36:09.855173+0000","last_undegraded":"2024-05-09T06:36:09.855173+0000","last_fullsized":"2024-05-09T06:36:09.855173+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_clean_scrub_stamp":"2024-05-09T06:36:05.833229+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:15:49.248390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T06:36:11.063612+0000","last_change":"2024-05-09T06:36:06.867498+0000","last_active":"2024-05-09T06:36:11.063612+0000","last_peered":"2024-05-09T06:36:11.063612+0000","last_clean":"2024-05-09T06:36:11.063612+0000","last_became_active":"2024-05-09T06:36:06.867177+0000","last_became_peered":"2024-05-09T06:36:06.867177+0000","last_unstale":"2024-05-09T06:36:11.063612+0000","last_undegraded":"2024-05-09T06:36:11.063612+0000","last_fullsized":"2024-05-09T06:36:11.063612+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_clean_scrub_stamp":"2024-05-09T06:36:05.833229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:23:13.976894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T06:36:11.063580+0000","last_change":"2024-05-09T06:36:07.871392+0000","last_active":"2024-05-09T06:36:11.063580+0000","last_peered":"2024-05-09T06:36:11.063580+0000","last_clean":"2024-05-09T06:36:11.063580+0000","last_became_active":"2024-05-09T06:36:07.871103+0000","last_became_peered":"2024-05-09T06:36:07.871103+0000","last_unstale":"2024-05-09T06:36:11.063580+0000","last_undegraded":"2024-05-09T06:36:11.063580+0000","last_fullsized":"2024-05-09T06:36:11.063580+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:19:58.370458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T06:36:11.063055+0000","last_change":"2024-05-09T06:36:06.865012+0000","last_active":"2024-05-09T06:36:11.063055+0000","last_peered":"2024-05-09T06:36:11.063055+0000","last_clean":"2024-05-09T06:36:11.063055+0000","last_became_active":"2024-05-09T06:36:06.864434+0000","last_became_peered":"2024-05-09T06:36:06.864434+0000","last_unstale":"2024-05-09T06:36:11.063055+0000","last_undegraded":"2024-05-09T06:36:11.063055+0000","last_fullsized":"2024-05-09T06:36:11.063055+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_clean_scrub_stamp":"2024-05-09T06:36:05.833229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:16:34.559063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T06:36:08.862189+0000","last_change":"2024-05-09T06:36:07.870720+0000","last_active":"2024-05-09T06:36:08.862189+0000","last_peered":"2024-05-09T06:36:08.862189+0000","last_clean":"2024-05-09T06:36:08.862189+0000","last_became_active":"2024-05-09T06:36:07.870431+0000","last_became_peered":"2024-05-09T06:36:07.870431+0000","last_unstale":"2024-05-09T06:36:08.862189+0000","last_undegraded":"2024-05-09T06:36:08.862189+0000","last_fullsized":"2024-05-09T06:36:08.862189+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:51:24.938206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T06:36:09.855474+0000","last_change":"2024-05-09T06:36:06.866523+0000","last_active":"2024-05-09T06:36:09.855474+0000","last_peered":"2024-05-09T06:36:09.855474+0000","last_clean":"2024-05-09T06:36:09.855474+0000","last_became_active":"2024-05-09T06:36:06.866220+0000","last_became_peered":"2024-05-09T06:36:06.866220+0000","last_unstale":"2024-05-09T06:36:09.855474+0000","last_undegraded":"2024-05-09T06:36:09.855474+0000","last_fullsized":"2024-05-09T06:36:09.855474+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_clean_scrub_stamp":"2024-05-09T06:36:05.833229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:40:41.085704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T06:36:11.063367+0000","last_change":"2024-05-09T06:36:07.877537+0000","last_active":"2024-05-09T06:36:11.063367+0000","last_peered":"2024-05-09T06:36:11.063367+0000","last_clean":"2024-05-09T06:36:11.063367+0000","last_became_active":"2024-05-09T06:36:07.877261+0000","last_became_peered":"2024-05-09T06:36:07.877261+0000","last_unstale":"2024-05-09T06:36:11.063367+0000","last_undegraded":"2024-05-09T06:36:11.063367+0000","last_fullsized":"2024-05-09T06:36:11.063367+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:24:23.190085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T06:36:11.063314+0000","last_change":"2024-05-09T06:36:06.864863+0000","last_active":"2024-05-09T06:36:11.063314+0000","last_peered":"2024-05-09T06:36:11.063314+0000","last_clean":"2024-05-09T06:36:11.063314+0000","last_became_active":"2024-05-09T06:36:06.864300+0000","last_became_peered":"2024-05-09T06:36:06.864300+0000","last_unstale":"2024-05-09T06:36:11.063314+0000","last_undegraded":"2024-05-09T06:36:11.063314+0000","last_fullsized":"2024-05-09T06:36:11.063314+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_clean_scrub_stamp":"2024-05-09T06:36:05.833229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:15:49.248390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T06:36:11.063876+0000","last_change":"2024-05-09T06:36:07.877642+0000","last_active":"2024-05-09T06:36:11.063876+0000","last_peered":"2024-05-09T06:36:11.063876+0000","last_clean":"2024-05-09T06:36:11.063876+0000","last_became_active":"2024-05-09T06:36:07.877358+0000","last_became_peered":"2024-05-09T06:36:07.877358+0000","last_unstale":"2024-05-09T06:36:11.063876+0000","last_undegraded":"2024-05-09T06:36:11.063876+0000","last_fullsized":"2024-05-09T06:36:11.063876+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:19:58.370458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T06:36:08.862471+0000","last_change":"2024-05-09T06:36:06.865905+0000","last_active":"2024-05-09T06:36:08.862471+0000","last_peered":"2024-05-09T06:36:08.862471+0000","last_clean":"2024-05-09T06:36:08.862471+0000","last_became_active":"2024-05-09T06:36:06.865528+0000","last_became_peered":"2024-05-09T06:36:06.865528+0000","last_unstale":"2024-05-09T06:36:08.862471+0000","last_undegraded":"2024-05-09T06:36:08.862471+0000","last_fullsized":"2024-05-09T06:36:08.862471+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_clean_scrub_stamp":"2024-05-09T06:36:05.833229+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:40:41.085704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T06:36:08.864334+0000","last_change":"2024-05-09T06:36:07.864517+0000","last_active":"2024-05-09T06:36:08.864334+0000","last_peered":"2024-05-09T06:36:08.864334+0000","last_clean":"2024-05-09T06:36:08.864334+0000","last_became_active":"2024-05-09T06:36:07.864076+0000","last_became_peered":"2024-05-09T06:36:07.864076+0000","last_unstale":"2024-05-09T06:36:08.864334+0000","last_undegraded":"2024-05-09T06:36:08.864334+0000","last_fullsized":"2024-05-09T06:36:08.864334+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:03:39.564328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T06:36:09.855507+0000","last_change":"2024-05-09T06:36:06.862854+0000","last_active":"2024-05-09T06:36:09.855507+0000","last_peered":"2024-05-09T06:36:09.855507+0000","last_clean":"2024-05-09T06:36:09.855507+0000","last_became_active":"2024-05-09T06:36:06.862538+0000","last_became_peered":"2024-05-09T06:36:06.862538+0000","last_unstale":"2024-05-09T06:36:09.855507+0000","last_undegraded":"2024-05-09T06:36:09.855507+0000","last_fullsized":"2024-05-09T06:36:09.855507+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_clean_scrub_stamp":"2024-05-09T06:36:05.833229+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:08:20.004456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T06:36:10.066521+0000","last_change":"2024-05-09T06:36:07.872477+0000","last_active":"2024-05-09T06:36:10.066521+0000","last_peered":"2024-05-09T06:36:10.066521+0000","last_clean":"2024-05-09T06:36:10.066521+0000","last_became_active":"2024-05-09T06:36:07.872181+0000","last_became_peered":"2024-05-09T06:36:07.872181+0000","last_unstale":"2024-05-09T06:36:10.066521+0000","last_undegraded":"2024-05-09T06:36:10.066521+0000","last_fullsized":"2024-05-09T06:36:10.066521+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:24:23.190085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T06:36:11.063897+0000","last_change":"2024-05-09T06:36:06.861757+0000","last_active":"2024-05-09T06:36:11.063897+0000","last_peered":"2024-05-09T06:36:11.063897+0000","last_clean":"2024-05-09T06:36:11.063897+0000","last_became_active":"2024-05-09T06:36:06.861434+0000","last_became_peered":"2024-05-09T06:36:06.861434+0000","last_unstale":"2024-05-09T06:36:11.063897+0000","last_undegraded":"2024-05-09T06:36:11.063897+0000","last_fullsized":"2024-05-09T06:36:11.063897+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_clean_scrub_stamp":"2024-05-09T06:36:05.833229+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:56:44.324093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T06:36:11.063318+0000","last_change":"2024-05-09T06:36:07.873646+0000","last_active":"2024-05-09T06:36:11.063318+0000","last_peered":"2024-05-09T06:36:11.063318+0000","last_clean":"2024-05-09T06:36:11.063318+0000","last_became_active":"2024-05-09T06:36:07.872272+0000","last_became_peered":"2024-05-09T06:36:07.872272+0000","last_unstale":"2024-05-09T06:36:11.063318+0000","last_undegraded":"2024-05-09T06:36:11.063318+0000","last_fullsized":"2024-05-09T06:36:11.063318+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:53:50.215704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T06:36:11.063298+0000","last_change":"2024-05-09T06:36:06.865709+0000","last_active":"2024-05-09T06:36:11.063298+0000","last_peered":"2024-05-09T06:36:11.063298+0000","last_clean":"2024-05-09T06:36:11.063298+0000","last_became_active":"2024-05-09T06:36:06.864907+0000","last_became_peered":"2024-05-09T06:36:06.864907+0000","last_unstale":"2024-05-09T06:36:11.063298+0000","last_undegraded":"2024-05-09T06:36:11.063298+0000","last_fullsized":"2024-05-09T06:36:11.063298+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_clean_scrub_stamp":"2024-05-09T06:36:05.833229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:40:41.085704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T06:36:10.066476+0000","last_change":"2024-05-09T06:36:07.873795+0000","last_active":"2024-05-09T06:36:10.066476+0000","last_peered":"2024-05-09T06:36:10.066476+0000","last_clean":"2024-05-09T06:36:10.066476+0000","last_became_active":"2024-05-09T06:36:07.873491+0000","last_became_peered":"2024-05-09T06:36:07.873491+0000","last_unstale":"2024-05-09T06:36:10.066476+0000","last_undegraded":"2024-05-09T06:36:10.066476+0000","last_fullsized":"2024-05-09T06:36:10.066476+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:56:41.722475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T06:36:09.855141+0000","last_change":"2024-05-09T06:36:06.862067+0000","last_active":"2024-05-09T06:36:09.855141+0000","last_peered":"2024-05-09T06:36:09.855141+0000","last_clean":"2024-05-09T06:36:09.855141+0000","last_became_active":"2024-05-09T06:36:06.861791+0000","last_became_peered":"2024-05-09T06:36:06.861791+0000","last_unstale":"2024-05-09T06:36:09.855141+0000","last_undegraded":"2024-05-09T06:36:09.855141+0000","last_fullsized":"2024-05-09T06:36:09.855141+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_clean_scrub_stamp":"2024-05-09T06:36:05.833229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:03:17.189398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T06:36:09.855531+0000","last_change":"2024-05-09T06:36:07.867637+0000","last_active":"2024-05-09T06:36:09.855531+0000","last_peered":"2024-05-09T06:36:09.855531+0000","last_clean":"2024-05-09T06:36:09.855531+0000","last_became_active":"2024-05-09T06:36:07.867028+0000","last_became_peered":"2024-05-09T06:36:07.867028+0000","last_unstale":"2024-05-09T06:36:09.855531+0000","last_undegraded":"2024-05-09T06:36:09.855531+0000","last_fullsized":"2024-05-09T06:36:09.855531+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:51:24.938206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":22,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T06:36:11.063615+0000","last_change":"2024-05-09T06:36:06.852528+0000","last_active":"2024-05-09T06:36:11.063615+0000","last_peered":"2024-05-09T06:36:11.063615+0000","last_clean":"2024-05-09T06:36:11.063615+0000","last_became_active":"2024-05-09T06:36:06.852202+0000","last_became_peered":"2024-05-09T06:36:06.852202+0000","last_unstale":"2024-05-09T06:36:11.063615+0000","last_undegraded":"2024-05-09T06:36:11.063615+0000","last_fullsized":"2024-05-09T06:36:11.063615+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_clean_scrub_stamp":"2024-05-09T06:36:05.833229+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:08:20.004456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T06:36:10.066312+0000","last_change":"2024-05-09T06:36:07.867532+0000","last_active":"2024-05-09T06:36:10.066312+0000","last_peered":"2024-05-09T06:36:10.066312+0000","last_clean":"2024-05-09T06:36:10.066312+0000","last_became_active":"2024-05-09T06:36:07.867262+0000","last_became_peered":"2024-05-09T06:36:07.867262+0000","last_unstale":"2024-05-09T06:36:10.066312+0000","last_undegraded":"2024-05-09T06:36:10.066312+0000","last_fullsized":"2024-05-09T06:36:10.066312+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:03:16.134498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T06:36:08.864156+0000","last_change":"2024-05-09T06:36:06.863522+0000","last_active":"2024-05-09T06:36:08.864156+0000","last_peered":"2024-05-09T06:36:08.864156+0000","last_clean":"2024-05-09T06:36:08.864156+0000","last_became_active":"2024-05-09T06:36:06.863099+0000","last_became_peered":"2024-05-09T06:36:06.863099+0000","last_unstale":"2024-05-09T06:36:08.864156+0000","last_undegraded":"2024-05-09T06:36:08.864156+0000","last_fullsized":"2024-05-09T06:36:08.864156+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_clean_scrub_stamp":"2024-05-09T06:36:05.833229+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:56:44.324093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T06:36:10.066683+0000","last_change":"2024-05-09T06:36:07.868102+0000","last_active":"2024-05-09T06:36:10.066683+0000","last_peered":"2024-05-09T06:36:10.066683+0000","last_clean":"2024-05-09T06:36:10.066683+0000","last_became_active":"2024-05-09T06:36:07.867800+0000","last_became_peered":"2024-05-09T06:36:07.867800+0000","last_unstale":"2024-05-09T06:36:10.066683+0000","last_undegraded":"2024-05-09T06:36:10.066683+0000","last_fullsized":"2024-05-09T06:36:10.066683+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:10:28.010336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T06:36:11.063508+0000","last_change":"2024-05-09T06:36:06.861458+0000","last_active":"2024-05-09T06:36:11.063508+0000","last_peered":"2024-05-09T06:36:11.063508+0000","last_clean":"2024-05-09T06:36:11.063508+0000","last_became_active":"2024-05-09T06:36:06.861098+0000","last_became_peered":"2024-05-09T06:36:06.861098+0000","last_unstale":"2024-05-09T06:36:11.063508+0000","last_undegraded":"2024-05-09T06:36:11.063508+0000","last_fullsized":"2024-05-09T06:36:11.063508+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_clean_scrub_stamp":"2024-05-09T06:36:05.833229+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:15:49.248390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T06:36:10.066945+0000","last_change":"2024-05-09T06:36:07.868487+0000","last_active":"2024-05-09T06:36:10.066945+0000","last_peered":"2024-05-09T06:36:10.066945+0000","last_clean":"2024-05-09T06:36:10.066945+0000","last_became_active":"2024-05-09T06:36:07.867705+0000","last_became_peered":"2024-05-09T06:36:07.867705+0000","last_unstale":"2024-05-09T06:36:10.066945+0000","last_undegraded":"2024-05-09T06:36:10.066945+0000","last_fullsized":"2024-05-09T06:36:10.066945+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:04:27.234790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T06:36:08.862567+0000","last_change":"2024-05-09T06:36:06.858041+0000","last_active":"2024-05-09T06:36:08.862567+0000","last_peered":"2024-05-09T06:36:08.862567+0000","last_clean":"2024-05-09T06:36:08.862567+0000","last_became_active":"2024-05-09T06:36:06.857745+0000","last_became_peered":"2024-05-09T06:36:06.857745+0000","last_unstale":"2024-05-09T06:36:08.862567+0000","last_undegraded":"2024-05-09T06:36:08.862567+0000","last_fullsized":"2024-05-09T06:36:08.862567+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_clean_scrub_stamp":"2024-05-09T06:36:05.833229+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:58:18.159070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T06:36:11.063484+0000","last_change":"2024-05-09T06:36:07.876459+0000","last_active":"2024-05-09T06:36:11.063484+0000","last_peered":"2024-05-09T06:36:11.063484+0000","last_clean":"2024-05-09T06:36:11.063484+0000","last_became_active":"2024-05-09T06:36:07.876151+0000","last_became_peered":"2024-05-09T06:36:07.876151+0000","last_unstale":"2024-05-09T06:36:11.063484+0000","last_undegraded":"2024-05-09T06:36:11.063484+0000","last_fullsized":"2024-05-09T06:36:11.063484+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:45:01.680949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T06:36:11.062797+0000","last_change":"2024-05-09T06:36:06.866239+0000","last_active":"2024-05-09T06:36:11.062797+0000","last_peered":"2024-05-09T06:36:11.062797+0000","last_clean":"2024-05-09T06:36:11.062797+0000","last_became_active":"2024-05-09T06:36:06.865929+0000","last_became_peered":"2024-05-09T06:36:06.865929+0000","last_unstale":"2024-05-09T06:36:11.062797+0000","last_undegraded":"2024-05-09T06:36:11.062797+0000","last_fullsized":"2024-05-09T06:36:11.062797+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_clean_scrub_stamp":"2024-05-09T06:36:05.833229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:03:17.189398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T06:36:10.066456+0000","last_change":"2024-05-09T06:36:07.877578+0000","last_active":"2024-05-09T06:36:10.066456+0000","last_peered":"2024-05-09T06:36:10.066456+0000","last_clean":"2024-05-09T06:36:10.066456+0000","last_became_active":"2024-05-09T06:36:07.877319+0000","last_became_peered":"2024-05-09T06:36:07.877319+0000","last_unstale":"2024-05-09T06:36:10.066456+0000","last_undegraded":"2024-05-09T06:36:10.066456+0000","last_fullsized":"2024-05-09T06:36:10.066456+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:47:30.595382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":170,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T06:36:10.066325+0000","last_change":"2024-05-09T06:35:55.895214+0000","last_active":"2024-05-09T06:36:10.066325+0000","last_peered":"2024-05-09T06:36:10.066325+0000","last_clean":"2024-05-09T06:36:10.066325+0000","last_became_active":"2024-05-09T06:35:55.894087+0000","last_became_peered":"2024-05-09T06:35:55.894087+0000","last_unstale":"2024-05-09T06:36:10.066325+0000","last_undegraded":"2024-05-09T06:36:10.066325+0000","last_fullsized":"2024-05-09T06:36:10.066325+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:35:54.820102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:35:54.820102+0000","last_clean_scrub_stamp":"2024-05-09T06:35:54.820102+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:29:20.408099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T06:36:11.062960+0000","last_change":"2024-05-09T06:36:06.864900+0000","last_active":"2024-05-09T06:36:11.062960+0000","last_peered":"2024-05-09T06:36:11.062960+0000","last_clean":"2024-05-09T06:36:11.062960+0000","last_became_active":"2024-05-09T06:36:06.864169+0000","last_became_peered":"2024-05-09T06:36:06.864169+0000","last_unstale":"2024-05-09T06:36:11.062960+0000","last_undegraded":"2024-05-09T06:36:11.062960+0000","last_fullsized":"2024-05-09T06:36:11.062960+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_clean_scrub_stamp":"2024-05-09T06:36:05.833229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:03:17.189398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T06:36:11.063181+0000","last_change":"2024-05-09T06:36:07.880436+0000","last_active":"2024-05-09T06:36:11.063181+0000","last_peered":"2024-05-09T06:36:11.063181+0000","last_clean":"2024-05-09T06:36:11.063181+0000","last_became_active":"2024-05-09T06:36:07.879931+0000","last_became_peered":"2024-05-09T06:36:07.879931+0000","last_unstale":"2024-05-09T06:36:11.063181+0000","last_undegraded":"2024-05-09T06:36:11.063181+0000","last_fullsized":"2024-05-09T06:36:11.063181+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:34:13.971998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T06:36:10.066153+0000","last_change":"2024-05-09T06:36:06.860567+0000","last_active":"2024-05-09T06:36:10.066153+0000","last_peered":"2024-05-09T06:36:10.066153+0000","last_clean":"2024-05-09T06:36:10.066153+0000","last_became_active":"2024-05-09T06:36:06.860284+0000","last_became_peered":"2024-05-09T06:36:06.860284+0000","last_unstale":"2024-05-09T06:36:10.066153+0000","last_undegraded":"2024-05-09T06:36:10.066153+0000","last_fullsized":"2024-05-09T06:36:10.066153+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_clean_scrub_stamp":"2024-05-09T06:36:05.833229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:03:38.559114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T06:36:11.062933+0000","last_change":"2024-05-09T06:36:07.865957+0000","last_active":"2024-05-09T06:36:11.062933+0000","last_peered":"2024-05-09T06:36:11.062933+0000","last_clean":"2024-05-09T06:36:11.062933+0000","last_became_active":"2024-05-09T06:36:07.865683+0000","last_became_peered":"2024-05-09T06:36:07.865683+0000","last_unstale":"2024-05-09T06:36:11.062933+0000","last_undegraded":"2024-05-09T06:36:11.062933+0000","last_fullsized":"2024-05-09T06:36:11.062933+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:47:56.779394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T06:36:11.062927+0000","last_change":"2024-05-09T06:36:06.862232+0000","last_active":"2024-05-09T06:36:11.062927+0000","last_peered":"2024-05-09T06:36:11.062927+0000","last_clean":"2024-05-09T06:36:11.062927+0000","last_became_active":"2024-05-09T06:36:06.861918+0000","last_became_peered":"2024-05-09T06:36:06.861918+0000","last_unstale":"2024-05-09T06:36:11.062927+0000","last_undegraded":"2024-05-09T06:36:11.062927+0000","last_fullsized":"2024-05-09T06:36:11.062927+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_clean_scrub_stamp":"2024-05-09T06:36:05.833229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:23:13.976894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T06:36:08.862162+0000","last_change":"2024-05-09T06:36:07.865850+0000","last_active":"2024-05-09T06:36:08.862162+0000","last_peered":"2024-05-09T06:36:08.862162+0000","last_clean":"2024-05-09T06:36:08.862162+0000","last_became_active":"2024-05-09T06:36:07.865520+0000","last_became_peered":"2024-05-09T06:36:07.865520+0000","last_unstale":"2024-05-09T06:36:08.862162+0000","last_undegraded":"2024-05-09T06:36:08.862162+0000","last_fullsized":"2024-05-09T06:36:08.862162+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:44:00.455366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T06:36:11.063482+0000","last_change":"2024-05-09T06:36:06.865545+0000","last_active":"2024-05-09T06:36:11.063482+0000","last_peered":"2024-05-09T06:36:11.063482+0000","last_clean":"2024-05-09T06:36:11.063482+0000","last_became_active":"2024-05-09T06:36:06.865255+0000","last_became_peered":"2024-05-09T06:36:06.865255+0000","last_unstale":"2024-05-09T06:36:11.063482+0000","last_undegraded":"2024-05-09T06:36:11.063482+0000","last_fullsized":"2024-05-09T06:36:11.063482+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_clean_scrub_stamp":"2024-05-09T06:36:05.833229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:59:29.736305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T06:36:08.864221+0000","last_change":"2024-05-09T06:36:07.867264+0000","last_active":"2024-05-09T06:36:08.864221+0000","last_peered":"2024-05-09T06:36:08.864221+0000","last_clean":"2024-05-09T06:36:08.864221+0000","last_became_active":"2024-05-09T06:36:07.866981+0000","last_became_peered":"2024-05-09T06:36:07.866981+0000","last_unstale":"2024-05-09T06:36:08.864221+0000","last_undegraded":"2024-05-09T06:36:08.864221+0000","last_fullsized":"2024-05-09T06:36:08.864221+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:16:35.564277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T06:36:08.862365+0000","last_change":"2024-05-09T06:36:06.853399+0000","last_active":"2024-05-09T06:36:08.862365+0000","last_peered":"2024-05-09T06:36:08.862365+0000","last_clean":"2024-05-09T06:36:08.862365+0000","last_became_active":"2024-05-09T06:36:06.852409+0000","last_became_peered":"2024-05-09T06:36:06.852409+0000","last_unstale":"2024-05-09T06:36:08.862365+0000","last_undegraded":"2024-05-09T06:36:08.862365+0000","last_fullsized":"2024-05-09T06:36:08.862365+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_clean_scrub_stamp":"2024-05-09T06:36:05.833229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:15:49.248390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T06:36:11.063331+0000","last_change":"2024-05-09T06:36:07.875455+0000","last_active":"2024-05-09T06:36:11.063331+0000","last_peered":"2024-05-09T06:36:11.063331+0000","last_clean":"2024-05-09T06:36:11.063331+0000","last_became_active":"2024-05-09T06:36:07.875153+0000","last_became_peered":"2024-05-09T06:36:07.875153+0000","last_unstale":"2024-05-09T06:36:11.063331+0000","last_undegraded":"2024-05-09T06:36:11.063331+0000","last_fullsized":"2024-05-09T06:36:11.063331+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:10:28.010336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T06:36:10.066710+0000","last_change":"2024-05-09T06:36:06.861152+0000","last_active":"2024-05-09T06:36:10.066710+0000","last_peered":"2024-05-09T06:36:10.066710+0000","last_clean":"2024-05-09T06:36:10.066710+0000","last_became_active":"2024-05-09T06:36:06.860905+0000","last_became_peered":"2024-05-09T06:36:06.860905+0000","last_unstale":"2024-05-09T06:36:10.066710+0000","last_undegraded":"2024-05-09T06:36:10.066710+0000","last_fullsized":"2024-05-09T06:36:10.066710+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_clean_scrub_stamp":"2024-05-09T06:36:05.833229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:59:29.736305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T06:36:08.862257+0000","last_change":"2024-05-09T06:36:07.865602+0000","last_active":"2024-05-09T06:36:08.862257+0000","last_peered":"2024-05-09T06:36:08.862257+0000","last_clean":"2024-05-09T06:36:08.862257+0000","last_became_active":"2024-05-09T06:36:07.865083+0000","last_became_peered":"2024-05-09T06:36:07.865083+0000","last_unstale":"2024-05-09T06:36:08.862257+0000","last_undegraded":"2024-05-09T06:36:08.862257+0000","last_fullsized":"2024-05-09T06:36:08.862257+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:50:02.245694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T06:36:08.862573+0000","last_change":"2024-05-09T06:36:06.855928+0000","last_active":"2024-05-09T06:36:08.862573+0000","last_peered":"2024-05-09T06:36:08.862573+0000","last_clean":"2024-05-09T06:36:08.862573+0000","last_became_active":"2024-05-09T06:36:06.855322+0000","last_became_peered":"2024-05-09T06:36:06.855322+0000","last_unstale":"2024-05-09T06:36:08.862573+0000","last_undegraded":"2024-05-09T06:36:08.862573+0000","last_fullsized":"2024-05-09T06:36:08.862573+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_clean_scrub_stamp":"2024-05-09T06:36:05.833229+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:03:38.559114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T06:36:11.063444+0000","last_change":"2024-05-09T06:36:07.870976+0000","last_active":"2024-05-09T06:36:11.063444+0000","last_peered":"2024-05-09T06:36:11.063444+0000","last_clean":"2024-05-09T06:36:11.063444+0000","last_became_active":"2024-05-09T06:36:07.870713+0000","last_became_peered":"2024-05-09T06:36:07.870713+0000","last_unstale":"2024-05-09T06:36:11.063444+0000","last_undegraded":"2024-05-09T06:36:11.063444+0000","last_fullsized":"2024-05-09T06:36:11.063444+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:51:24.938206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T06:36:09.855506+0000","last_change":"2024-05-09T06:36:06.863328+0000","last_active":"2024-05-09T06:36:09.855506+0000","last_peered":"2024-05-09T06:36:09.855506+0000","last_clean":"2024-05-09T06:36:09.855506+0000","last_became_active":"2024-05-09T06:36:06.862686+0000","last_became_peered":"2024-05-09T06:36:06.862686+0000","last_unstale":"2024-05-09T06:36:09.855506+0000","last_undegraded":"2024-05-09T06:36:09.855506+0000","last_fullsized":"2024-05-09T06:36:09.855506+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_clean_scrub_stamp":"2024-05-09T06:36:05.833229+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:59:29.736305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T06:36:11.063488+0000","last_change":"2024-05-09T06:36:07.868926+0000","last_active":"2024-05-09T06:36:11.063488+0000","last_peered":"2024-05-09T06:36:11.063488+0000","last_clean":"2024-05-09T06:36:11.063488+0000","last_became_active":"2024-05-09T06:36:07.868613+0000","last_became_peered":"2024-05-09T06:36:07.868613+0000","last_unstale":"2024-05-09T06:36:11.063488+0000","last_undegraded":"2024-05-09T06:36:11.063488+0000","last_fullsized":"2024-05-09T06:36:11.063488+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:24:51.534066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T06:36:10.066153+0000","last_change":"2024-05-09T06:36:06.861104+0000","last_active":"2024-05-09T06:36:10.066153+0000","last_peered":"2024-05-09T06:36:10.066153+0000","last_clean":"2024-05-09T06:36:10.066153+0000","last_became_active":"2024-05-09T06:36:06.860799+0000","last_became_peered":"2024-05-09T06:36:06.860799+0000","last_unstale":"2024-05-09T06:36:10.066153+0000","last_undegraded":"2024-05-09T06:36:10.066153+0000","last_fullsized":"2024-05-09T06:36:10.066153+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_clean_scrub_stamp":"2024-05-09T06:36:05.833229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:58:18.159070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T06:36:08.863915+0000","last_change":"2024-05-09T06:36:07.863846+0000","last_active":"2024-05-09T06:36:08.863915+0000","last_peered":"2024-05-09T06:36:08.863915+0000","last_clean":"2024-05-09T06:36:08.863915+0000","last_became_active":"2024-05-09T06:36:07.863161+0000","last_became_peered":"2024-05-09T06:36:07.863161+0000","last_unstale":"2024-05-09T06:36:08.863915+0000","last_undegraded":"2024-05-09T06:36:08.863915+0000","last_fullsized":"2024-05-09T06:36:08.863915+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:59:30.741519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T06:36:08.862295+0000","last_change":"2024-05-09T06:36:06.861276+0000","last_active":"2024-05-09T06:36:08.862295+0000","last_peered":"2024-05-09T06:36:08.862295+0000","last_clean":"2024-05-09T06:36:08.862295+0000","last_became_active":"2024-05-09T06:36:06.860977+0000","last_became_peered":"2024-05-09T06:36:06.860977+0000","last_unstale":"2024-05-09T06:36:08.862295+0000","last_undegraded":"2024-05-09T06:36:08.862295+0000","last_fullsized":"2024-05-09T06:36:08.862295+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_clean_scrub_stamp":"2024-05-09T06:36:05.833229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:16:34.559063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T06:36:11.063737+0000","last_change":"2024-05-09T06:36:07.870964+0000","last_active":"2024-05-09T06:36:11.063737+0000","last_peered":"2024-05-09T06:36:11.063737+0000","last_clean":"2024-05-09T06:36:11.063737+0000","last_became_active":"2024-05-09T06:36:07.870669+0000","last_became_peered":"2024-05-09T06:36:07.870669+0000","last_unstale":"2024-05-09T06:36:11.063737+0000","last_undegraded":"2024-05-09T06:36:11.063737+0000","last_fullsized":"2024-05-09T06:36:11.063737+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:24:50.227387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T06:36:08.862272+0000","last_change":"2024-05-09T06:36:06.861162+0000","last_active":"2024-05-09T06:36:08.862272+0000","last_peered":"2024-05-09T06:36:08.862272+0000","last_clean":"2024-05-09T06:36:08.862272+0000","last_became_active":"2024-05-09T06:36:06.860880+0000","last_became_peered":"2024-05-09T06:36:06.860880+0000","last_unstale":"2024-05-09T06:36:08.862272+0000","last_undegraded":"2024-05-09T06:36:08.862272+0000","last_fullsized":"2024-05-09T06:36:08.862272+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_clean_scrub_stamp":"2024-05-09T06:36:05.833229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:59:29.736305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T06:36:11.063967+0000","last_change":"2024-05-09T06:36:07.872872+0000","last_active":"2024-05-09T06:36:11.063967+0000","last_peered":"2024-05-09T06:36:11.063967+0000","last_clean":"2024-05-09T06:36:11.063967+0000","last_became_active":"2024-05-09T06:36:07.872556+0000","last_became_peered":"2024-05-09T06:36:07.872556+0000","last_unstale":"2024-05-09T06:36:11.063967+0000","last_undegraded":"2024-05-09T06:36:11.063967+0000","last_fullsized":"2024-05-09T06:36:11.063967+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:25:55.622101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T06:36:10.066553+0000","last_change":"2024-05-09T06:36:06.862160+0000","last_active":"2024-05-09T06:36:10.066553+0000","last_peered":"2024-05-09T06:36:10.066553+0000","last_clean":"2024-05-09T06:36:10.066553+0000","last_became_active":"2024-05-09T06:36:06.861881+0000","last_became_peered":"2024-05-09T06:36:06.861881+0000","last_unstale":"2024-05-09T06:36:10.066553+0000","last_undegraded":"2024-05-09T06:36:10.066553+0000","last_fullsized":"2024-05-09T06:36:10.066553+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_clean_scrub_stamp":"2024-05-09T06:36:05.833229+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:16:34.559063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T06:36:09.855248+0000","last_change":"2024-05-09T06:36:07.869000+0000","last_active":"2024-05-09T06:36:09.855248+0000","last_peered":"2024-05-09T06:36:09.855248+0000","last_clean":"2024-05-09T06:36:09.855248+0000","last_became_active":"2024-05-09T06:36:07.868522+0000","last_became_peered":"2024-05-09T06:36:07.868522+0000","last_unstale":"2024-05-09T06:36:09.855248+0000","last_undegraded":"2024-05-09T06:36:09.855248+0000","last_fullsized":"2024-05-09T06:36:09.855248+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:24:51.534066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T06:36:11.063243+0000","last_change":"2024-05-09T06:36:07.875476+0000","last_active":"2024-05-09T06:36:11.063243+0000","last_peered":"2024-05-09T06:36:11.063243+0000","last_clean":"2024-05-09T06:36:11.063243+0000","last_became_active":"2024-05-09T06:36:07.875170+0000","last_became_peered":"2024-05-09T06:36:07.875170+0000","last_unstale":"2024-05-09T06:36:11.063243+0000","last_undegraded":"2024-05-09T06:36:11.063243+0000","last_fullsized":"2024-05-09T06:36:11.063243+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:25:55.622101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T06:36:10.066379+0000","last_change":"2024-05-09T06:36:06.866061+0000","last_active":"2024-05-09T06:36:10.066379+0000","last_peered":"2024-05-09T06:36:10.066379+0000","last_clean":"2024-05-09T06:36:10.066379+0000","last_became_active":"2024-05-09T06:36:06.865783+0000","last_became_peered":"2024-05-09T06:36:06.865783+0000","last_unstale":"2024-05-09T06:36:10.066379+0000","last_undegraded":"2024-05-09T06:36:10.066379+0000","last_fullsized":"2024-05-09T06:36:10.066379+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_clean_scrub_stamp":"2024-05-09T06:36:05.833229+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:15:49.248390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T06:36:11.063196+0000","last_change":"2024-05-09T06:36:07.876132+0000","last_active":"2024-05-09T06:36:11.063196+0000","last_peered":"2024-05-09T06:36:11.063196+0000","last_clean":"2024-05-09T06:36:11.063196+0000","last_became_active":"2024-05-09T06:36:07.874679+0000","last_became_peered":"2024-05-09T06:36:07.874679+0000","last_unstale":"2024-05-09T06:36:11.063196+0000","last_undegraded":"2024-05-09T06:36:11.063196+0000","last_fullsized":"2024-05-09T06:36:11.063196+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:17:19.091125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T06:36:10.066569+0000","last_change":"2024-05-09T06:36:06.860619+0000","last_active":"2024-05-09T06:36:10.066569+0000","last_peered":"2024-05-09T06:36:10.066569+0000","last_clean":"2024-05-09T06:36:10.066569+0000","last_became_active":"2024-05-09T06:36:06.860323+0000","last_became_peered":"2024-05-09T06:36:06.860323+0000","last_unstale":"2024-05-09T06:36:10.066569+0000","last_undegraded":"2024-05-09T06:36:10.066569+0000","last_fullsized":"2024-05-09T06:36:10.066569+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_clean_scrub_stamp":"2024-05-09T06:36:05.833229+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:40:41.085704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T06:36:11.063734+0000","last_change":"2024-05-09T06:36:07.880245+0000","last_active":"2024-05-09T06:36:11.063734+0000","last_peered":"2024-05-09T06:36:11.063734+0000","last_clean":"2024-05-09T06:36:11.063734+0000","last_became_active":"2024-05-09T06:36:07.879926+0000","last_became_peered":"2024-05-09T06:36:07.879926+0000","last_unstale":"2024-05-09T06:36:11.063734+0000","last_undegraded":"2024-05-09T06:36:11.063734+0000","last_fullsized":"2024-05-09T06:36:11.063734+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:50:02.245694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T06:36:11.063354+0000","last_change":"2024-05-09T06:36:06.864934+0000","last_active":"2024-05-09T06:36:11.063354+0000","last_peered":"2024-05-09T06:36:11.063354+0000","last_clean":"2024-05-09T06:36:11.063354+0000","last_became_active":"2024-05-09T06:36:06.864680+0000","last_became_peered":"2024-05-09T06:36:06.864680+0000","last_unstale":"2024-05-09T06:36:11.063354+0000","last_undegraded":"2024-05-09T06:36:11.063354+0000","last_fullsized":"2024-05-09T06:36:11.063354+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_clean_scrub_stamp":"2024-05-09T06:36:05.833229+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:03:17.189398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T06:36:11.063160+0000","last_change":"2024-05-09T06:36:07.870194+0000","last_active":"2024-05-09T06:36:11.063160+0000","last_peered":"2024-05-09T06:36:11.063160+0000","last_clean":"2024-05-09T06:36:11.063160+0000","last_became_active":"2024-05-09T06:36:07.869895+0000","last_became_peered":"2024-05-09T06:36:07.869895+0000","last_unstale":"2024-05-09T06:36:11.063160+0000","last_undegraded":"2024-05-09T06:36:11.063160+0000","last_fullsized":"2024-05-09T06:36:11.063160+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:44:00.455366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T06:36:11.064034+0000","last_change":"2024-05-09T06:36:06.861090+0000","last_active":"2024-05-09T06:36:11.064034+0000","last_peered":"2024-05-09T06:36:11.064034+0000","last_clean":"2024-05-09T06:36:11.064034+0000","last_became_active":"2024-05-09T06:36:06.860774+0000","last_became_peered":"2024-05-09T06:36:06.860774+0000","last_unstale":"2024-05-09T06:36:11.064034+0000","last_undegraded":"2024-05-09T06:36:11.064034+0000","last_fullsized":"2024-05-09T06:36:11.064034+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_clean_scrub_stamp":"2024-05-09T06:36:05.833229+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:05:58.090556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574856,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27804,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344036,"statfs":{"total":96636764160,"available":96608292864,"internally_reserved":0,"allocated":802816,"data_stored":661884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574856,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574855,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":208896,"data_stored":71024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574855,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":71060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574856,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574855,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":204800,"data_stored":70934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574855,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":212992,"data_stored":71094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574856,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27812,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344028,"statfs":{"total":96636764160,"available":96608284672,"internally_reserved":0,"allocated":811008,"data_stored":661940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-09T06:36:16.361 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-09T06:36:16.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:16.520+0000 7f8bfb6bc640 1 Processor -- start 2024-05-09T06:36:16.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:16.520+0000 7f8bfb6bc640 1 -- start start 2024-05-09T06:36:16.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:16.521+0000 7f8bfb6bc640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8bf40d3c70 0x7f8bf40d2510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:16.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:16.521+0000 7f8bfb6bc640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8bf40d2bd0 con 0x7f8bf415e4c0 2024-05-09T06:36:16.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:16.521+0000 7f8bfb6bc640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8bf40d2eb0 con 0x7f8bf40cf5b0 2024-05-09T06:36:16.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:16.521+0000 7f8bfb6bc640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f8bf40d3190 con 0x7f8bf40d3c70 2024-05-09T06:36:16.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:16.521+0000 7f8bf9458640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f8bf415e4c0 0x7f8bf40ceea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:49294/0 (socket says 172.21.15.3:49294) 2024-05-09T06:36:16.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:16.521+0000 7f8bf9458640 1 -- 172.21.15.3:0/1690811068 learned_addr learned my addr 172.21.15.3:0/1690811068 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-05-09T06:36:16.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:16.521+0000 7f8bf9c59640 1 --2- 172.21.15.3:0/1690811068 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8bf40d3c70 0x7f8bf40d2510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:16.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:16.521+0000 7f8bebfff640 1 -- 172.21.15.3:0/1690811068 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2101303126 0 0) 0x7f8bf40d2bd0 con 0x7f8bf415e4c0 2024-05-09T06:36:16.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:16.521+0000 7f8bebfff640 1 -- 172.21.15.3:0/1690811068 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8be0003520 con 0x7f8bf415e4c0 2024-05-09T06:36:16.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:16.522+0000 7f8bebfff640 1 -- 172.21.15.3:0/1690811068 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 839756131 0 0) 0x7f8be0003520 con 0x7f8bf415e4c0 2024-05-09T06:36:16.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:16.522+0000 7f8bebfff640 1 -- 172.21.15.3:0/1690811068 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8bf40d3c70 msgr2=0x7f8bf40d2510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:16.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:16.522+0000 7f8bebfff640 1 --2- 172.21.15.3:0/1690811068 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8bf40d3c70 0x7f8bf40d2510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:16.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:16.522+0000 7f8bebfff640 1 -- 172.21.15.3:0/1690811068 >> v1:172.21.15.190:6789/0 conn(0x7f8bf40cf5b0 legacy=0x7f8bf40b7260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:16.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:16.522+0000 7f8bebfff640 1 -- 172.21.15.3:0/1690811068 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bf40d3470 con 0x7f8bf415e4c0 2024-05-09T06:36:16.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:16.522+0000 7f8bebfff640 1 -- 172.21.15.3:0/1690811068 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 563312031 0 0) 0x7f8bf0001670 con 0x7f8bf415e4c0 2024-05-09T06:36:16.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:16.522+0000 7f8bebfff640 1 -- 172.21.15.3:0/1690811068 >> v1:172.21.15.3:6789/0 conn(0x7f8bf415e4c0 legacy=0x7f8bf40ceea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:16.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:16.522+0000 7f8bebfff640 1 --2- 172.21.15.3:0/1690811068 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8be00039e0 0x7f8be0003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:16.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:16.522+0000 7f8bebfff640 1 --2- 172.21.15.3:0/1690811068 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8bf40d3c70 0x7f8be0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:16.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:16.522+0000 7f8bebfff640 1 --2- 172.21.15.3:0/1690811068 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8be0004d40 0x7f8be00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:16.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:16.522+0000 7f8bebfff640 1 -- 172.21.15.3:0/1690811068 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f8bf40d3190 con 0x7f8be00039e0 2024-05-09T06:36:16.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:16.522+0000 7f8bebfff640 1 -- 172.21.15.3:0/1690811068 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f8be000bb80 con 0x7f8be0004d40 2024-05-09T06:36:16.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:16.522+0000 7f8bebfff640 1 -- 172.21.15.3:0/1690811068 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f8be000bd80 con 0x7f8bf40d3c70 2024-05-09T06:36:16.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:16.522+0000 7f8bf9458640 1 --2- 172.21.15.3:0/1690811068 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8be00039e0 0x7f8be0003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:16.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:16.522+0000 7f8bf8c57640 1 --2- 172.21.15.3:0/1690811068 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8bf40d3c70 0x7f8be0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:16.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:16.522+0000 7f8bf9c59640 1 --2- 172.21.15.3:0/1690811068 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8be0004d40 0x7f8be00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:16.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:16.522+0000 7f8bf9458640 1 -- 172.21.15.3:0/1690811068 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8bf40d3c70 msgr2=0x7f8be0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:16.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:16.522+0000 7f8bf9458640 1 --2- 172.21.15.3:0/1690811068 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8bf40d3c70 0x7f8be0004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:16.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:16.522+0000 7f8bf9458640 1 -- 172.21.15.3:0/1690811068 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8be0004d40 msgr2=0x7f8be00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:16.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:16.522+0000 7f8bf9458640 1 --2- 172.21.15.3:0/1690811068 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8be0004d40 0x7f8be00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:16.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:16.522+0000 7f8bf9458640 1 -- 172.21.15.3:0/1690811068 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8be0003520 con 0x7f8be00039e0 2024-05-09T06:36:16.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:16.523+0000 7f8bf9458640 1 --2- 172.21.15.3:0/1690811068 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8be00039e0 0x7f8be0003e00 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f8bf000c240 tx=0x7f8be0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=248d85f67c76858c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:16.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:16.523+0000 7f8bebfff640 1 -- 172.21.15.3:0/1690811068 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bf0007d10 con 0x7f8be00039e0 2024-05-09T06:36:16.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:16.523+0000 7f8bebfff640 1 -- 172.21.15.3:0/1690811068 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8bf0005ad0 con 0x7f8be00039e0 2024-05-09T06:36:16.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:16.523+0000 7f8bebfff640 1 -- 172.21.15.3:0/1690811068 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bf0007070 con 0x7f8be00039e0 2024-05-09T06:36:16.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:16.523+0000 7f8bfb6bc640 1 -- 172.21.15.3:0/1690811068 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8be00039e0 msgr2=0x7f8be0003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:16.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:16.523+0000 7f8bfb6bc640 1 --2- 172.21.15.3:0/1690811068 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8be00039e0 0x7f8be0003e00 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f8bf000c240 tx=0x7f8be0005830 comp rx=0 tx=0).stop 2024-05-09T06:36:16.512 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:16.524+0000 7f8bf9c59640 1 -- 172.21.15.3:0/1690811068 reap_dead start 2024-05-09T06:36:16.512 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:16.524+0000 7f8bfb6bc640 1 -- 172.21.15.3:0/1690811068 shutdown_connections 2024-05-09T06:36:16.512 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:16.524+0000 7f8bfb6bc640 1 -- 172.21.15.3:0/1690811068 >> 172.21.15.3:0/1690811068 conn(0x7f8bf40673b0 msgr2=0x7f8bf40677c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:16.512 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:16.524+0000 7f8bfb6bc640 1 -- 172.21.15.3:0/1690811068 shutdown_connections 2024-05-09T06:36:16.512 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:16.524+0000 7f8bfb6bc640 1 -- 172.21.15.3:0/1690811068 wait complete. 2024-05-09T06:36:16.512 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:16.525+0000 7f8bfb6bc640 1 Processor -- start 2024-05-09T06:36:16.512 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:16.525+0000 7f8bfb6bc640 1 -- start start 2024-05-09T06:36:16.512 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:16.526+0000 7f8bfb6bc640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8be00039e0 0x7f8bf40c88b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:16.512 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:16.526+0000 7f8bfb6bc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8bf40cdd10 0x7f8bf41f3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:16.512 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:16.526+0000 7f8bfb6bc640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8bf40c9f30 0x7f8bf41604d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:16.512 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:16.526+0000 7f8bfb6bc640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f8bf40d2690 con 0x7f8bf40cdd10 2024-05-09T06:36:16.512 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:16.526+0000 7f8bfb6bc640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f8bf40d2510 con 0x7f8be00039e0 2024-05-09T06:36:16.512 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:16.526+0000 7f8bfb6bc640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f8bf40d2810 con 0x7f8bf40c9f30 2024-05-09T06:36:16.513 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:16.526+0000 7f8bf8c57640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8bf40cdd10 0x7f8bf41f3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:16.513 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:16.526+0000 7f8bf8c57640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8bf40cdd10 0x7f8bf41f3cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49814/0 (socket says 172.21.15.3:49814) 2024-05-09T06:36:16.513 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:16.526+0000 7f8bf8c57640 1 -- 172.21.15.3:0/2686534297 learned_addr learned my addr 172.21.15.3:0/2686534297 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:36:16.513 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:16.526+0000 7f8bf9458640 1 --2- 172.21.15.3:0/2686534297 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8be00039e0 0x7f8bf40c88b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:16.513 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:16.526+0000 7f8bf9c59640 1 --2- 172.21.15.3:0/2686534297 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8bf40c9f30 0x7f8bf41604d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:16.513 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:16.526+0000 7f8bf8c57640 1 -- 172.21.15.3:0/2686534297 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8bf40c9f30 msgr2=0x7f8bf41604d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:16.513 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:16.526+0000 7f8bf8c57640 1 --2- 172.21.15.3:0/2686534297 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8bf40c9f30 0x7f8bf41604d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:16.513 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:16.526+0000 7f8bf8c57640 1 -- 172.21.15.3:0/2686534297 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8be00039e0 msgr2=0x7f8bf40c88b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:16.513 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:16.526+0000 7f8bf8c57640 1 --2- 172.21.15.3:0/2686534297 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8be00039e0 0x7f8bf40c88b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:16.513 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:16.526+0000 7f8bf8c57640 1 -- 172.21.15.3:0/2686534297 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bf40ce190 con 0x7f8bf40cdd10 2024-05-09T06:36:16.513 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:16.526+0000 7f8bf8c57640 1 --2- 172.21.15.3:0/2686534297 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8bf40cdd10 0x7f8bf41f3cc0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f8be400f820 tx=0x7f8be400fce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:16.513 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:16.526+0000 7f8be9ffb640 1 -- 172.21.15.3:0/2686534297 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8be40050d0 con 0x7f8bf40cdd10 2024-05-09T06:36:16.513 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:16.526+0000 7f8be9ffb640 1 -- 172.21.15.3:0/2686534297 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8be4018070 con 0x7f8bf40cdd10 2024-05-09T06:36:16.513 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:16.526+0000 7f8bfb6bc640 1 -- 172.21.15.3:0/2686534297 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8bf40c8e90 con 0x7f8bf40cdd10 2024-05-09T06:36:16.513 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:16.526+0000 7f8be9ffb640 1 -- 172.21.15.3:0/2686534297 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8be4005490 con 0x7f8bf40cdd10 2024-05-09T06:36:16.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:16.527+0000 7f8bfb6bc640 1 -- 172.21.15.3:0/2686534297 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8bf40c9730 con 0x7f8bf40cdd10 2024-05-09T06:36:16.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:16.528+0000 7f8bfb6bc640 1 -- 172.21.15.3:0/2686534297 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8bf40cf390 con 0x7f8bf40cdd10 2024-05-09T06:36:16.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:16.528+0000 7f8be9ffb640 1 -- 172.21.15.3:0/2686534297 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8be400d730 con 0x7f8bf40cdd10 2024-05-09T06:36:16.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:16.529+0000 7f8be9ffb640 1 --2- 172.21.15.3:0/2686534297 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f8bd806be30 0x7f8bd806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:16.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:16.529+0000 7f8be9ffb640 1 -- 172.21.15.3:0/2686534297 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f8be408ac20 con 0x7f8bf40cdd10 2024-05-09T06:36:16.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:16.529+0000 7f8bf9458640 1 --2- 172.21.15.3:0/2686534297 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f8bd806be30 0x7f8bd806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:16.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:16.529+0000 7f8bf9458640 1 --2- 172.21.15.3:0/2686534297 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f8bd806be30 0x7f8bd806e2f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f8bf0033000 tx=0x7f8bf002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:16.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:16.533+0000 7f8be9ffb640 1 -- 172.21.15.3:0/2686534297 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f8bf40cf390 con 0x7f8bf40cdd10 2024-05-09T06:36:16.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:16.715+0000 7f8bfb6bc640 1 -- 172.21.15.3:0/2686534297 --> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f8bf404f660 con 0x7f8bd806be30 2024-05-09T06:36:16.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:16.722+0000 7f8be9ffb640 1 -- 172.21.15.3:0/2686534297 <== mgr.4105 v2:172.21.15.190:6832/975893321 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251836 (secure 0 0 0) 0x7f8bf404f660 con 0x7f8bd806be30 2024-05-09T06:36:16.705 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-09T06:36:16.706 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2024-05-09T06:36:16.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:16.725+0000 7f8bfb6bc640 1 -- 172.21.15.3:0/2686534297 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f8bd806be30 msgr2=0x7f8bd806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:16.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:16.725+0000 7f8bfb6bc640 1 --2- 172.21.15.3:0/2686534297 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f8bd806be30 0x7f8bd806e2f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f8bf0033000 tx=0x7f8bf002e000 comp rx=0 tx=0).stop 2024-05-09T06:36:16.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:16.725+0000 7f8bfb6bc640 1 -- 172.21.15.3:0/2686534297 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8bf40cdd10 msgr2=0x7f8bf41f3cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:16.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:16.725+0000 7f8bfb6bc640 1 --2- 172.21.15.3:0/2686534297 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8bf40cdd10 0x7f8bf41f3cc0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f8be400f820 tx=0x7f8be400fce0 comp rx=0 tx=0).stop 2024-05-09T06:36:16.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:16.725+0000 7f8bfb6bc640 1 -- 172.21.15.3:0/2686534297 shutdown_connections 2024-05-09T06:36:16.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:16.725+0000 7f8bfb6bc640 1 --2- 172.21.15.3:0/2686534297 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f8bd806be30 0x7f8bd806e2f0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:16.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:16.725+0000 7f8bfb6bc640 1 --2- 172.21.15.3:0/2686534297 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8bf40c9f30 0x7f8bf41604d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:16.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:16.725+0000 7f8bfb6bc640 1 --2- 172.21.15.3:0/2686534297 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8bf40cdd10 0x7f8bf41f3cc0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:16.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:16.725+0000 7f8bfb6bc640 1 --2- 172.21.15.3:0/2686534297 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8be00039e0 0x7f8bf40c88b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:16.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:16.725+0000 7f8bfb6bc640 1 -- 172.21.15.3:0/2686534297 >> 172.21.15.3:0/2686534297 conn(0x7f8bf40673b0 msgr2=0x7f8bf40b7ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:16.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:16.725+0000 7f8bfb6bc640 1 -- 172.21.15.3:0/2686534297 shutdown_connections 2024-05-09T06:36:16.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:16.725+0000 7f8bfb6bc640 1 -- 172.21.15.3:0/2686534297 wait complete. 2024-05-09T06:36:16.717 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2024-05-09T06:36:14.777499+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218996,"kb_used_data":2824,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755724,"statfs":{"total":773094113280,"available":772869861376,"internally_reserved":0,"allocated":2891776,"data_stored":1753028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.933717"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T06:36:10.066849+0000","last_change":"2024-05-09T06:36:07.870322+0000","last_active":"2024-05-09T06:36:10.066849+0000","last_peered":"2024-05-09T06:36:10.066849+0000","last_clean":"2024-05-09T06:36:10.066849+0000","last_became_active":"2024-05-09T06:36:07.870000+0000","last_became_peered":"2024-05-09T06:36:07.870000+0000","last_unstale":"2024-05-09T06:36:10.066849+0000","last_undegraded":"2024-05-09T06:36:10.066849+0000","last_fullsized":"2024-05-09T06:36:10.066849+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:45:01.680949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T06:36:08.862440+0000","last_change":"2024-05-09T06:36:07.865401+0000","last_active":"2024-05-09T06:36:08.862440+0000","last_peered":"2024-05-09T06:36:08.862440+0000","last_clean":"2024-05-09T06:36:08.862440+0000","last_became_active":"2024-05-09T06:36:07.865119+0000","last_became_peered":"2024-05-09T06:36:07.865119+0000","last_unstale":"2024-05-09T06:36:08.862440+0000","last_undegraded":"2024-05-09T06:36:08.862440+0000","last_fullsized":"2024-05-09T06:36:08.862440+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:19:58.370458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T06:36:11.063203+0000","last_change":"2024-05-09T06:36:07.865504+0000","last_active":"2024-05-09T06:36:11.063203+0000","last_peered":"2024-05-09T06:36:11.063203+0000","last_clean":"2024-05-09T06:36:11.063203+0000","last_became_active":"2024-05-09T06:36:07.864109+0000","last_became_peered":"2024-05-09T06:36:07.864109+0000","last_unstale":"2024-05-09T06:36:11.063203+0000","last_undegraded":"2024-05-09T06:36:11.063203+0000","last_fullsized":"2024-05-09T06:36:11.063203+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:41:33.459622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T06:36:11.063245+0000","last_change":"2024-05-09T06:36:07.879922+0000","last_active":"2024-05-09T06:36:11.063245+0000","last_peered":"2024-05-09T06:36:11.063245+0000","last_clean":"2024-05-09T06:36:11.063245+0000","last_became_active":"2024-05-09T06:36:07.879559+0000","last_became_peered":"2024-05-09T06:36:07.879559+0000","last_unstale":"2024-05-09T06:36:11.063245+0000","last_undegraded":"2024-05-09T06:36:11.063245+0000","last_fullsized":"2024-05-09T06:36:11.063245+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:47:30.595382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T06:36:10.066399+0000","last_change":"2024-05-09T06:36:07.867057+0000","last_active":"2024-05-09T06:36:10.066399+0000","last_peered":"2024-05-09T06:36:10.066399+0000","last_clean":"2024-05-09T06:36:10.066399+0000","last_became_active":"2024-05-09T06:36:07.866657+0000","last_became_peered":"2024-05-09T06:36:07.866657+0000","last_unstale":"2024-05-09T06:36:10.066399+0000","last_undegraded":"2024-05-09T06:36:10.066399+0000","last_fullsized":"2024-05-09T06:36:10.066399+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:25:55.622101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T06:36:11.063093+0000","last_change":"2024-05-09T06:36:07.883141+0000","last_active":"2024-05-09T06:36:11.063093+0000","last_peered":"2024-05-09T06:36:11.063093+0000","last_clean":"2024-05-09T06:36:11.063093+0000","last_became_active":"2024-05-09T06:36:07.882826+0000","last_became_peered":"2024-05-09T06:36:07.882826+0000","last_unstale":"2024-05-09T06:36:11.063093+0000","last_undegraded":"2024-05-09T06:36:11.063093+0000","last_fullsized":"2024-05-09T06:36:11.063093+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:03:16.134498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T06:36:10.066319+0000","last_change":"2024-05-09T06:36:07.876998+0000","last_active":"2024-05-09T06:36:10.066319+0000","last_peered":"2024-05-09T06:36:10.066319+0000","last_clean":"2024-05-09T06:36:10.066319+0000","last_became_active":"2024-05-09T06:36:07.876672+0000","last_became_peered":"2024-05-09T06:36:07.876672+0000","last_unstale":"2024-05-09T06:36:10.066319+0000","last_undegraded":"2024-05-09T06:36:10.066319+0000","last_fullsized":"2024-05-09T06:36:10.066319+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:57:30.164280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T06:36:11.064025+0000","last_change":"2024-05-09T06:36:07.878020+0000","last_active":"2024-05-09T06:36:11.064025+0000","last_peered":"2024-05-09T06:36:11.064025+0000","last_clean":"2024-05-09T06:36:11.064025+0000","last_became_active":"2024-05-09T06:36:07.877767+0000","last_became_peered":"2024-05-09T06:36:07.877767+0000","last_unstale":"2024-05-09T06:36:11.064025+0000","last_undegraded":"2024-05-09T06:36:11.064025+0000","last_fullsized":"2024-05-09T06:36:11.064025+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:47:30.595382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T06:36:10.066848+0000","last_change":"2024-05-09T06:36:07.877214+0000","last_active":"2024-05-09T06:36:10.066848+0000","last_peered":"2024-05-09T06:36:10.066848+0000","last_clean":"2024-05-09T06:36:10.066848+0000","last_became_active":"2024-05-09T06:36:07.876924+0000","last_became_peered":"2024-05-09T06:36:07.876924+0000","last_unstale":"2024-05-09T06:36:10.066848+0000","last_undegraded":"2024-05-09T06:36:10.066848+0000","last_fullsized":"2024-05-09T06:36:10.066848+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:40:20.267663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T06:36:09.855529+0000","last_change":"2024-05-09T06:36:07.879229+0000","last_active":"2024-05-09T06:36:09.855529+0000","last_peered":"2024-05-09T06:36:09.855529+0000","last_clean":"2024-05-09T06:36:09.855529+0000","last_became_active":"2024-05-09T06:36:07.879000+0000","last_became_peered":"2024-05-09T06:36:07.879000+0000","last_unstale":"2024-05-09T06:36:09.855529+0000","last_undegraded":"2024-05-09T06:36:09.855529+0000","last_fullsized":"2024-05-09T06:36:09.855529+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:19:58.370458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T06:36:09.855365+0000","last_change":"2024-05-09T06:36:07.878671+0000","last_active":"2024-05-09T06:36:09.855365+0000","last_peered":"2024-05-09T06:36:09.855365+0000","last_clean":"2024-05-09T06:36:09.855365+0000","last_became_active":"2024-05-09T06:36:07.878389+0000","last_became_peered":"2024-05-09T06:36:07.878389+0000","last_unstale":"2024-05-09T06:36:09.855365+0000","last_undegraded":"2024-05-09T06:36:09.855365+0000","last_fullsized":"2024-05-09T06:36:09.855365+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:34:13.971998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T06:36:08.864278+0000","last_change":"2024-05-09T06:36:07.859955+0000","last_active":"2024-05-09T06:36:08.864278+0000","last_peered":"2024-05-09T06:36:08.864278+0000","last_clean":"2024-05-09T06:36:08.864278+0000","last_became_active":"2024-05-09T06:36:07.859649+0000","last_became_peered":"2024-05-09T06:36:07.859649+0000","last_unstale":"2024-05-09T06:36:08.864278+0000","last_undegraded":"2024-05-09T06:36:08.864278+0000","last_fullsized":"2024-05-09T06:36:08.864278+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:58:19.164284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T06:36:11.063503+0000","last_change":"2024-05-09T06:36:07.870408+0000","last_active":"2024-05-09T06:36:11.063503+0000","last_peered":"2024-05-09T06:36:11.063503+0000","last_clean":"2024-05-09T06:36:11.063503+0000","last_became_active":"2024-05-09T06:36:07.868679+0000","last_became_peered":"2024-05-09T06:36:07.868679+0000","last_unstale":"2024-05-09T06:36:11.063503+0000","last_undegraded":"2024-05-09T06:36:11.063503+0000","last_fullsized":"2024-05-09T06:36:11.063503+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:03:16.134498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T06:36:11.063203+0000","last_change":"2024-05-09T06:36:07.872146+0000","last_active":"2024-05-09T06:36:11.063203+0000","last_peered":"2024-05-09T06:36:11.063203+0000","last_clean":"2024-05-09T06:36:11.063203+0000","last_became_active":"2024-05-09T06:36:07.871858+0000","last_became_peered":"2024-05-09T06:36:07.871858+0000","last_unstale":"2024-05-09T06:36:11.063203+0000","last_undegraded":"2024-05-09T06:36:11.063203+0000","last_fullsized":"2024-05-09T06:36:11.063203+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:44:00.455366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T06:36:08.862179+0000","last_change":"2024-05-09T06:36:07.864771+0000","last_active":"2024-05-09T06:36:08.862179+0000","last_peered":"2024-05-09T06:36:08.862179+0000","last_clean":"2024-05-09T06:36:08.862179+0000","last_became_active":"2024-05-09T06:36:07.864352+0000","last_became_peered":"2024-05-09T06:36:07.864352+0000","last_unstale":"2024-05-09T06:36:08.862179+0000","last_undegraded":"2024-05-09T06:36:08.862179+0000","last_fullsized":"2024-05-09T06:36:08.862179+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:34:13.971998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T06:36:11.062927+0000","last_change":"2024-05-09T06:36:07.883189+0000","last_active":"2024-05-09T06:36:11.062927+0000","last_peered":"2024-05-09T06:36:11.062927+0000","last_clean":"2024-05-09T06:36:11.062927+0000","last_became_active":"2024-05-09T06:36:07.882906+0000","last_became_peered":"2024-05-09T06:36:07.882906+0000","last_unstale":"2024-05-09T06:36:11.062927+0000","last_undegraded":"2024-05-09T06:36:11.062927+0000","last_fullsized":"2024-05-09T06:36:11.062927+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:19:58.370458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T06:36:11.063637+0000","last_change":"2024-05-09T06:36:07.878966+0000","last_active":"2024-05-09T06:36:11.063637+0000","last_peered":"2024-05-09T06:36:11.063637+0000","last_clean":"2024-05-09T06:36:11.063637+0000","last_became_active":"2024-05-09T06:36:07.878678+0000","last_became_peered":"2024-05-09T06:36:07.878678+0000","last_unstale":"2024-05-09T06:36:11.063637+0000","last_undegraded":"2024-05-09T06:36:11.063637+0000","last_fullsized":"2024-05-09T06:36:11.063637+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:40:20.267663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T06:36:11.063413+0000","last_change":"2024-05-09T06:36:07.866493+0000","last_active":"2024-05-09T06:36:11.063413+0000","last_peered":"2024-05-09T06:36:11.063413+0000","last_clean":"2024-05-09T06:36:11.063413+0000","last_became_active":"2024-05-09T06:36:07.866157+0000","last_became_peered":"2024-05-09T06:36:07.866157+0000","last_unstale":"2024-05-09T06:36:11.063413+0000","last_undegraded":"2024-05-09T06:36:11.063413+0000","last_fullsized":"2024-05-09T06:36:11.063413+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:03:39.564328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T06:36:10.066784+0000","last_change":"2024-05-09T06:36:07.866352+0000","last_active":"2024-05-09T06:36:10.066784+0000","last_peered":"2024-05-09T06:36:10.066784+0000","last_clean":"2024-05-09T06:36:10.066784+0000","last_became_active":"2024-05-09T06:36:07.866066+0000","last_became_peered":"2024-05-09T06:36:07.866066+0000","last_unstale":"2024-05-09T06:36:10.066784+0000","last_undegraded":"2024-05-09T06:36:10.066784+0000","last_fullsized":"2024-05-09T06:36:10.066784+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:17:19.091125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T06:36:09.855483+0000","last_change":"2024-05-09T06:36:07.858555+0000","last_active":"2024-05-09T06:36:09.855483+0000","last_peered":"2024-05-09T06:36:09.855483+0000","last_clean":"2024-05-09T06:36:09.855483+0000","last_became_active":"2024-05-09T06:36:07.858259+0000","last_became_peered":"2024-05-09T06:36:07.858259+0000","last_unstale":"2024-05-09T06:36:09.855483+0000","last_undegraded":"2024-05-09T06:36:09.855483+0000","last_fullsized":"2024-05-09T06:36:09.855483+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:50:02.245694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T06:36:11.063981+0000","last_change":"2024-05-09T06:36:07.875125+0000","last_active":"2024-05-09T06:36:11.063981+0000","last_peered":"2024-05-09T06:36:11.063981+0000","last_clean":"2024-05-09T06:36:11.063981+0000","last_became_active":"2024-05-09T06:36:07.874432+0000","last_became_peered":"2024-05-09T06:36:07.874432+0000","last_unstale":"2024-05-09T06:36:11.063981+0000","last_undegraded":"2024-05-09T06:36:11.063981+0000","last_fullsized":"2024-05-09T06:36:11.063981+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:24:50.227387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T06:36:11.063112+0000","last_change":"2024-05-09T06:36:07.876701+0000","last_active":"2024-05-09T06:36:11.063112+0000","last_peered":"2024-05-09T06:36:11.063112+0000","last_clean":"2024-05-09T06:36:11.063112+0000","last_became_active":"2024-05-09T06:36:07.876416+0000","last_became_peered":"2024-05-09T06:36:07.876416+0000","last_unstale":"2024-05-09T06:36:11.063112+0000","last_undegraded":"2024-05-09T06:36:11.063112+0000","last_fullsized":"2024-05-09T06:36:11.063112+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:57:30.164280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T06:36:11.063138+0000","last_change":"2024-05-09T06:36:07.868404+0000","last_active":"2024-05-09T06:36:11.063138+0000","last_peered":"2024-05-09T06:36:11.063138+0000","last_clean":"2024-05-09T06:36:11.063138+0000","last_became_active":"2024-05-09T06:36:07.867909+0000","last_became_peered":"2024-05-09T06:36:07.867909+0000","last_unstale":"2024-05-09T06:36:11.063138+0000","last_undegraded":"2024-05-09T06:36:11.063138+0000","last_fullsized":"2024-05-09T06:36:11.063138+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:24:51.534066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T06:36:11.063002+0000","last_change":"2024-05-09T06:36:07.880411+0000","last_active":"2024-05-09T06:36:11.063002+0000","last_peered":"2024-05-09T06:36:11.063002+0000","last_clean":"2024-05-09T06:36:11.063002+0000","last_became_active":"2024-05-09T06:36:07.879887+0000","last_became_peered":"2024-05-09T06:36:07.879887+0000","last_unstale":"2024-05-09T06:36:11.063002+0000","last_undegraded":"2024-05-09T06:36:11.063002+0000","last_fullsized":"2024-05-09T06:36:11.063002+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:24:50.227387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T06:36:11.063293+0000","last_change":"2024-05-09T06:36:07.868320+0000","last_active":"2024-05-09T06:36:11.063293+0000","last_peered":"2024-05-09T06:36:11.063293+0000","last_clean":"2024-05-09T06:36:11.063293+0000","last_became_active":"2024-05-09T06:36:07.867969+0000","last_became_peered":"2024-05-09T06:36:07.867969+0000","last_unstale":"2024-05-09T06:36:11.063293+0000","last_undegraded":"2024-05-09T06:36:11.063293+0000","last_fullsized":"2024-05-09T06:36:11.063293+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:50:02.245694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T06:36:09.855614+0000","last_change":"2024-05-09T06:36:07.869956+0000","last_active":"2024-05-09T06:36:09.855614+0000","last_peered":"2024-05-09T06:36:09.855614+0000","last_clean":"2024-05-09T06:36:09.855614+0000","last_became_active":"2024-05-09T06:36:07.869695+0000","last_became_peered":"2024-05-09T06:36:07.869695+0000","last_unstale":"2024-05-09T06:36:09.855614+0000","last_undegraded":"2024-05-09T06:36:09.855614+0000","last_fullsized":"2024-05-09T06:36:09.855614+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:24:50.227387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T06:36:08.862523+0000","last_change":"2024-05-09T06:36:07.867102+0000","last_active":"2024-05-09T06:36:08.862523+0000","last_peered":"2024-05-09T06:36:08.862523+0000","last_clean":"2024-05-09T06:36:08.862523+0000","last_became_active":"2024-05-09T06:36:07.866785+0000","last_became_peered":"2024-05-09T06:36:07.866785+0000","last_unstale":"2024-05-09T06:36:08.862523+0000","last_undegraded":"2024-05-09T06:36:08.862523+0000","last_fullsized":"2024-05-09T06:36:08.862523+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:24:50.227387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T06:36:11.063773+0000","last_change":"2024-05-09T06:36:07.871788+0000","last_active":"2024-05-09T06:36:11.063773+0000","last_peered":"2024-05-09T06:36:11.063773+0000","last_clean":"2024-05-09T06:36:11.063773+0000","last_became_active":"2024-05-09T06:36:07.871500+0000","last_became_peered":"2024-05-09T06:36:07.871500+0000","last_unstale":"2024-05-09T06:36:11.063773+0000","last_undegraded":"2024-05-09T06:36:11.063773+0000","last_fullsized":"2024-05-09T06:36:11.063773+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:57:30.164280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T06:36:11.063659+0000","last_change":"2024-05-09T06:36:07.876623+0000","last_active":"2024-05-09T06:36:11.063659+0000","last_peered":"2024-05-09T06:36:11.063659+0000","last_clean":"2024-05-09T06:36:11.063659+0000","last_became_active":"2024-05-09T06:36:07.876358+0000","last_became_peered":"2024-05-09T06:36:07.876358+0000","last_unstale":"2024-05-09T06:36:11.063659+0000","last_undegraded":"2024-05-09T06:36:11.063659+0000","last_fullsized":"2024-05-09T06:36:11.063659+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:34:13.971998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T06:36:11.063353+0000","last_change":"2024-05-09T06:36:07.879149+0000","last_active":"2024-05-09T06:36:11.063353+0000","last_peered":"2024-05-09T06:36:11.063353+0000","last_clean":"2024-05-09T06:36:11.063353+0000","last_became_active":"2024-05-09T06:36:07.878868+0000","last_became_peered":"2024-05-09T06:36:07.878868+0000","last_unstale":"2024-05-09T06:36:11.063353+0000","last_undegraded":"2024-05-09T06:36:11.063353+0000","last_fullsized":"2024-05-09T06:36:11.063353+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:56:41.722475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T06:36:11.062828+0000","last_change":"2024-05-09T06:36:07.876443+0000","last_active":"2024-05-09T06:36:11.062828+0000","last_peered":"2024-05-09T06:36:11.062828+0000","last_clean":"2024-05-09T06:36:11.062828+0000","last_became_active":"2024-05-09T06:36:07.876122+0000","last_became_peered":"2024-05-09T06:36:07.876122+0000","last_unstale":"2024-05-09T06:36:11.062828+0000","last_undegraded":"2024-05-09T06:36:11.062828+0000","last_fullsized":"2024-05-09T06:36:11.062828+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:58:19.164284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T06:36:09.855285+0000","last_change":"2024-05-09T06:36:07.867899+0000","last_active":"2024-05-09T06:36:09.855285+0000","last_peered":"2024-05-09T06:36:09.855285+0000","last_clean":"2024-05-09T06:36:09.855285+0000","last_became_active":"2024-05-09T06:36:07.867371+0000","last_became_peered":"2024-05-09T06:36:07.867371+0000","last_unstale":"2024-05-09T06:36:09.855285+0000","last_undegraded":"2024-05-09T06:36:09.855285+0000","last_fullsized":"2024-05-09T06:36:09.855285+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:44:00.455366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T06:36:08.863886+0000","last_change":"2024-05-09T06:36:07.871355+0000","last_active":"2024-05-09T06:36:08.863886+0000","last_peered":"2024-05-09T06:36:08.863886+0000","last_clean":"2024-05-09T06:36:08.863886+0000","last_became_active":"2024-05-09T06:36:07.871025+0000","last_became_peered":"2024-05-09T06:36:07.871025+0000","last_unstale":"2024-05-09T06:36:08.863886+0000","last_undegraded":"2024-05-09T06:36:08.863886+0000","last_fullsized":"2024-05-09T06:36:08.863886+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:40:42.090918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T06:36:11.064068+0000","last_change":"2024-05-09T06:36:06.865248+0000","last_active":"2024-05-09T06:36:11.064068+0000","last_peered":"2024-05-09T06:36:11.064068+0000","last_clean":"2024-05-09T06:36:11.064068+0000","last_became_active":"2024-05-09T06:36:06.864940+0000","last_became_peered":"2024-05-09T06:36:06.864940+0000","last_unstale":"2024-05-09T06:36:11.064068+0000","last_undegraded":"2024-05-09T06:36:11.064068+0000","last_fullsized":"2024-05-09T06:36:11.064068+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_clean_scrub_stamp":"2024-05-09T06:36:05.833229+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:08:20.004456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T06:36:11.063797+0000","last_change":"2024-05-09T06:36:07.874223+0000","last_active":"2024-05-09T06:36:11.063797+0000","last_peered":"2024-05-09T06:36:11.063797+0000","last_clean":"2024-05-09T06:36:11.063797+0000","last_became_active":"2024-05-09T06:36:07.873787+0000","last_became_peered":"2024-05-09T06:36:07.873787+0000","last_unstale":"2024-05-09T06:36:11.063797+0000","last_undegraded":"2024-05-09T06:36:11.063797+0000","last_fullsized":"2024-05-09T06:36:11.063797+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:51:24.938206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T06:36:09.855173+0000","last_change":"2024-05-09T06:36:06.862586+0000","last_active":"2024-05-09T06:36:09.855173+0000","last_peered":"2024-05-09T06:36:09.855173+0000","last_clean":"2024-05-09T06:36:09.855173+0000","last_became_active":"2024-05-09T06:36:06.861545+0000","last_became_peered":"2024-05-09T06:36:06.861545+0000","last_unstale":"2024-05-09T06:36:09.855173+0000","last_undegraded":"2024-05-09T06:36:09.855173+0000","last_fullsized":"2024-05-09T06:36:09.855173+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_clean_scrub_stamp":"2024-05-09T06:36:05.833229+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:15:49.248390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T06:36:11.063612+0000","last_change":"2024-05-09T06:36:06.867498+0000","last_active":"2024-05-09T06:36:11.063612+0000","last_peered":"2024-05-09T06:36:11.063612+0000","last_clean":"2024-05-09T06:36:11.063612+0000","last_became_active":"2024-05-09T06:36:06.867177+0000","last_became_peered":"2024-05-09T06:36:06.867177+0000","last_unstale":"2024-05-09T06:36:11.063612+0000","last_undegraded":"2024-05-09T06:36:11.063612+0000","last_fullsized":"2024-05-09T06:36:11.063612+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_clean_scrub_stamp":"2024-05-09T06:36:05.833229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:23:13.976894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T06:36:11.063580+0000","last_change":"2024-05-09T06:36:07.871392+0000","last_active":"2024-05-09T06:36:11.063580+0000","last_peered":"2024-05-09T06:36:11.063580+0000","last_clean":"2024-05-09T06:36:11.063580+0000","last_became_active":"2024-05-09T06:36:07.871103+0000","last_became_peered":"2024-05-09T06:36:07.871103+0000","last_unstale":"2024-05-09T06:36:11.063580+0000","last_undegraded":"2024-05-09T06:36:11.063580+0000","last_fullsized":"2024-05-09T06:36:11.063580+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:19:58.370458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T06:36:11.063055+0000","last_change":"2024-05-09T06:36:06.865012+0000","last_active":"2024-05-09T06:36:11.063055+0000","last_peered":"2024-05-09T06:36:11.063055+0000","last_clean":"2024-05-09T06:36:11.063055+0000","last_became_active":"2024-05-09T06:36:06.864434+0000","last_became_peered":"2024-05-09T06:36:06.864434+0000","last_unstale":"2024-05-09T06:36:11.063055+0000","last_undegraded":"2024-05-09T06:36:11.063055+0000","last_fullsized":"2024-05-09T06:36:11.063055+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_clean_scrub_stamp":"2024-05-09T06:36:05.833229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:16:34.559063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T06:36:08.862189+0000","last_change":"2024-05-09T06:36:07.870720+0000","last_active":"2024-05-09T06:36:08.862189+0000","last_peered":"2024-05-09T06:36:08.862189+0000","last_clean":"2024-05-09T06:36:08.862189+0000","last_became_active":"2024-05-09T06:36:07.870431+0000","last_became_peered":"2024-05-09T06:36:07.870431+0000","last_unstale":"2024-05-09T06:36:08.862189+0000","last_undegraded":"2024-05-09T06:36:08.862189+0000","last_fullsized":"2024-05-09T06:36:08.862189+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:51:24.938206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T06:36:09.855474+0000","last_change":"2024-05-09T06:36:06.866523+0000","last_active":"2024-05-09T06:36:09.855474+0000","last_peered":"2024-05-09T06:36:09.855474+0000","last_clean":"2024-05-09T06:36:09.855474+0000","last_became_active":"2024-05-09T06:36:06.866220+0000","last_became_peered":"2024-05-09T06:36:06.866220+0000","last_unstale":"2024-05-09T06:36:09.855474+0000","last_undegraded":"2024-05-09T06:36:09.855474+0000","last_fullsized":"2024-05-09T06:36:09.855474+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_clean_scrub_stamp":"2024-05-09T06:36:05.833229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:40:41.085704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T06:36:11.063367+0000","last_change":"2024-05-09T06:36:07.877537+0000","last_active":"2024-05-09T06:36:11.063367+0000","last_peered":"2024-05-09T06:36:11.063367+0000","last_clean":"2024-05-09T06:36:11.063367+0000","last_became_active":"2024-05-09T06:36:07.877261+0000","last_became_peered":"2024-05-09T06:36:07.877261+0000","last_unstale":"2024-05-09T06:36:11.063367+0000","last_undegraded":"2024-05-09T06:36:11.063367+0000","last_fullsized":"2024-05-09T06:36:11.063367+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:24:23.190085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T06:36:11.063314+0000","last_change":"2024-05-09T06:36:06.864863+0000","last_active":"2024-05-09T06:36:11.063314+0000","last_peered":"2024-05-09T06:36:11.063314+0000","last_clean":"2024-05-09T06:36:11.063314+0000","last_became_active":"2024-05-09T06:36:06.864300+0000","last_became_peered":"2024-05-09T06:36:06.864300+0000","last_unstale":"2024-05-09T06:36:11.063314+0000","last_undegraded":"2024-05-09T06:36:11.063314+0000","last_fullsized":"2024-05-09T06:36:11.063314+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_clean_scrub_stamp":"2024-05-09T06:36:05.833229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:15:49.248390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T06:36:11.063876+0000","last_change":"2024-05-09T06:36:07.877642+0000","last_active":"2024-05-09T06:36:11.063876+0000","last_peered":"2024-05-09T06:36:11.063876+0000","last_clean":"2024-05-09T06:36:11.063876+0000","last_became_active":"2024-05-09T06:36:07.877358+0000","last_became_peered":"2024-05-09T06:36:07.877358+0000","last_unstale":"2024-05-09T06:36:11.063876+0000","last_undegraded":"2024-05-09T06:36:11.063876+0000","last_fullsized":"2024-05-09T06:36:11.063876+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:19:58.370458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T06:36:08.862471+0000","last_change":"2024-05-09T06:36:06.865905+0000","last_active":"2024-05-09T06:36:08.862471+0000","last_peered":"2024-05-09T06:36:08.862471+0000","last_clean":"2024-05-09T06:36:08.862471+0000","last_became_active":"2024-05-09T06:36:06.865528+0000","last_became_peered":"2024-05-09T06:36:06.865528+0000","last_unstale":"2024-05-09T06:36:08.862471+0000","last_undegraded":"2024-05-09T06:36:08.862471+0000","last_fullsized":"2024-05-09T06:36:08.862471+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_clean_scrub_stamp":"2024-05-09T06:36:05.833229+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:40:41.085704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T06:36:08.864334+0000","last_change":"2024-05-09T06:36:07.864517+0000","last_active":"2024-05-09T06:36:08.864334+0000","last_peered":"2024-05-09T06:36:08.864334+0000","last_clean":"2024-05-09T06:36:08.864334+0000","last_became_active":"2024-05-09T06:36:07.864076+0000","last_became_peered":"2024-05-09T06:36:07.864076+0000","last_unstale":"2024-05-09T06:36:08.864334+0000","last_undegraded":"2024-05-09T06:36:08.864334+0000","last_fullsized":"2024-05-09T06:36:08.864334+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:03:39.564328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T06:36:09.855507+0000","last_change":"2024-05-09T06:36:06.862854+0000","last_active":"2024-05-09T06:36:09.855507+0000","last_peered":"2024-05-09T06:36:09.855507+0000","last_clean":"2024-05-09T06:36:09.855507+0000","last_became_active":"2024-05-09T06:36:06.862538+0000","last_became_peered":"2024-05-09T06:36:06.862538+0000","last_unstale":"2024-05-09T06:36:09.855507+0000","last_undegraded":"2024-05-09T06:36:09.855507+0000","last_fullsized":"2024-05-09T06:36:09.855507+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_clean_scrub_stamp":"2024-05-09T06:36:05.833229+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:08:20.004456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T06:36:10.066521+0000","last_change":"2024-05-09T06:36:07.872477+0000","last_active":"2024-05-09T06:36:10.066521+0000","last_peered":"2024-05-09T06:36:10.066521+0000","last_clean":"2024-05-09T06:36:10.066521+0000","last_became_active":"2024-05-09T06:36:07.872181+0000","last_became_peered":"2024-05-09T06:36:07.872181+0000","last_unstale":"2024-05-09T06:36:10.066521+0000","last_undegraded":"2024-05-09T06:36:10.066521+0000","last_fullsized":"2024-05-09T06:36:10.066521+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:24:23.190085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T06:36:11.063897+0000","last_change":"2024-05-09T06:36:06.861757+0000","last_active":"2024-05-09T06:36:11.063897+0000","last_peered":"2024-05-09T06:36:11.063897+0000","last_clean":"2024-05-09T06:36:11.063897+0000","last_became_active":"2024-05-09T06:36:06.861434+0000","last_became_peered":"2024-05-09T06:36:06.861434+0000","last_unstale":"2024-05-09T06:36:11.063897+0000","last_undegraded":"2024-05-09T06:36:11.063897+0000","last_fullsized":"2024-05-09T06:36:11.063897+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_clean_scrub_stamp":"2024-05-09T06:36:05.833229+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:56:44.324093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T06:36:11.063318+0000","last_change":"2024-05-09T06:36:07.873646+0000","last_active":"2024-05-09T06:36:11.063318+0000","last_peered":"2024-05-09T06:36:11.063318+0000","last_clean":"2024-05-09T06:36:11.063318+0000","last_became_active":"2024-05-09T06:36:07.872272+0000","last_became_peered":"2024-05-09T06:36:07.872272+0000","last_unstale":"2024-05-09T06:36:11.063318+0000","last_undegraded":"2024-05-09T06:36:11.063318+0000","last_fullsized":"2024-05-09T06:36:11.063318+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:53:50.215704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T06:36:11.063298+0000","last_change":"2024-05-09T06:36:06.865709+0000","last_active":"2024-05-09T06:36:11.063298+0000","last_peered":"2024-05-09T06:36:11.063298+0000","last_clean":"2024-05-09T06:36:11.063298+0000","last_became_active":"2024-05-09T06:36:06.864907+0000","last_became_peered":"2024-05-09T06:36:06.864907+0000","last_unstale":"2024-05-09T06:36:11.063298+0000","last_undegraded":"2024-05-09T06:36:11.063298+0000","last_fullsized":"2024-05-09T06:36:11.063298+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_clean_scrub_stamp":"2024-05-09T06:36:05.833229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:40:41.085704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T06:36:10.066476+0000","last_change":"2024-05-09T06:36:07.873795+0000","last_active":"2024-05-09T06:36:10.066476+0000","last_peered":"2024-05-09T06:36:10.066476+0000","last_clean":"2024-05-09T06:36:10.066476+0000","last_became_active":"2024-05-09T06:36:07.873491+0000","last_became_peered":"2024-05-09T06:36:07.873491+0000","last_unstale":"2024-05-09T06:36:10.066476+0000","last_undegraded":"2024-05-09T06:36:10.066476+0000","last_fullsized":"2024-05-09T06:36:10.066476+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:56:41.722475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T06:36:09.855141+0000","last_change":"2024-05-09T06:36:06.862067+0000","last_active":"2024-05-09T06:36:09.855141+0000","last_peered":"2024-05-09T06:36:09.855141+0000","last_clean":"2024-05-09T06:36:09.855141+0000","last_became_active":"2024-05-09T06:36:06.861791+0000","last_became_peered":"2024-05-09T06:36:06.861791+0000","last_unstale":"2024-05-09T06:36:09.855141+0000","last_undegraded":"2024-05-09T06:36:09.855141+0000","last_fullsized":"2024-05-09T06:36:09.855141+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_clean_scrub_stamp":"2024-05-09T06:36:05.833229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:03:17.189398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T06:36:09.855531+0000","last_change":"2024-05-09T06:36:07.867637+0000","last_active":"2024-05-09T06:36:09.855531+0000","last_peered":"2024-05-09T06:36:09.855531+0000","last_clean":"2024-05-09T06:36:09.855531+0000","last_became_active":"2024-05-09T06:36:07.867028+0000","last_became_peered":"2024-05-09T06:36:07.867028+0000","last_unstale":"2024-05-09T06:36:09.855531+0000","last_undegraded":"2024-05-09T06:36:09.855531+0000","last_fullsized":"2024-05-09T06:36:09.855531+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:51:24.938206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":22,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T06:36:11.063615+0000","last_change":"2024-05-09T06:36:06.852528+0000","last_active":"2024-05-09T06:36:11.063615+0000","last_peered":"2024-05-09T06:36:11.063615+0000","last_clean":"2024-05-09T06:36:11.063615+0000","last_became_active":"2024-05-09T06:36:06.852202+0000","last_became_peered":"2024-05-09T06:36:06.852202+0000","last_unstale":"2024-05-09T06:36:11.063615+0000","last_undegraded":"2024-05-09T06:36:11.063615+0000","last_fullsized":"2024-05-09T06:36:11.063615+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_clean_scrub_stamp":"2024-05-09T06:36:05.833229+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:08:20.004456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T06:36:10.066312+0000","last_change":"2024-05-09T06:36:07.867532+0000","last_active":"2024-05-09T06:36:10.066312+0000","last_peered":"2024-05-09T06:36:10.066312+0000","last_clean":"2024-05-09T06:36:10.066312+0000","last_became_active":"2024-05-09T06:36:07.867262+0000","last_became_peered":"2024-05-09T06:36:07.867262+0000","last_unstale":"2024-05-09T06:36:10.066312+0000","last_undegraded":"2024-05-09T06:36:10.066312+0000","last_fullsized":"2024-05-09T06:36:10.066312+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:03:16.134498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T06:36:08.864156+0000","last_change":"2024-05-09T06:36:06.863522+0000","last_active":"2024-05-09T06:36:08.864156+0000","last_peered":"2024-05-09T06:36:08.864156+0000","last_clean":"2024-05-09T06:36:08.864156+0000","last_became_active":"2024-05-09T06:36:06.863099+0000","last_became_peered":"2024-05-09T06:36:06.863099+0000","last_unstale":"2024-05-09T06:36:08.864156+0000","last_undegraded":"2024-05-09T06:36:08.864156+0000","last_fullsized":"2024-05-09T06:36:08.864156+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_clean_scrub_stamp":"2024-05-09T06:36:05.833229+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:56:44.324093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T06:36:10.066683+0000","last_change":"2024-05-09T06:36:07.868102+0000","last_active":"2024-05-09T06:36:10.066683+0000","last_peered":"2024-05-09T06:36:10.066683+0000","last_clean":"2024-05-09T06:36:10.066683+0000","last_became_active":"2024-05-09T06:36:07.867800+0000","last_became_peered":"2024-05-09T06:36:07.867800+0000","last_unstale":"2024-05-09T06:36:10.066683+0000","last_undegraded":"2024-05-09T06:36:10.066683+0000","last_fullsized":"2024-05-09T06:36:10.066683+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:10:28.010336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T06:36:11.063508+0000","last_change":"2024-05-09T06:36:06.861458+0000","last_active":"2024-05-09T06:36:11.063508+0000","last_peered":"2024-05-09T06:36:11.063508+0000","last_clean":"2024-05-09T06:36:11.063508+0000","last_became_active":"2024-05-09T06:36:06.861098+0000","last_became_peered":"2024-05-09T06:36:06.861098+0000","last_unstale":"2024-05-09T06:36:11.063508+0000","last_undegraded":"2024-05-09T06:36:11.063508+0000","last_fullsized":"2024-05-09T06:36:11.063508+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_clean_scrub_stamp":"2024-05-09T06:36:05.833229+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:15:49.248390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T06:36:10.066945+0000","last_change":"2024-05-09T06:36:07.868487+0000","last_active":"2024-05-09T06:36:10.066945+0000","last_peered":"2024-05-09T06:36:10.066945+0000","last_clean":"2024-05-09T06:36:10.066945+0000","last_became_active":"2024-05-09T06:36:07.867705+0000","last_became_peered":"2024-05-09T06:36:07.867705+0000","last_unstale":"2024-05-09T06:36:10.066945+0000","last_undegraded":"2024-05-09T06:36:10.066945+0000","last_fullsized":"2024-05-09T06:36:10.066945+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:04:27.234790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T06:36:08.862567+0000","last_change":"2024-05-09T06:36:06.858041+0000","last_active":"2024-05-09T06:36:08.862567+0000","last_peered":"2024-05-09T06:36:08.862567+0000","last_clean":"2024-05-09T06:36:08.862567+0000","last_became_active":"2024-05-09T06:36:06.857745+0000","last_became_peered":"2024-05-09T06:36:06.857745+0000","last_unstale":"2024-05-09T06:36:08.862567+0000","last_undegraded":"2024-05-09T06:36:08.862567+0000","last_fullsized":"2024-05-09T06:36:08.862567+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_clean_scrub_stamp":"2024-05-09T06:36:05.833229+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:58:18.159070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T06:36:11.063484+0000","last_change":"2024-05-09T06:36:07.876459+0000","last_active":"2024-05-09T06:36:11.063484+0000","last_peered":"2024-05-09T06:36:11.063484+0000","last_clean":"2024-05-09T06:36:11.063484+0000","last_became_active":"2024-05-09T06:36:07.876151+0000","last_became_peered":"2024-05-09T06:36:07.876151+0000","last_unstale":"2024-05-09T06:36:11.063484+0000","last_undegraded":"2024-05-09T06:36:11.063484+0000","last_fullsized":"2024-05-09T06:36:11.063484+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:45:01.680949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T06:36:11.062797+0000","last_change":"2024-05-09T06:36:06.866239+0000","last_active":"2024-05-09T06:36:11.062797+0000","last_peered":"2024-05-09T06:36:11.062797+0000","last_clean":"2024-05-09T06:36:11.062797+0000","last_became_active":"2024-05-09T06:36:06.865929+0000","last_became_peered":"2024-05-09T06:36:06.865929+0000","last_unstale":"2024-05-09T06:36:11.062797+0000","last_undegraded":"2024-05-09T06:36:11.062797+0000","last_fullsized":"2024-05-09T06:36:11.062797+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_clean_scrub_stamp":"2024-05-09T06:36:05.833229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:03:17.189398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T06:36:10.066456+0000","last_change":"2024-05-09T06:36:07.877578+0000","last_active":"2024-05-09T06:36:10.066456+0000","last_peered":"2024-05-09T06:36:10.066456+0000","last_clean":"2024-05-09T06:36:10.066456+0000","last_became_active":"2024-05-09T06:36:07.877319+0000","last_became_peered":"2024-05-09T06:36:07.877319+0000","last_unstale":"2024-05-09T06:36:10.066456+0000","last_undegraded":"2024-05-09T06:36:10.066456+0000","last_fullsized":"2024-05-09T06:36:10.066456+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:47:30.595382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":170,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T06:36:10.066325+0000","last_change":"2024-05-09T06:35:55.895214+0000","last_active":"2024-05-09T06:36:10.066325+0000","last_peered":"2024-05-09T06:36:10.066325+0000","last_clean":"2024-05-09T06:36:10.066325+0000","last_became_active":"2024-05-09T06:35:55.894087+0000","last_became_peered":"2024-05-09T06:35:55.894087+0000","last_unstale":"2024-05-09T06:36:10.066325+0000","last_undegraded":"2024-05-09T06:36:10.066325+0000","last_fullsized":"2024-05-09T06:36:10.066325+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:35:54.820102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:35:54.820102+0000","last_clean_scrub_stamp":"2024-05-09T06:35:54.820102+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:29:20.408099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T06:36:11.062960+0000","last_change":"2024-05-09T06:36:06.864900+0000","last_active":"2024-05-09T06:36:11.062960+0000","last_peered":"2024-05-09T06:36:11.062960+0000","last_clean":"2024-05-09T06:36:11.062960+0000","last_became_active":"2024-05-09T06:36:06.864169+0000","last_became_peered":"2024-05-09T06:36:06.864169+0000","last_unstale":"2024-05-09T06:36:11.062960+0000","last_undegraded":"2024-05-09T06:36:11.062960+0000","last_fullsized":"2024-05-09T06:36:11.062960+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_clean_scrub_stamp":"2024-05-09T06:36:05.833229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:03:17.189398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T06:36:11.063181+0000","last_change":"2024-05-09T06:36:07.880436+0000","last_active":"2024-05-09T06:36:11.063181+0000","last_peered":"2024-05-09T06:36:11.063181+0000","last_clean":"2024-05-09T06:36:11.063181+0000","last_became_active":"2024-05-09T06:36:07.879931+0000","last_became_peered":"2024-05-09T06:36:07.879931+0000","last_unstale":"2024-05-09T06:36:11.063181+0000","last_undegraded":"2024-05-09T06:36:11.063181+0000","last_fullsized":"2024-05-09T06:36:11.063181+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:34:13.971998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T06:36:10.066153+0000","last_change":"2024-05-09T06:36:06.860567+0000","last_active":"2024-05-09T06:36:10.066153+0000","last_peered":"2024-05-09T06:36:10.066153+0000","last_clean":"2024-05-09T06:36:10.066153+0000","last_became_active":"2024-05-09T06:36:06.860284+0000","last_became_peered":"2024-05-09T06:36:06.860284+0000","last_unstale":"2024-05-09T06:36:10.066153+0000","last_undegraded":"2024-05-09T06:36:10.066153+0000","last_fullsized":"2024-05-09T06:36:10.066153+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_clean_scrub_stamp":"2024-05-09T06:36:05.833229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:03:38.559114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T06:36:11.062933+0000","last_change":"2024-05-09T06:36:07.865957+0000","last_active":"2024-05-09T06:36:11.062933+0000","last_peered":"2024-05-09T06:36:11.062933+0000","last_clean":"2024-05-09T06:36:11.062933+0000","last_became_active":"2024-05-09T06:36:07.865683+0000","last_became_peered":"2024-05-09T06:36:07.865683+0000","last_unstale":"2024-05-09T06:36:11.062933+0000","last_undegraded":"2024-05-09T06:36:11.062933+0000","last_fullsized":"2024-05-09T06:36:11.062933+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:47:56.779394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T06:36:11.062927+0000","last_change":"2024-05-09T06:36:06.862232+0000","last_active":"2024-05-09T06:36:11.062927+0000","last_peered":"2024-05-09T06:36:11.062927+0000","last_clean":"2024-05-09T06:36:11.062927+0000","last_became_active":"2024-05-09T06:36:06.861918+0000","last_became_peered":"2024-05-09T06:36:06.861918+0000","last_unstale":"2024-05-09T06:36:11.062927+0000","last_undegraded":"2024-05-09T06:36:11.062927+0000","last_fullsized":"2024-05-09T06:36:11.062927+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_clean_scrub_stamp":"2024-05-09T06:36:05.833229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:23:13.976894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T06:36:08.862162+0000","last_change":"2024-05-09T06:36:07.865850+0000","last_active":"2024-05-09T06:36:08.862162+0000","last_peered":"2024-05-09T06:36:08.862162+0000","last_clean":"2024-05-09T06:36:08.862162+0000","last_became_active":"2024-05-09T06:36:07.865520+0000","last_became_peered":"2024-05-09T06:36:07.865520+0000","last_unstale":"2024-05-09T06:36:08.862162+0000","last_undegraded":"2024-05-09T06:36:08.862162+0000","last_fullsized":"2024-05-09T06:36:08.862162+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:44:00.455366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T06:36:11.063482+0000","last_change":"2024-05-09T06:36:06.865545+0000","last_active":"2024-05-09T06:36:11.063482+0000","last_peered":"2024-05-09T06:36:11.063482+0000","last_clean":"2024-05-09T06:36:11.063482+0000","last_became_active":"2024-05-09T06:36:06.865255+0000","last_became_peered":"2024-05-09T06:36:06.865255+0000","last_unstale":"2024-05-09T06:36:11.063482+0000","last_undegraded":"2024-05-09T06:36:11.063482+0000","last_fullsized":"2024-05-09T06:36:11.063482+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_clean_scrub_stamp":"2024-05-09T06:36:05.833229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:59:29.736305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T06:36:08.864221+0000","last_change":"2024-05-09T06:36:07.867264+0000","last_active":"2024-05-09T06:36:08.864221+0000","last_peered":"2024-05-09T06:36:08.864221+0000","last_clean":"2024-05-09T06:36:08.864221+0000","last_became_active":"2024-05-09T06:36:07.866981+0000","last_became_peered":"2024-05-09T06:36:07.866981+0000","last_unstale":"2024-05-09T06:36:08.864221+0000","last_undegraded":"2024-05-09T06:36:08.864221+0000","last_fullsized":"2024-05-09T06:36:08.864221+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:16:35.564277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T06:36:08.862365+0000","last_change":"2024-05-09T06:36:06.853399+0000","last_active":"2024-05-09T06:36:08.862365+0000","last_peered":"2024-05-09T06:36:08.862365+0000","last_clean":"2024-05-09T06:36:08.862365+0000","last_became_active":"2024-05-09T06:36:06.852409+0000","last_became_peered":"2024-05-09T06:36:06.852409+0000","last_unstale":"2024-05-09T06:36:08.862365+0000","last_undegraded":"2024-05-09T06:36:08.862365+0000","last_fullsized":"2024-05-09T06:36:08.862365+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_clean_scrub_stamp":"2024-05-09T06:36:05.833229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:15:49.248390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T06:36:11.063331+0000","last_change":"2024-05-09T06:36:07.875455+0000","last_active":"2024-05-09T06:36:11.063331+0000","last_peered":"2024-05-09T06:36:11.063331+0000","last_clean":"2024-05-09T06:36:11.063331+0000","last_became_active":"2024-05-09T06:36:07.875153+0000","last_became_peered":"2024-05-09T06:36:07.875153+0000","last_unstale":"2024-05-09T06:36:11.063331+0000","last_undegraded":"2024-05-09T06:36:11.063331+0000","last_fullsized":"2024-05-09T06:36:11.063331+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:10:28.010336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T06:36:10.066710+0000","last_change":"2024-05-09T06:36:06.861152+0000","last_active":"2024-05-09T06:36:10.066710+0000","last_peered":"2024-05-09T06:36:10.066710+0000","last_clean":"2024-05-09T06:36:10.066710+0000","last_became_active":"2024-05-09T06:36:06.860905+0000","last_became_peered":"2024-05-09T06:36:06.860905+0000","last_unstale":"2024-05-09T06:36:10.066710+0000","last_undegraded":"2024-05-09T06:36:10.066710+0000","last_fullsized":"2024-05-09T06:36:10.066710+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_clean_scrub_stamp":"2024-05-09T06:36:05.833229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:59:29.736305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T06:36:08.862257+0000","last_change":"2024-05-09T06:36:07.865602+0000","last_active":"2024-05-09T06:36:08.862257+0000","last_peered":"2024-05-09T06:36:08.862257+0000","last_clean":"2024-05-09T06:36:08.862257+0000","last_became_active":"2024-05-09T06:36:07.865083+0000","last_became_peered":"2024-05-09T06:36:07.865083+0000","last_unstale":"2024-05-09T06:36:08.862257+0000","last_undegraded":"2024-05-09T06:36:08.862257+0000","last_fullsized":"2024-05-09T06:36:08.862257+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:50:02.245694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T06:36:08.862573+0000","last_change":"2024-05-09T06:36:06.855928+0000","last_active":"2024-05-09T06:36:08.862573+0000","last_peered":"2024-05-09T06:36:08.862573+0000","last_clean":"2024-05-09T06:36:08.862573+0000","last_became_active":"2024-05-09T06:36:06.855322+0000","last_became_peered":"2024-05-09T06:36:06.855322+0000","last_unstale":"2024-05-09T06:36:08.862573+0000","last_undegraded":"2024-05-09T06:36:08.862573+0000","last_fullsized":"2024-05-09T06:36:08.862573+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_clean_scrub_stamp":"2024-05-09T06:36:05.833229+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:03:38.559114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T06:36:11.063444+0000","last_change":"2024-05-09T06:36:07.870976+0000","last_active":"2024-05-09T06:36:11.063444+0000","last_peered":"2024-05-09T06:36:11.063444+0000","last_clean":"2024-05-09T06:36:11.063444+0000","last_became_active":"2024-05-09T06:36:07.870713+0000","last_became_peered":"2024-05-09T06:36:07.870713+0000","last_unstale":"2024-05-09T06:36:11.063444+0000","last_undegraded":"2024-05-09T06:36:11.063444+0000","last_fullsized":"2024-05-09T06:36:11.063444+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:51:24.938206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T06:36:09.855506+0000","last_change":"2024-05-09T06:36:06.863328+0000","last_active":"2024-05-09T06:36:09.855506+0000","last_peered":"2024-05-09T06:36:09.855506+0000","last_clean":"2024-05-09T06:36:09.855506+0000","last_became_active":"2024-05-09T06:36:06.862686+0000","last_became_peered":"2024-05-09T06:36:06.862686+0000","last_unstale":"2024-05-09T06:36:09.855506+0000","last_undegraded":"2024-05-09T06:36:09.855506+0000","last_fullsized":"2024-05-09T06:36:09.855506+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_clean_scrub_stamp":"2024-05-09T06:36:05.833229+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:59:29.736305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T06:36:11.063488+0000","last_change":"2024-05-09T06:36:07.868926+0000","last_active":"2024-05-09T06:36:11.063488+0000","last_peered":"2024-05-09T06:36:11.063488+0000","last_clean":"2024-05-09T06:36:11.063488+0000","last_became_active":"2024-05-09T06:36:07.868613+0000","last_became_peered":"2024-05-09T06:36:07.868613+0000","last_unstale":"2024-05-09T06:36:11.063488+0000","last_undegraded":"2024-05-09T06:36:11.063488+0000","last_fullsized":"2024-05-09T06:36:11.063488+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:24:51.534066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T06:36:10.066153+0000","last_change":"2024-05-09T06:36:06.861104+0000","last_active":"2024-05-09T06:36:10.066153+0000","last_peered":"2024-05-09T06:36:10.066153+0000","last_clean":"2024-05-09T06:36:10.066153+0000","last_became_active":"2024-05-09T06:36:06.860799+0000","last_became_peered":"2024-05-09T06:36:06.860799+0000","last_unstale":"2024-05-09T06:36:10.066153+0000","last_undegraded":"2024-05-09T06:36:10.066153+0000","last_fullsized":"2024-05-09T06:36:10.066153+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_clean_scrub_stamp":"2024-05-09T06:36:05.833229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:58:18.159070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T06:36:08.863915+0000","last_change":"2024-05-09T06:36:07.863846+0000","last_active":"2024-05-09T06:36:08.863915+0000","last_peered":"2024-05-09T06:36:08.863915+0000","last_clean":"2024-05-09T06:36:08.863915+0000","last_became_active":"2024-05-09T06:36:07.863161+0000","last_became_peered":"2024-05-09T06:36:07.863161+0000","last_unstale":"2024-05-09T06:36:08.863915+0000","last_undegraded":"2024-05-09T06:36:08.863915+0000","last_fullsized":"2024-05-09T06:36:08.863915+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:59:30.741519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T06:36:08.862295+0000","last_change":"2024-05-09T06:36:06.861276+0000","last_active":"2024-05-09T06:36:08.862295+0000","last_peered":"2024-05-09T06:36:08.862295+0000","last_clean":"2024-05-09T06:36:08.862295+0000","last_became_active":"2024-05-09T06:36:06.860977+0000","last_became_peered":"2024-05-09T06:36:06.860977+0000","last_unstale":"2024-05-09T06:36:08.862295+0000","last_undegraded":"2024-05-09T06:36:08.862295+0000","last_fullsized":"2024-05-09T06:36:08.862295+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_clean_scrub_stamp":"2024-05-09T06:36:05.833229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:16:34.559063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T06:36:11.063737+0000","last_change":"2024-05-09T06:36:07.870964+0000","last_active":"2024-05-09T06:36:11.063737+0000","last_peered":"2024-05-09T06:36:11.063737+0000","last_clean":"2024-05-09T06:36:11.063737+0000","last_became_active":"2024-05-09T06:36:07.870669+0000","last_became_peered":"2024-05-09T06:36:07.870669+0000","last_unstale":"2024-05-09T06:36:11.063737+0000","last_undegraded":"2024-05-09T06:36:11.063737+0000","last_fullsized":"2024-05-09T06:36:11.063737+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:24:50.227387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T06:36:08.862272+0000","last_change":"2024-05-09T06:36:06.861162+0000","last_active":"2024-05-09T06:36:08.862272+0000","last_peered":"2024-05-09T06:36:08.862272+0000","last_clean":"2024-05-09T06:36:08.862272+0000","last_became_active":"2024-05-09T06:36:06.860880+0000","last_became_peered":"2024-05-09T06:36:06.860880+0000","last_unstale":"2024-05-09T06:36:08.862272+0000","last_undegraded":"2024-05-09T06:36:08.862272+0000","last_fullsized":"2024-05-09T06:36:08.862272+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_clean_scrub_stamp":"2024-05-09T06:36:05.833229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:59:29.736305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T06:36:11.063967+0000","last_change":"2024-05-09T06:36:07.872872+0000","last_active":"2024-05-09T06:36:11.063967+0000","last_peered":"2024-05-09T06:36:11.063967+0000","last_clean":"2024-05-09T06:36:11.063967+0000","last_became_active":"2024-05-09T06:36:07.872556+0000","last_became_peered":"2024-05-09T06:36:07.872556+0000","last_unstale":"2024-05-09T06:36:11.063967+0000","last_undegraded":"2024-05-09T06:36:11.063967+0000","last_fullsized":"2024-05-09T06:36:11.063967+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:25:55.622101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T06:36:10.066553+0000","last_change":"2024-05-09T06:36:06.862160+0000","last_active":"2024-05-09T06:36:10.066553+0000","last_peered":"2024-05-09T06:36:10.066553+0000","last_clean":"2024-05-09T06:36:10.066553+0000","last_became_active":"2024-05-09T06:36:06.861881+0000","last_became_peered":"2024-05-09T06:36:06.861881+0000","last_unstale":"2024-05-09T06:36:10.066553+0000","last_undegraded":"2024-05-09T06:36:10.066553+0000","last_fullsized":"2024-05-09T06:36:10.066553+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_clean_scrub_stamp":"2024-05-09T06:36:05.833229+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:16:34.559063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T06:36:09.855248+0000","last_change":"2024-05-09T06:36:07.869000+0000","last_active":"2024-05-09T06:36:09.855248+0000","last_peered":"2024-05-09T06:36:09.855248+0000","last_clean":"2024-05-09T06:36:09.855248+0000","last_became_active":"2024-05-09T06:36:07.868522+0000","last_became_peered":"2024-05-09T06:36:07.868522+0000","last_unstale":"2024-05-09T06:36:09.855248+0000","last_undegraded":"2024-05-09T06:36:09.855248+0000","last_fullsized":"2024-05-09T06:36:09.855248+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:24:51.534066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T06:36:11.063243+0000","last_change":"2024-05-09T06:36:07.875476+0000","last_active":"2024-05-09T06:36:11.063243+0000","last_peered":"2024-05-09T06:36:11.063243+0000","last_clean":"2024-05-09T06:36:11.063243+0000","last_became_active":"2024-05-09T06:36:07.875170+0000","last_became_peered":"2024-05-09T06:36:07.875170+0000","last_unstale":"2024-05-09T06:36:11.063243+0000","last_undegraded":"2024-05-09T06:36:11.063243+0000","last_fullsized":"2024-05-09T06:36:11.063243+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:25:55.622101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T06:36:10.066379+0000","last_change":"2024-05-09T06:36:06.866061+0000","last_active":"2024-05-09T06:36:10.066379+0000","last_peered":"2024-05-09T06:36:10.066379+0000","last_clean":"2024-05-09T06:36:10.066379+0000","last_became_active":"2024-05-09T06:36:06.865783+0000","last_became_peered":"2024-05-09T06:36:06.865783+0000","last_unstale":"2024-05-09T06:36:10.066379+0000","last_undegraded":"2024-05-09T06:36:10.066379+0000","last_fullsized":"2024-05-09T06:36:10.066379+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_clean_scrub_stamp":"2024-05-09T06:36:05.833229+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:15:49.248390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T06:36:11.063196+0000","last_change":"2024-05-09T06:36:07.876132+0000","last_active":"2024-05-09T06:36:11.063196+0000","last_peered":"2024-05-09T06:36:11.063196+0000","last_clean":"2024-05-09T06:36:11.063196+0000","last_became_active":"2024-05-09T06:36:07.874679+0000","last_became_peered":"2024-05-09T06:36:07.874679+0000","last_unstale":"2024-05-09T06:36:11.063196+0000","last_undegraded":"2024-05-09T06:36:11.063196+0000","last_fullsized":"2024-05-09T06:36:11.063196+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:17:19.091125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T06:36:10.066569+0000","last_change":"2024-05-09T06:36:06.860619+0000","last_active":"2024-05-09T06:36:10.066569+0000","last_peered":"2024-05-09T06:36:10.066569+0000","last_clean":"2024-05-09T06:36:10.066569+0000","last_became_active":"2024-05-09T06:36:06.860323+0000","last_became_peered":"2024-05-09T06:36:06.860323+0000","last_unstale":"2024-05-09T06:36:10.066569+0000","last_undegraded":"2024-05-09T06:36:10.066569+0000","last_fullsized":"2024-05-09T06:36:10.066569+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_clean_scrub_stamp":"2024-05-09T06:36:05.833229+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:40:41.085704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T06:36:11.063734+0000","last_change":"2024-05-09T06:36:07.880245+0000","last_active":"2024-05-09T06:36:11.063734+0000","last_peered":"2024-05-09T06:36:11.063734+0000","last_clean":"2024-05-09T06:36:11.063734+0000","last_became_active":"2024-05-09T06:36:07.879926+0000","last_became_peered":"2024-05-09T06:36:07.879926+0000","last_unstale":"2024-05-09T06:36:11.063734+0000","last_undegraded":"2024-05-09T06:36:11.063734+0000","last_fullsized":"2024-05-09T06:36:11.063734+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:50:02.245694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T06:36:11.063354+0000","last_change":"2024-05-09T06:36:06.864934+0000","last_active":"2024-05-09T06:36:11.063354+0000","last_peered":"2024-05-09T06:36:11.063354+0000","last_clean":"2024-05-09T06:36:11.063354+0000","last_became_active":"2024-05-09T06:36:06.864680+0000","last_became_peered":"2024-05-09T06:36:06.864680+0000","last_unstale":"2024-05-09T06:36:11.063354+0000","last_undegraded":"2024-05-09T06:36:11.063354+0000","last_fullsized":"2024-05-09T06:36:11.063354+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_clean_scrub_stamp":"2024-05-09T06:36:05.833229+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:03:17.189398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T06:36:11.063160+0000","last_change":"2024-05-09T06:36:07.870194+0000","last_active":"2024-05-09T06:36:11.063160+0000","last_peered":"2024-05-09T06:36:11.063160+0000","last_clean":"2024-05-09T06:36:11.063160+0000","last_became_active":"2024-05-09T06:36:07.869895+0000","last_became_peered":"2024-05-09T06:36:07.869895+0000","last_unstale":"2024-05-09T06:36:11.063160+0000","last_undegraded":"2024-05-09T06:36:11.063160+0000","last_fullsized":"2024-05-09T06:36:11.063160+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:06.838443+0000","last_clean_scrub_stamp":"2024-05-09T06:36:06.838443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:44:00.455366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T06:36:11.064034+0000","last_change":"2024-05-09T06:36:06.861090+0000","last_active":"2024-05-09T06:36:11.064034+0000","last_peered":"2024-05-09T06:36:11.064034+0000","last_clean":"2024-05-09T06:36:11.064034+0000","last_became_active":"2024-05-09T06:36:06.860774+0000","last_became_peered":"2024-05-09T06:36:06.860774+0000","last_unstale":"2024-05-09T06:36:11.064034+0000","last_undegraded":"2024-05-09T06:36:11.064034+0000","last_fullsized":"2024-05-09T06:36:11.064034+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:05.833229+0000","last_clean_scrub_stamp":"2024-05-09T06:36:05.833229+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:05:58.090556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574856,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27804,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344036,"statfs":{"total":96636764160,"available":96608292864,"internally_reserved":0,"allocated":802816,"data_stored":661884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574856,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574855,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":208896,"data_stored":71024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574855,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":71060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574856,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574855,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":204800,"data_stored":70934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574855,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":212992,"data_stored":71094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574856,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27812,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344028,"statfs":{"total":96636764160,"available":96608284672,"internally_reserved":0,"allocated":811008,"data_stored":661940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-09T06:36:16.721 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-05-09T06:36:16.721 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-05-09T06:36:16.721 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-05-09T06:36:16.721 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-05-09T06:36:16.860 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:16.879+0000 7f80c42da640 1 Processor -- start 2024-05-09T06:36:16.860 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:16.879+0000 7f80c42da640 1 -- start start 2024-05-09T06:36:16.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:16.880+0000 7f80c42da640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f80bc15e470 0x7f80bc0d2550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:16.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:16.880+0000 7f80c42da640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f80bc15e850 con 0x7f80bc0cf660 2024-05-09T06:36:16.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:16.880+0000 7f80c42da640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f80bc0d2bc0 con 0x7f80bc0d3c60 2024-05-09T06:36:16.862 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:16.880+0000 7f80c42da640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f80bc0d2ea0 con 0x7f80bc15e470 2024-05-09T06:36:16.862 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:16.880+0000 7f80c2076640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f80bc15e470 0x7f80bc0d2550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:16.862 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:16.880+0000 7f80c2877640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f80bc0cf660 0x7f80bc0b72b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:49296/0 (socket says 172.21.15.3:49296) 2024-05-09T06:36:16.862 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:16.880+0000 7f80c2877640 1 -- 172.21.15.3:0/1752382762 learned_addr learned my addr 172.21.15.3:0/1752382762 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-05-09T06:36:16.862 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:16.881+0000 7f80c2076640 1 -- 172.21.15.3:0/1752382762 >> v1:172.21.15.190:6789/0 conn(0x7f80bc0d3c60 legacy=0x7f80bc0cef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:16.862 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:16.881+0000 7f80c2076640 1 -- 172.21.15.3:0/1752382762 >> v1:172.21.15.3:6789/0 conn(0x7f80bc0cf660 legacy=0x7f80bc0b72b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:16.862 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:16.881+0000 7f80c2076640 1 -- 172.21.15.3:0/1752382762 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80bc0d3180 con 0x7f80bc15e470 2024-05-09T06:36:16.863 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:16.881+0000 7f80c2076640 1 --2- 172.21.15.3:0/1752382762 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f80bc15e470 0x7f80bc0d2550 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f80b40097a0 tx=0x7f80b402a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98df914317f7aeb8 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:16.863 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:16.882+0000 7f80c1074640 1 -- 172.21.15.3:0/1752382762 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80b4037070 con 0x7f80bc15e470 2024-05-09T06:36:16.863 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:16.882+0000 7f80c1074640 1 -- 172.21.15.3:0/1752382762 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f80b402ae50 con 0x7f80bc15e470 2024-05-09T06:36:16.863 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:16.882+0000 7f80c1074640 1 -- 172.21.15.3:0/1752382762 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80b4032660 con 0x7f80bc15e470 2024-05-09T06:36:16.863 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:16.882+0000 7f80c42da640 1 -- 172.21.15.3:0/1752382762 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f80bc15e470 msgr2=0x7f80bc0d2550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:16.863 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:16.882+0000 7f80c42da640 1 --2- 172.21.15.3:0/1752382762 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f80bc15e470 0x7f80bc0d2550 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f80b40097a0 tx=0x7f80b402a140 comp rx=0 tx=0).stop 2024-05-09T06:36:16.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:16.882+0000 7f80c42da640 1 -- 172.21.15.3:0/1752382762 shutdown_connections 2024-05-09T06:36:16.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:16.882+0000 7f80c42da640 1 --2- 172.21.15.3:0/1752382762 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f80bc15e470 0x7f80bc0d2550 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:16.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:16.882+0000 7f80c42da640 1 -- 172.21.15.3:0/1752382762 >> 172.21.15.3:0/1752382762 conn(0x7f80bc0675c0 msgr2=0x7f80bc06c2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:16.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:16.882+0000 7f80c42da640 1 -- 172.21.15.3:0/1752382762 shutdown_connections 2024-05-09T06:36:16.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:16.882+0000 7f80c42da640 1 -- 172.21.15.3:0/1752382762 wait complete. 2024-05-09T06:36:16.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:16.883+0000 7f80c42da640 1 Processor -- start 2024-05-09T06:36:16.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:16.883+0000 7f80c42da640 1 -- start start 2024-05-09T06:36:16.867 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:16.884+0000 7f80c42da640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f80bc0cf660 0x7f80bc0ca3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:16.867 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:16.884+0000 7f80c42da640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f80bc0d3c60 0x7f80bc0cb250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:16.867 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:16.884+0000 7f80c42da640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f80bc15e470 0x7f80bc160160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:16.867 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:16.884+0000 7f80c42da640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f80bc0b9c60 con 0x7f80bc15e470 2024-05-09T06:36:16.867 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:16.884+0000 7f80c42da640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f80bc0d3740 con 0x7f80bc0d3c60 2024-05-09T06:36:16.867 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:16.884+0000 7f80c42da640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f80bc0b9de0 con 0x7f80bc0cf660 2024-05-09T06:36:16.867 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:16.884+0000 7f80c2877640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f80bc15e470 0x7f80bc160160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:16.867 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:16.884+0000 7f80c2877640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f80bc15e470 0x7f80bc160160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49826/0 (socket says 172.21.15.3:49826) 2024-05-09T06:36:16.867 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:16.884+0000 7f80c2877640 1 -- 172.21.15.3:0/2626836924 learned_addr learned my addr 172.21.15.3:0/2626836924 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:36:16.868 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:16.884+0000 7f80c2076640 1 --2- 172.21.15.3:0/2626836924 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f80bc0cf660 0x7f80bc0ca3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:16.868 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:16.884+0000 7f80c2877640 1 -- 172.21.15.3:0/2626836924 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f80bc0cf660 msgr2=0x7f80bc0ca3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:16.868 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:16.884+0000 7f80c1875640 1 --2- 172.21.15.3:0/2626836924 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f80bc0d3c60 0x7f80bc0cb250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:16.868 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:16.885+0000 7f80c2877640 1 --2- 172.21.15.3:0/2626836924 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f80bc0cf660 0x7f80bc0ca3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:16.868 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:16.885+0000 7f80c2877640 1 -- 172.21.15.3:0/2626836924 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f80bc0d3c60 msgr2=0x7f80bc0cb250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:16.868 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:16.885+0000 7f80c2877640 1 --2- 172.21.15.3:0/2626836924 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f80bc0d3c60 0x7f80bc0cb250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:16.868 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:16.885+0000 7f80c2877640 1 -- 172.21.15.3:0/2626836924 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80bc0cb790 con 0x7f80bc15e470 2024-05-09T06:36:16.868 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:16.885+0000 7f80c2877640 1 --2- 172.21.15.3:0/2626836924 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f80bc15e470 0x7f80bc160160 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f80ac00cbf0 tx=0x7f80ac00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:16.868 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:16.885+0000 7f80baffd640 1 -- 172.21.15.3:0/2626836924 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80ac017070 con 0x7f80bc15e470 2024-05-09T06:36:16.868 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:16.885+0000 7f80baffd640 1 -- 172.21.15.3:0/2626836924 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f80ac00ad40 con 0x7f80bc15e470 2024-05-09T06:36:16.868 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:16.885+0000 7f80baffd640 1 -- 172.21.15.3:0/2626836924 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80ac012600 con 0x7f80bc15e470 2024-05-09T06:36:16.868 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:16.885+0000 7f80c42da640 1 -- 172.21.15.3:0/2626836924 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80bc0cba30 con 0x7f80bc15e470 2024-05-09T06:36:16.868 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:16.885+0000 7f80c42da640 1 -- 172.21.15.3:0/2626836924 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80bc0cbff0 con 0x7f80bc15e470 2024-05-09T06:36:16.868 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:16.886+0000 7f80c42da640 1 -- 172.21.15.3:0/2626836924 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80bc069550 con 0x7f80bc15e470 2024-05-09T06:36:16.868 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:16.886+0000 7f80baffd640 1 -- 172.21.15.3:0/2626836924 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f80ac01a070 con 0x7f80bc15e470 2024-05-09T06:36:16.869 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:16.887+0000 7f80baffd640 1 --2- 172.21.15.3:0/2626836924 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f80a806be30 0x7f80a806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:16.869 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:16.887+0000 7f80baffd640 1 -- 172.21.15.3:0/2626836924 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f80ac08ab20 con 0x7f80bc15e470 2024-05-09T06:36:16.869 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:16.887+0000 7f80c2076640 1 --2- 172.21.15.3:0/2626836924 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f80a806be30 0x7f80a806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:16.869 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:16.888+0000 7f80c2076640 1 --2- 172.21.15.3:0/2626836924 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f80a806be30 0x7f80a806e2f0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f80b4034000 tx=0x7f80b402e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:16.872 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:16.891+0000 7f80baffd640 1 -- 172.21.15.3:0/2626836924 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f80ac058e80 con 0x7f80bc15e470 2024-05-09T06:36:17.093 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:17.112+0000 7f80c42da640 1 -- 172.21.15.3:0/2626836924 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f80bc0d0180 con 0x7f80bc15e470 2024-05-09T06:36:17.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:17.113+0000 7f80baffd640 1 -- 172.21.15.3:0/2626836924 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f80ac05d180 con 0x7f80bc15e470 2024-05-09T06:36:17.094 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-09T06:36:17.094 INFO:teuthology.orchestra.run.smithi003.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-05-09T06:36:17.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:17.115+0000 7f80c42da640 1 -- 172.21.15.3:0/2626836924 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f80a806be30 msgr2=0x7f80a806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:17.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:17.115+0000 7f80c42da640 1 --2- 172.21.15.3:0/2626836924 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f80a806be30 0x7f80a806e2f0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f80b4034000 tx=0x7f80b402e000 comp rx=0 tx=0).stop 2024-05-09T06:36:17.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:17.115+0000 7f80c42da640 1 -- 172.21.15.3:0/2626836924 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f80bc15e470 msgr2=0x7f80bc160160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:17.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:17.115+0000 7f80c42da640 1 --2- 172.21.15.3:0/2626836924 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f80bc15e470 0x7f80bc160160 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f80ac00cbf0 tx=0x7f80ac00a430 comp rx=0 tx=0).stop 2024-05-09T06:36:17.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:17.115+0000 7f80c42da640 1 -- 172.21.15.3:0/2626836924 shutdown_connections 2024-05-09T06:36:17.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:17.115+0000 7f80c42da640 1 --2- 172.21.15.3:0/2626836924 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f80a806be30 0x7f80a806e2f0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:17.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:17.115+0000 7f80c42da640 1 --2- 172.21.15.3:0/2626836924 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f80bc15e470 0x7f80bc160160 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:17.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:17.115+0000 7f80c42da640 1 --2- 172.21.15.3:0/2626836924 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f80bc0d3c60 0x7f80bc0cb250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:17.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:17.115+0000 7f80c42da640 1 --2- 172.21.15.3:0/2626836924 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f80bc0cf660 0x7f80bc0ca3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:17.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:17.115+0000 7f80c42da640 1 -- 172.21.15.3:0/2626836924 >> 172.21.15.3:0/2626836924 conn(0x7f80bc0675c0 msgr2=0x7f80bc0b5c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:17.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:17.115+0000 7f80c42da640 1 -- 172.21.15.3:0/2626836924 shutdown_connections 2024-05-09T06:36:17.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:17.115+0000 7f80c42da640 1 -- 172.21.15.3:0/2626836924 wait complete. 2024-05-09T06:36:17.106 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-05-09T06:36:17.107 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T06:36:17.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:17.230+0000 7fd416942640 1 Processor -- start 2024-05-09T06:36:17.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:17.230+0000 7fd416942640 1 -- start start 2024-05-09T06:36:17.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:17.231+0000 7fd416942640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fd41015bd90 0x7fd41006f530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:17.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:17.231+0000 7fd416942640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd41006fa70 0x7fd41015ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:17.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:17.231+0000 7fd416942640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd410070c10 con 0x7fd41015f130 2024-05-09T06:36:17.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:17.231+0000 7fd416942640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fd4100719d0 con 0x7fd41015bd90 2024-05-09T06:36:17.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:17.231+0000 7fd416942640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fd41006fed0 con 0x7fd41006fa70 2024-05-09T06:36:17.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:17.231+0000 7fd40ffff640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fd41015bd90 0x7fd41006f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:17.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:17.231+0000 7fd40ffff640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fd41015bd90 0x7fd41006f530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49842/0 (socket says 172.21.15.3:49842) 2024-05-09T06:36:17.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:17.231+0000 7fd40ffff640 1 -- 172.21.15.3:0/330202374 learned_addr learned my addr 172.21.15.3:0/330202374 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:36:17.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:17.231+0000 7fd40f7fe640 1 --2- 172.21.15.3:0/330202374 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd41006fa70 0x7fd41015ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:17.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:17.231+0000 7fd40ffff640 1 -- 172.21.15.3:0/330202374 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd41006fa70 msgr2=0x7fd41015ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:17.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:17.231+0000 7fd40ffff640 1 --2- 172.21.15.3:0/330202374 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd41006fa70 0x7fd41015ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:17.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:17.231+0000 7fd40ffff640 1 -- 172.21.15.3:0/330202374 >> v1:172.21.15.190:6789/0 conn(0x7fd41015f130 legacy=0x7fd410161fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:17.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:17.231+0000 7fd40ffff640 1 -- 172.21.15.3:0/330202374 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd410070ef0 con 0x7fd41015bd90 2024-05-09T06:36:17.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:17.232+0000 7fd40ffff640 1 --2- 172.21.15.3:0/330202374 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd41015bd90 0x7fd41006f530 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fd400009870 tx=0x7fd40002a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f9dd6c12886e395 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:17.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:17.232+0000 7fd40effd640 1 -- 172.21.15.3:0/330202374 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd400037070 con 0x7fd41015bd90 2024-05-09T06:36:17.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:17.232+0000 7fd40effd640 1 -- 172.21.15.3:0/330202374 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd400032440 con 0x7fd41015bd90 2024-05-09T06:36:17.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:17.232+0000 7fd40effd640 1 -- 172.21.15.3:0/330202374 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd400032810 con 0x7fd41015bd90 2024-05-09T06:36:17.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:17.232+0000 7fd416942640 1 -- 172.21.15.3:0/330202374 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd41015bd90 msgr2=0x7fd41006f530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:17.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:17.232+0000 7fd416942640 1 --2- 172.21.15.3:0/330202374 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd41015bd90 0x7fd41006f530 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fd400009870 tx=0x7fd40002a1c0 comp rx=0 tx=0).stop 2024-05-09T06:36:17.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:17.233+0000 7fd416942640 1 -- 172.21.15.3:0/330202374 shutdown_connections 2024-05-09T06:36:17.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:17.233+0000 7fd416942640 1 --2- 172.21.15.3:0/330202374 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd41006fa70 0x7fd41015ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:17.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:17.233+0000 7fd416942640 1 --2- 172.21.15.3:0/330202374 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd41015bd90 0x7fd41006f530 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:17.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:17.233+0000 7fd416942640 1 -- 172.21.15.3:0/330202374 >> 172.21.15.3:0/330202374 conn(0x7fd4100d1f70 msgr2=0x7fd4100d2350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:17.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:17.233+0000 7fd416942640 1 -- 172.21.15.3:0/330202374 shutdown_connections 2024-05-09T06:36:17.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:17.233+0000 7fd416942640 1 -- 172.21.15.3:0/330202374 wait complete. 2024-05-09T06:36:17.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:17.234+0000 7fd416942640 1 Processor -- start 2024-05-09T06:36:17.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:17.234+0000 7fd416942640 1 -- start start 2024-05-09T06:36:17.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:17.235+0000 7fd416942640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd41006fa70 0x7fd4101f4a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:17.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:17.235+0000 7fd416942640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd41015bd90 0x7fd41015ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:17.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:17.235+0000 7fd416942640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd41015f130 0x7fd4101f4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:17.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:17.235+0000 7fd416942640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fd41006d880 con 0x7fd41006fa70 2024-05-09T06:36:17.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:17.235+0000 7fd416942640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fd4100714b0 con 0x7fd41015bd90 2024-05-09T06:36:17.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:17.235+0000 7fd416942640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fd41006da00 con 0x7fd41015f130 2024-05-09T06:36:17.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:17.235+0000 7fd40ffff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd41006fa70 0x7fd4101f4a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:17.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:17.235+0000 7fd40ffff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd41006fa70 0x7fd4101f4a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49852/0 (socket says 172.21.15.3:49852) 2024-05-09T06:36:17.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:17.235+0000 7fd40ffff640 1 -- 172.21.15.3:0/4121200887 learned_addr learned my addr 172.21.15.3:0/4121200887 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:36:17.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:17.235+0000 7fd414edf640 1 --2- 172.21.15.3:0/4121200887 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd41015f130 0x7fd4101f4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:17.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:17.235+0000 7fd40f7fe640 1 --2- 172.21.15.3:0/4121200887 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd41015bd90 0x7fd41015ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:17.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:17.235+0000 7fd40ffff640 1 -- 172.21.15.3:0/4121200887 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd41015f130 msgr2=0x7fd4101f4ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:17.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:17.235+0000 7fd40ffff640 1 --2- 172.21.15.3:0/4121200887 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd41015f130 0x7fd4101f4ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:17.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:17.235+0000 7fd40ffff640 1 -- 172.21.15.3:0/4121200887 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd41015bd90 msgr2=0x7fd41015ae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:17.219 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:17.235+0000 7fd40ffff640 1 --2- 172.21.15.3:0/4121200887 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd41015bd90 0x7fd41015ae90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:17.219 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:17.235+0000 7fd40ffff640 1 -- 172.21.15.3:0/4121200887 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4101f5530 con 0x7fd41006fa70 2024-05-09T06:36:17.219 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:17.235+0000 7fd40ffff640 1 --2- 172.21.15.3:0/4121200887 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd41006fa70 0x7fd4101f4a20 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fd400000c00 tx=0x7fd4000046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:17.219 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:17.235+0000 7fd40cff9640 1 -- 172.21.15.3:0/4121200887 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd400041020 con 0x7fd41006fa70 2024-05-09T06:36:17.219 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:17.236+0000 7fd40cff9640 1 -- 172.21.15.3:0/4121200887 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd4000306c0 con 0x7fd41006fa70 2024-05-09T06:36:17.219 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:17.236+0000 7fd416942640 1 -- 172.21.15.3:0/4121200887 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4101f5800 con 0x7fd41006fa70 2024-05-09T06:36:17.219 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:17.236+0000 7fd40cff9640 1 -- 172.21.15.3:0/4121200887 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd400037040 con 0x7fd41006fa70 2024-05-09T06:36:17.219 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:17.236+0000 7fd416942640 1 -- 172.21.15.3:0/4121200887 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4101f5dc0 con 0x7fd41006fa70 2024-05-09T06:36:17.219 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:17.237+0000 7fd416942640 1 -- 172.21.15.3:0/4121200887 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd41004f660 con 0x7fd41006fa70 2024-05-09T06:36:17.220 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:17.237+0000 7fd40cff9640 1 -- 172.21.15.3:0/4121200887 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd400044020 con 0x7fd41006fa70 2024-05-09T06:36:17.220 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:17.238+0000 7fd40cff9640 1 --2- 172.21.15.3:0/4121200887 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fd3fc06be30 0x7fd3fc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:17.220 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:17.238+0000 7fd40cff9640 1 -- 172.21.15.3:0/4121200887 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fd4000ab4c0 con 0x7fd41006fa70 2024-05-09T06:36:17.220 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:17.238+0000 7fd40f7fe640 1 --2- 172.21.15.3:0/4121200887 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fd3fc06be30 0x7fd3fc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:17.220 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:17.238+0000 7fd40f7fe640 1 --2- 172.21.15.3:0/4121200887 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fd3fc06be30 0x7fd3fc06e2f0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fd4101474f0 tx=0x7fd404008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:17.222 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:17.242+0000 7fd40cff9640 1 -- 172.21.15.3:0/4121200887 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fd40004b030 con 0x7fd41006fa70 2024-05-09T06:36:17.408 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:17.427+0000 7fd416942640 1 -- 172.21.15.3:0/4121200887 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd4100b1f30 con 0x7fd41006fa70 2024-05-09T06:36:17.409 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:17.428+0000 7fd40cff9640 1 -- 172.21.15.3:0/4121200887 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16521 (secure 0 0 0) 0x7fd4000800d0 con 0x7fd41006fa70 2024-05-09T06:36:17.409 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-09T06:36:17.409 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":20,"fsid":"16ed737d-2665-431a-8ac6-ef242be3975a","created":"2024-05-09T06:35:42.541368+0000","modified":"2024-05-09T06:36:08.856391+0000","last_up_change":"2024-05-09T06:35:53.778274+0000","last_in_change":"2024-05-09T06:35:44.780586+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T06:35:54.819999+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T06:36:04.931314+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T06:36:06.200433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fae3a8db-4826-4f3c-b33a-12c09bb5df6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":1897849827},{"type":"v1","addr":"172.21.15.3:6817","nonce":1897849827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":1897849827},{"type":"v1","addr":"172.21.15.3:6819","nonce":1897849827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":1897849827},{"type":"v1","addr":"172.21.15.3:6823","nonce":1897849827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":1897849827},{"type":"v1","addr":"172.21.15.3:6821","nonce":1897849827}]},"public_addr":"172.21.15.3:6817/1897849827","cluster_addr":"172.21.15.3:6819/1897849827","heartbeat_back_addr":"172.21.15.3:6823/1897849827","heartbeat_front_addr":"172.21.15.3:6821/1897849827","state":["exists","up"]},{"osd":1,"uuid":"646e882d-4ce4-4720-8c84-f71df563a35e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":926805498},{"type":"v1","addr":"172.21.15.3:6801","nonce":926805498}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":926805498},{"type":"v1","addr":"172.21.15.3:6803","nonce":926805498}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":926805498},{"type":"v1","addr":"172.21.15.3:6807","nonce":926805498}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":926805498},{"type":"v1","addr":"172.21.15.3:6805","nonce":926805498}]},"public_addr":"172.21.15.3:6801/926805498","cluster_addr":"172.21.15.3:6803/926805498","heartbeat_back_addr":"172.21.15.3:6807/926805498","heartbeat_front_addr":"172.21.15.3:6805/926805498","state":["exists","up"]},{"osd":2,"uuid":"1abbadb5-a570-4359-9643-acff3ede35a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":801295451},{"type":"v1","addr":"172.21.15.3:6809","nonce":801295451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":801295451},{"type":"v1","addr":"172.21.15.3:6811","nonce":801295451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":801295451},{"type":"v1","addr":"172.21.15.3:6815","nonce":801295451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":801295451},{"type":"v1","addr":"172.21.15.3:6813","nonce":801295451}]},"public_addr":"172.21.15.3:6809/801295451","cluster_addr":"172.21.15.3:6811/801295451","heartbeat_back_addr":"172.21.15.3:6815/801295451","heartbeat_front_addr":"172.21.15.3:6813/801295451","state":["exists","up"]},{"osd":3,"uuid":"62ac4dc1-2371-4dd2-acd3-e2dbb11aedf5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":1903389857},{"type":"v1","addr":"172.21.15.3:6825","nonce":1903389857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":1903389857},{"type":"v1","addr":"172.21.15.3:6827","nonce":1903389857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":1903389857},{"type":"v1","addr":"172.21.15.3:6831","nonce":1903389857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":1903389857},{"type":"v1","addr":"172.21.15.3:6829","nonce":1903389857}]},"public_addr":"172.21.15.3:6825/1903389857","cluster_addr":"172.21.15.3:6827/1903389857","heartbeat_back_addr":"172.21.15.3:6831/1903389857","heartbeat_front_addr":"172.21.15.3:6829/1903389857","state":["exists","up"]},{"osd":4,"uuid":"ae808398-8326-4d7c-9319-a2a0eafffb59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":2143624576},{"type":"v1","addr":"172.21.15.190:6801","nonce":2143624576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":2143624576},{"type":"v1","addr":"172.21.15.190:6803","nonce":2143624576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":2143624576},{"type":"v1","addr":"172.21.15.190:6807","nonce":2143624576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":2143624576},{"type":"v1","addr":"172.21.15.190:6805","nonce":2143624576}]},"public_addr":"172.21.15.190:6801/2143624576","cluster_addr":"172.21.15.190:6803/2143624576","heartbeat_back_addr":"172.21.15.190:6807/2143624576","heartbeat_front_addr":"172.21.15.190:6805/2143624576","state":["exists","up"]},{"osd":5,"uuid":"918adf50-c440-48ac-9a8d-4837562d7275","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":1766923126},{"type":"v1","addr":"172.21.15.190:6814","nonce":1766923126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":1766923126},{"type":"v1","addr":"172.21.15.190:6818","nonce":1766923126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":1766923126},{"type":"v1","addr":"172.21.15.190:6823","nonce":1766923126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":1766923126},{"type":"v1","addr":"172.21.15.190:6821","nonce":1766923126}]},"public_addr":"172.21.15.190:6814/1766923126","cluster_addr":"172.21.15.190:6818/1766923126","heartbeat_back_addr":"172.21.15.190:6823/1766923126","heartbeat_front_addr":"172.21.15.190:6821/1766923126","state":["exists","up"]},{"osd":6,"uuid":"ac2882bc-2a7c-463c-8f6c-76ff49828adc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":342414248},{"type":"v1","addr":"172.21.15.190:6825","nonce":342414248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":342414248},{"type":"v1","addr":"172.21.15.190:6827","nonce":342414248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":342414248},{"type":"v1","addr":"172.21.15.190:6831","nonce":342414248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":342414248},{"type":"v1","addr":"172.21.15.190:6829","nonce":342414248}]},"public_addr":"172.21.15.190:6825/342414248","cluster_addr":"172.21.15.190:6827/342414248","heartbeat_back_addr":"172.21.15.190:6831/342414248","heartbeat_front_addr":"172.21.15.190:6829/342414248","state":["exists","up"]},{"osd":7,"uuid":"dd4e767a-4730-4c70-8bd5-3f088140a5a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":537849891},{"type":"v1","addr":"172.21.15.190:6809","nonce":537849891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":537849891},{"type":"v1","addr":"172.21.15.190:6811","nonce":537849891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6817","nonce":537849891},{"type":"v1","addr":"172.21.15.190:6819","nonce":537849891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6813","nonce":537849891},{"type":"v1","addr":"172.21.15.190:6815","nonce":537849891}]},"public_addr":"172.21.15.190:6809/537849891","cluster_addr":"172.21.15.190:6811/537849891","heartbeat_back_addr":"172.21.15.190:6819/537849891","heartbeat_front_addr":"172.21.15.190:6815/537849891","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.889784+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.974002+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.973549+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.955396+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.928471+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.902418+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.958696+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.926474+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T06:36:17.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:17.430+0000 7fd416942640 1 -- 172.21.15.3:0/4121200887 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fd3fc06be30 msgr2=0x7fd3fc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:17.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:17.430+0000 7fd416942640 1 --2- 172.21.15.3:0/4121200887 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fd3fc06be30 0x7fd3fc06e2f0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fd4101474f0 tx=0x7fd404008040 comp rx=0 tx=0).stop 2024-05-09T06:36:17.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:17.430+0000 7fd416942640 1 -- 172.21.15.3:0/4121200887 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd41006fa70 msgr2=0x7fd4101f4a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:17.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:17.430+0000 7fd416942640 1 --2- 172.21.15.3:0/4121200887 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd41006fa70 0x7fd4101f4a20 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fd400000c00 tx=0x7fd4000046c0 comp rx=0 tx=0).stop 2024-05-09T06:36:17.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:17.430+0000 7fd416942640 1 -- 172.21.15.3:0/4121200887 shutdown_connections 2024-05-09T06:36:17.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:17.430+0000 7fd416942640 1 --2- 172.21.15.3:0/4121200887 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fd3fc06be30 0x7fd3fc06e2f0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:17.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:17.430+0000 7fd416942640 1 --2- 172.21.15.3:0/4121200887 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd41015f130 0x7fd4101f4ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:17.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:17.430+0000 7fd416942640 1 --2- 172.21.15.3:0/4121200887 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd41015bd90 0x7fd41015ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:17.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:17.430+0000 7fd416942640 1 --2- 172.21.15.3:0/4121200887 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd41006fa70 0x7fd4101f4a20 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:17.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:17.430+0000 7fd416942640 1 -- 172.21.15.3:0/4121200887 >> 172.21.15.3:0/4121200887 conn(0x7fd4100d1f70 msgr2=0x7fd41015ea20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:17.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:17.430+0000 7fd416942640 1 -- 172.21.15.3:0/4121200887 shutdown_connections 2024-05-09T06:36:17.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:17.430+0000 7fd416942640 1 -- 172.21.15.3:0/4121200887 wait complete. 2024-05-09T06:36:17.422 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-09T06:35:54.819999+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-09T06:36:04.931314+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-09T06:36:06.200433+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-09T06:36:17.423 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-09T06:36:17.530 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:17.549+0000 7f00f1e53640 1 Processor -- start 2024-05-09T06:36:17.530 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:17.549+0000 7f00f1e53640 1 -- start start 2024-05-09T06:36:17.531 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:17.550+0000 7f00f1e53640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f00ec06c260 0x7f00ec169210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:17.531 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:17.550+0000 7f00f1e53640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f00ec162ca0 con 0x7f00ec15e0b0 2024-05-09T06:36:17.531 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:17.550+0000 7f00f1e53640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f00ec06cad0 con 0x7f00ec15e490 2024-05-09T06:36:17.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:17.550+0000 7f00f1e53640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f00ec0b3eb0 con 0x7f00ec06c260 2024-05-09T06:36:17.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:17.550+0000 7f00eb7fe640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f00ec15e0b0 0x7f00ec161e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:49300/0 (socket says 172.21.15.3:49300) 2024-05-09T06:36:17.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:17.550+0000 7f00eb7fe640 1 -- 172.21.15.3:0/2989689355 learned_addr learned my addr 172.21.15.3:0/2989689355 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-05-09T06:36:17.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:17.550+0000 7f00ebfff640 1 --2- 172.21.15.3:0/2989689355 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f00ec06c260 0x7f00ec169210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:17.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:17.550+0000 7f00ea7fc640 1 -- 172.21.15.3:0/2989689355 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1673957974 0 0) 0x7f00ec162ca0 con 0x7f00ec15e0b0 2024-05-09T06:36:17.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:17.550+0000 7f00ea7fc640 1 -- 172.21.15.3:0/2989689355 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f00d4003520 con 0x7f00ec15e0b0 2024-05-09T06:36:17.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:17.551+0000 7f00ea7fc640 1 -- 172.21.15.3:0/2989689355 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 961417078 0 0) 0x7f00ec06cad0 con 0x7f00ec15e490 2024-05-09T06:36:17.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:17.551+0000 7f00ebfff640 1 -- 172.21.15.3:0/2989689355 >> v1:172.21.15.190:6789/0 conn(0x7f00ec15e490 legacy=0x7f00ec162590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:17.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:17.551+0000 7f00ebfff640 1 -- 172.21.15.3:0/2989689355 >> v1:172.21.15.3:6789/0 conn(0x7f00ec15e0b0 legacy=0x7f00ec161e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:17.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:17.551+0000 7f00ebfff640 1 -- 172.21.15.3:0/2989689355 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00ec0b4190 con 0x7f00ec06c260 2024-05-09T06:36:17.533 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:17.551+0000 7f00ebfff640 1 --2- 172.21.15.3:0/2989689355 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f00ec06c260 0x7f00ec169210 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f00e4009dc0 tx=0x7f00e402a640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=38e0dd5b82468c97 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:17.534 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:17.552+0000 7f00ea7fc640 1 -- 172.21.15.3:0/2989689355 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00e4035020 con 0x7f00ec06c260 2024-05-09T06:36:17.534 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:17.552+0000 7f00ea7fc640 1 -- 172.21.15.3:0/2989689355 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00e4007540 con 0x7f00ec06c260 2024-05-09T06:36:17.534 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:17.552+0000 7f00ea7fc640 1 -- 172.21.15.3:0/2989689355 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00dc001670 con 0x7f00ec06c260 2024-05-09T06:36:17.534 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:17.552+0000 7f00f1e53640 1 -- 172.21.15.3:0/2989689355 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f00ec06c260 msgr2=0x7f00ec169210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:17.534 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:17.552+0000 7f00f1e53640 1 --2- 172.21.15.3:0/2989689355 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f00ec06c260 0x7f00ec169210 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f00e4009dc0 tx=0x7f00e402a640 comp rx=0 tx=0).stop 2024-05-09T06:36:17.534 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:17.552+0000 7f00f1e53640 1 -- 172.21.15.3:0/2989689355 shutdown_connections 2024-05-09T06:36:17.534 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:17.552+0000 7f00f1e53640 1 --2- 172.21.15.3:0/2989689355 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f00ec06c260 0x7f00ec169210 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:17.534 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:17.552+0000 7f00f1e53640 1 -- 172.21.15.3:0/2989689355 >> 172.21.15.3:0/2989689355 conn(0x7f00ec0d07b0 msgr2=0x7f00ec0d0bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:17.534 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:17.552+0000 7f00f1e53640 1 -- 172.21.15.3:0/2989689355 shutdown_connections 2024-05-09T06:36:17.534 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:17.552+0000 7f00f1e53640 1 -- 172.21.15.3:0/2989689355 wait complete. 2024-05-09T06:36:17.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:17.553+0000 7f00f1e53640 1 Processor -- start 2024-05-09T06:36:17.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:17.554+0000 7f00f1e53640 1 -- start start 2024-05-09T06:36:17.537 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:17.554+0000 7f00f1e53640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f00ec06c260 0x7f00ec15f920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:17.537 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:17.554+0000 7f00f1e53640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f00ec15e0b0 0x7f00ec1450d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:17.537 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:17.554+0000 7f00f1e53640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f00ec15e490 0x7f00ec159170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:17.537 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:17.554+0000 7f00f1e53640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f00ec06c6d0 con 0x7f00ec15e0b0 2024-05-09T06:36:17.537 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:17.554+0000 7f00f1e53640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f00ec0b4750 con 0x7f00ec15e490 2024-05-09T06:36:17.537 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:17.554+0000 7f00f1e53640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f00ec06c850 con 0x7f00ec06c260 2024-05-09T06:36:17.537 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:17.555+0000 7f00eaffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f00ec15e0b0 0x7f00ec1450d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:17.537 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:17.555+0000 7f00eaffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f00ec15e0b0 0x7f00ec1450d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49854/0 (socket says 172.21.15.3:49854) 2024-05-09T06:36:17.537 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:17.555+0000 7f00eaffd640 1 -- 172.21.15.3:0/2268274661 learned_addr learned my addr 172.21.15.3:0/2268274661 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:36:17.537 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:17.555+0000 7f00eb7fe640 1 --2- 172.21.15.3:0/2268274661 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f00ec06c260 0x7f00ec15f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:17.537 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:17.555+0000 7f00ebfff640 1 --2- 172.21.15.3:0/2268274661 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f00ec15e490 0x7f00ec159170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:17.537 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:17.555+0000 7f00eaffd640 1 -- 172.21.15.3:0/2268274661 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f00ec06c260 msgr2=0x7f00ec15f920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:17.537 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:17.555+0000 7f00eaffd640 1 --2- 172.21.15.3:0/2268274661 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f00ec06c260 0x7f00ec15f920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:17.537 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:17.555+0000 7f00eaffd640 1 -- 172.21.15.3:0/2268274661 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f00ec15e490 msgr2=0x7f00ec159170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:17.537 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:17.555+0000 7f00eaffd640 1 --2- 172.21.15.3:0/2268274661 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f00ec15e490 0x7f00ec159170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:17.538 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:17.555+0000 7f00eaffd640 1 -- 172.21.15.3:0/2268274661 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00ec15e8c0 con 0x7f00ec15e0b0 2024-05-09T06:36:17.538 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:17.555+0000 7f00eaffd640 1 --2- 172.21.15.3:0/2268274661 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f00ec15e0b0 0x7f00ec1450d0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f00e000c440 tx=0x7f00e000c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:17.538 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:17.555+0000 7f00d3fff640 1 -- 172.21.15.3:0/2268274661 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00e0017070 con 0x7f00ec15e0b0 2024-05-09T06:36:17.538 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:17.555+0000 7f00d3fff640 1 -- 172.21.15.3:0/2268274661 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00e0005480 con 0x7f00ec15e0b0 2024-05-09T06:36:17.538 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:17.555+0000 7f00f1e53640 1 -- 172.21.15.3:0/2268274661 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00ec15eb60 con 0x7f00ec15e0b0 2024-05-09T06:36:17.538 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:17.555+0000 7f00d3fff640 1 -- 172.21.15.3:0/2268274661 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00e0003830 con 0x7f00ec15e0b0 2024-05-09T06:36:17.538 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:17.555+0000 7f00f1e53640 1 -- 172.21.15.3:0/2268274661 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00ec15f120 con 0x7f00ec15e0b0 2024-05-09T06:36:17.539 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:17.556+0000 7f00f1e53640 1 -- 172.21.15.3:0/2268274661 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00ec069550 con 0x7f00ec15e0b0 2024-05-09T06:36:17.539 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:17.557+0000 7f00d3fff640 1 -- 172.21.15.3:0/2268274661 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f00e001a070 con 0x7f00ec15e0b0 2024-05-09T06:36:17.539 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:17.557+0000 7f00d3fff640 1 --2- 172.21.15.3:0/2268274661 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f00d806be30 0x7f00d806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:17.539 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:17.557+0000 7f00d3fff640 1 -- 172.21.15.3:0/2268274661 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f00e008a130 con 0x7f00ec15e0b0 2024-05-09T06:36:17.539 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:17.557+0000 7f00eb7fe640 1 --2- 172.21.15.3:0/2268274661 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f00d806be30 0x7f00d806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:17.539 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:17.558+0000 7f00eb7fe640 1 --2- 172.21.15.3:0/2268274661 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f00d806be30 0x7f00d806e2f0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f00dc001600 tx=0x7f00dc001570 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:17.542 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:17.562+0000 7f00d3fff640 1 -- 172.21.15.3:0/2268274661 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f00e0058400 con 0x7f00ec15e0b0 2024-05-09T06:36:17.725 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:17.744+0000 7f00f1e53640 1 -- 172.21.15.3:0/2268274661 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f00ec15a260 con 0x7f00ec15e0b0 2024-05-09T06:36:17.726 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:17.745+0000 7f00d3fff640 1 -- 172.21.15.3:0/2268274661 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7f00e005c700 con 0x7f00ec15e0b0 2024-05-09T06:36:17.726 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 1 2024-05-09T06:36:17.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:17.747+0000 7f00f1e53640 1 -- 172.21.15.3:0/2268274661 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f00d806be30 msgr2=0x7f00d806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:17.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:17.747+0000 7f00f1e53640 1 --2- 172.21.15.3:0/2268274661 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f00d806be30 0x7f00d806e2f0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f00dc001600 tx=0x7f00dc001570 comp rx=0 tx=0).stop 2024-05-09T06:36:17.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:17.747+0000 7f00f1e53640 1 -- 172.21.15.3:0/2268274661 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f00ec15e0b0 msgr2=0x7f00ec1450d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:17.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:17.747+0000 7f00f1e53640 1 --2- 172.21.15.3:0/2268274661 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f00ec15e0b0 0x7f00ec1450d0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f00e000c440 tx=0x7f00e000c900 comp rx=0 tx=0).stop 2024-05-09T06:36:17.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:17.747+0000 7f00f1e53640 1 -- 172.21.15.3:0/2268274661 shutdown_connections 2024-05-09T06:36:17.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:17.747+0000 7f00f1e53640 1 --2- 172.21.15.3:0/2268274661 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f00d806be30 0x7f00d806e2f0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:17.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:17.747+0000 7f00f1e53640 1 --2- 172.21.15.3:0/2268274661 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f00ec15e490 0x7f00ec159170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:17.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:17.747+0000 7f00f1e53640 1 --2- 172.21.15.3:0/2268274661 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f00ec15e0b0 0x7f00ec1450d0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:17.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:17.747+0000 7f00f1e53640 1 --2- 172.21.15.3:0/2268274661 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f00ec06c260 0x7f00ec15f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:17.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:17.747+0000 7f00f1e53640 1 -- 172.21.15.3:0/2268274661 >> 172.21.15.3:0/2268274661 conn(0x7f00ec0d07b0 msgr2=0x7f00ec15a4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:17.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:17.747+0000 7f00f1e53640 1 -- 172.21.15.3:0/2268274661 shutdown_connections 2024-05-09T06:36:17.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:17.747+0000 7f00f1e53640 1 -- 172.21.15.3:0/2268274661 wait complete. 2024-05-09T06:36:17.739 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-09T06:36:17.845 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:17.865+0000 7f01ecfed640 1 Processor -- start 2024-05-09T06:36:17.846 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:17.865+0000 7f01ecfed640 1 -- start start 2024-05-09T06:36:17.848 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:17.866+0000 7f01ecfed640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f01e806cd70 0x7f01e8162a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:17.848 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:17.866+0000 7f01ecfed640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f01e80d02b0 0x7f01e806d150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:17.848 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:17.866+0000 7f01ecfed640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f01e8162fa0 con 0x7f01e809aa30 2024-05-09T06:36:17.848 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:17.866+0000 7f01ecfed640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f01e8161e40 con 0x7f01e806cd70 2024-05-09T06:36:17.848 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:17.866+0000 7f01ecfed640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f01e806d690 con 0x7f01e80d02b0 2024-05-09T06:36:17.848 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:17.866+0000 7f01e659c640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f01e806cd70 0x7f01e8162a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:17.848 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:17.866+0000 7f01e659c640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f01e806cd70 0x7f01e8162a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49870/0 (socket says 172.21.15.3:49870) 2024-05-09T06:36:17.848 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:17.866+0000 7f01e659c640 1 -- 172.21.15.3:0/2459392023 learned_addr learned my addr 172.21.15.3:0/2459392023 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:36:17.848 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:17.866+0000 7f01ddd9b640 1 --2- 172.21.15.3:0/2459392023 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f01e80d02b0 0x7f01e806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:17.848 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:17.866+0000 7f01e659c640 1 -- 172.21.15.3:0/2459392023 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f01e80d02b0 msgr2=0x7f01e806d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:17.848 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:17.866+0000 7f01e659c640 1 --2- 172.21.15.3:0/2459392023 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f01e80d02b0 0x7f01e806d150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:17.848 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:17.866+0000 7f01e659c640 1 -- 172.21.15.3:0/2459392023 >> v1:172.21.15.190:6789/0 conn(0x7f01e809aa30 legacy=0x7f01e809aec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:17.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:17.866+0000 7f01e659c640 1 -- 172.21.15.3:0/2459392023 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01e80d0870 con 0x7f01e806cd70 2024-05-09T06:36:17.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:17.866+0000 7f01ddd9b640 1 --2- 172.21.15.3:0/2459392023 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f01e80d02b0 0x7f01e806d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:36:17.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:17.867+0000 7f01e659c640 1 --2- 172.21.15.3:0/2459392023 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f01e806cd70 0x7f01e8162a60 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f01d8009870 tx=0x7f01d802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1df49ca454ddd56e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:17.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:17.867+0000 7f01e5d9b640 1 -- 172.21.15.3:0/2459392023 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01d8037070 con 0x7f01e806cd70 2024-05-09T06:36:17.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:17.867+0000 7f01e5d9b640 1 -- 172.21.15.3:0/2459392023 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01d8032440 con 0x7f01e806cd70 2024-05-09T06:36:17.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:17.867+0000 7f01e5d9b640 1 -- 172.21.15.3:0/2459392023 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01d8032810 con 0x7f01e806cd70 2024-05-09T06:36:17.852 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:17.867+0000 7f01ecfed640 1 -- 172.21.15.3:0/2459392023 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f01e806cd70 msgr2=0x7f01e8162a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:17.852 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:17.867+0000 7f01ecfed640 1 --2- 172.21.15.3:0/2459392023 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f01e806cd70 0x7f01e8162a60 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f01d8009870 tx=0x7f01d802a270 comp rx=0 tx=0).stop 2024-05-09T06:36:17.852 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:17.867+0000 7f01ecfed640 1 -- 172.21.15.3:0/2459392023 shutdown_connections 2024-05-09T06:36:17.852 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:17.867+0000 7f01ecfed640 1 --2- 172.21.15.3:0/2459392023 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f01e80d02b0 0x7f01e806d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:17.852 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:17.867+0000 7f01ecfed640 1 --2- 172.21.15.3:0/2459392023 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f01e806cd70 0x7f01e8162a60 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:17.852 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:17.867+0000 7f01ecfed640 1 -- 172.21.15.3:0/2459392023 >> 172.21.15.3:0/2459392023 conn(0x7f01e80b3af0 msgr2=0x7f01e80b3f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:17.853 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:17.868+0000 7f01ecfed640 1 -- 172.21.15.3:0/2459392023 shutdown_connections 2024-05-09T06:36:17.853 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:17.868+0000 7f01ecfed640 1 -- 172.21.15.3:0/2459392023 wait complete. 2024-05-09T06:36:17.853 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:17.869+0000 7f01ecfed640 1 Processor -- start 2024-05-09T06:36:17.853 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:17.869+0000 7f01ecfed640 1 -- start start 2024-05-09T06:36:17.853 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:17.870+0000 7f01ecfed640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f01e806cd70 0x7f01e81f8780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:17.853 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:17.870+0000 7f01ecfed640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f01e809aa30 0x7f01e80c74d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:17.853 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:17.870+0000 7f01ecfed640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f01e80d02b0 0x7f01e806f280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:17.853 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:17.870+0000 7f01ecfed640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f01e80d1110 con 0x7f01e806cd70 2024-05-09T06:36:17.853 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:17.870+0000 7f01ecfed640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f01e80d0e30 con 0x7f01e809aa30 2024-05-09T06:36:17.853 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:17.870+0000 7f01ecfed640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f01e80d13f0 con 0x7f01e80d02b0 2024-05-09T06:36:17.853 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:17.870+0000 7f01e659c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f01e806cd70 0x7f01e81f8780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:17.853 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:17.870+0000 7f01e659c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f01e806cd70 0x7f01e81f8780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49880/0 (socket says 172.21.15.3:49880) 2024-05-09T06:36:17.853 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:17.870+0000 7f01e659c640 1 -- 172.21.15.3:0/1355386508 learned_addr learned my addr 172.21.15.3:0/1355386508 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:36:17.853 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:17.870+0000 7f01ddd9b640 1 --2- 172.21.15.3:0/1355386508 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f01e809aa30 0x7f01e80c74d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:17.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:17.870+0000 7f01e6d9d640 1 --2- 172.21.15.3:0/1355386508 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f01e80d02b0 0x7f01e806f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:17.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:17.870+0000 7f01e659c640 1 -- 172.21.15.3:0/1355386508 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f01e80d02b0 msgr2=0x7f01e806f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:17.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:17.870+0000 7f01e659c640 1 --2- 172.21.15.3:0/1355386508 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f01e80d02b0 0x7f01e806f280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:17.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:17.870+0000 7f01e659c640 1 -- 172.21.15.3:0/1355386508 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f01e809aa30 msgr2=0x7f01e80c74d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:17.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:17.870+0000 7f01e659c640 1 --2- 172.21.15.3:0/1355386508 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f01e809aa30 0x7f01e80c74d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:17.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:17.870+0000 7f01e659c640 1 -- 172.21.15.3:0/1355386508 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01e80b6440 con 0x7f01e806cd70 2024-05-09T06:36:17.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:17.870+0000 7f01ddd9b640 1 --2- 172.21.15.3:0/1355386508 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f01e809aa30 0x7f01e80c74d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:36:17.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:17.870+0000 7f01e6d9d640 1 --2- 172.21.15.3:0/1355386508 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f01e80d02b0 0x7f01e806f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:36:17.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:17.870+0000 7f01e659c640 1 --2- 172.21.15.3:0/1355386508 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f01e806cd70 0x7f01e81f8780 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f01d8000c00 tx=0x7f01d80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:17.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:17.871+0000 7f01df7fe640 1 -- 172.21.15.3:0/1355386508 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01d8041020 con 0x7f01e806cd70 2024-05-09T06:36:17.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:17.871+0000 7f01df7fe640 1 -- 172.21.15.3:0/1355386508 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01d80306c0 con 0x7f01e806cd70 2024-05-09T06:36:17.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:17.871+0000 7f01df7fe640 1 -- 172.21.15.3:0/1355386508 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01d8037040 con 0x7f01e806cd70 2024-05-09T06:36:17.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:17.871+0000 7f01ecfed640 1 -- 172.21.15.3:0/1355386508 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01e80c8360 con 0x7f01e806cd70 2024-05-09T06:36:17.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:17.871+0000 7f01ecfed640 1 -- 172.21.15.3:0/1355386508 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01e80b66e0 con 0x7f01e806cd70 2024-05-09T06:36:17.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:17.872+0000 7f01ecfed640 1 -- 172.21.15.3:0/1355386508 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01e8069550 con 0x7f01e806cd70 2024-05-09T06:36:17.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:17.872+0000 7f01df7fe640 1 -- 172.21.15.3:0/1355386508 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f01d8044020 con 0x7f01e806cd70 2024-05-09T06:36:17.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:17.873+0000 7f01df7fe640 1 --2- 172.21.15.3:0/1355386508 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f01c806be30 0x7f01c806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:17.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:17.873+0000 7f01df7fe640 1 -- 172.21.15.3:0/1355386508 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f01d80ab610 con 0x7f01e806cd70 2024-05-09T06:36:17.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:17.873+0000 7f01ddd9b640 1 --2- 172.21.15.3:0/1355386508 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f01c806be30 0x7f01c806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:17.856 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:17.873+0000 7f01ddd9b640 1 --2- 172.21.15.3:0/1355386508 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f01c806be30 0x7f01c806e2f0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f01d0006fd0 tx=0x7f01d0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:17.858 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:17.877+0000 7f01df7fe640 1 -- 172.21.15.3:0/1355386508 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f01d804b030 con 0x7f01e806cd70 2024-05-09T06:36:18.042 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.062+0000 7f01ecfed640 1 -- 172.21.15.3:0/1355386508 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f01e80ce1a0 con 0x7f01e806cd70 2024-05-09T06:36:18.043 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.062+0000 7f01df7fe640 1 -- 172.21.15.3:0/1355386508 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7f01d807a050 con 0x7f01e806cd70 2024-05-09T06:36:18.043 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 32 2024-05-09T06:36:18.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.064+0000 7f01ecfed640 1 -- 172.21.15.3:0/1355386508 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f01c806be30 msgr2=0x7f01c806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:18.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.064+0000 7f01ecfed640 1 --2- 172.21.15.3:0/1355386508 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f01c806be30 0x7f01c806e2f0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f01d0006fd0 tx=0x7f01d0008040 comp rx=0 tx=0).stop 2024-05-09T06:36:18.046 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.064+0000 7f01ecfed640 1 -- 172.21.15.3:0/1355386508 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f01e806cd70 msgr2=0x7f01e81f8780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:18.046 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.064+0000 7f01ecfed640 1 --2- 172.21.15.3:0/1355386508 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f01e806cd70 0x7f01e81f8780 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f01d8000c00 tx=0x7f01d80046c0 comp rx=0 tx=0).stop 2024-05-09T06:36:18.046 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.064+0000 7f01ecfed640 1 -- 172.21.15.3:0/1355386508 shutdown_connections 2024-05-09T06:36:18.046 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.064+0000 7f01ecfed640 1 --2- 172.21.15.3:0/1355386508 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f01c806be30 0x7f01c806e2f0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:18.046 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.064+0000 7f01ecfed640 1 --2- 172.21.15.3:0/1355386508 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f01e80d02b0 0x7f01e806f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:18.046 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.064+0000 7f01ecfed640 1 --2- 172.21.15.3:0/1355386508 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f01e809aa30 0x7f01e80c74d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:18.046 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.064+0000 7f01ecfed640 1 --2- 172.21.15.3:0/1355386508 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f01e806cd70 0x7f01e81f8780 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:18.046 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.064+0000 7f01ecfed640 1 -- 172.21.15.3:0/1355386508 >> 172.21.15.3:0/1355386508 conn(0x7f01e80b3af0 msgr2=0x7f01e80708d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:18.046 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.065+0000 7f01ecfed640 1 -- 172.21.15.3:0/1355386508 shutdown_connections 2024-05-09T06:36:18.046 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.065+0000 7f01ecfed640 1 -- 172.21.15.3:0/1355386508 wait complete. 2024-05-09T06:36:18.056 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-09T06:36:18.163 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.182+0000 7f864f8f5640 1 Processor -- start 2024-05-09T06:36:18.163 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.183+0000 7f864f8f5640 1 -- start start 2024-05-09T06:36:18.167 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.183+0000 7f864f8f5640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f86480d1300 0x7f86480d2800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:18.167 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.183+0000 7f864f8f5640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f864815e9a0 con 0x7f864815e5c0 2024-05-09T06:36:18.167 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.183+0000 7f864f8f5640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f86480d31c0 con 0x7f86480d4c40 2024-05-09T06:36:18.167 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.183+0000 7f864f8f5640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f86480d2ee0 con 0x7f86480d1300 2024-05-09T06:36:18.167 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.183+0000 7f864d691640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f864815e5c0 0x7f86480d0bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:49310/0 (socket says 172.21.15.3:49310) 2024-05-09T06:36:18.167 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.183+0000 7f864d691640 1 -- 172.21.15.3:0/1160954409 learned_addr learned my addr 172.21.15.3:0/1160954409 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-05-09T06:36:18.167 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.184+0000 7f864ce90640 1 --2- 172.21.15.3:0/1160954409 >> v2:172.21.15.190:3300/0 conn(0x7f86480d1300 0x7f86480d2800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:18.167 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.184+0000 7f8647fff640 1 -- 172.21.15.3:0/1160954409 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 955824774 0 0) 0x7f864815e9a0 con 0x7f864815e5c0 2024-05-09T06:36:18.167 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.184+0000 7f8647fff640 1 -- 172.21.15.3:0/1160954409 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8630003520 con 0x7f864815e5c0 2024-05-09T06:36:18.167 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.184+0000 7f8647fff640 1 -- 172.21.15.3:0/1160954409 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1992832124 0 0) 0x7f8630003520 con 0x7f864815e5c0 2024-05-09T06:36:18.167 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.184+0000 7f8647fff640 1 -- 172.21.15.3:0/1160954409 >> v2:172.21.15.190:3300/0 conn(0x7f86480d1300 msgr2=0x7f86480d2800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:18.167 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.184+0000 7f8647fff640 1 --2- 172.21.15.3:0/1160954409 >> v2:172.21.15.190:3300/0 conn(0x7f86480d1300 0x7f86480d2800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:18.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.184+0000 7f8647fff640 1 -- 172.21.15.3:0/1160954409 >> v1:172.21.15.190:6789/0 conn(0x7f86480d4c40 legacy=0x7f86480b47e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:18.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.184+0000 7f8647fff640 1 -- 172.21.15.3:0/1160954409 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86480b3d00 con 0x7f864815e5c0 2024-05-09T06:36:18.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.184+0000 7f8647fff640 1 -- 172.21.15.3:0/1160954409 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 563312031 0 0) 0x7f8640001670 con 0x7f864815e5c0 2024-05-09T06:36:18.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.184+0000 7f8647fff640 1 -- 172.21.15.3:0/1160954409 >> v1:172.21.15.3:6789/0 conn(0x7f864815e5c0 legacy=0x7f86480d0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:18.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.184+0000 7f8647fff640 1 --2- 172.21.15.3:0/1160954409 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f86300039b0 0x7f8630003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:18.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.184+0000 7f8647fff640 1 --2- 172.21.15.3:0/1160954409 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8630004800 0x7f8630004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:18.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.184+0000 7f8647fff640 1 --2- 172.21.15.3:0/1160954409 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f86300051c0 0x7f863000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:18.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.184+0000 7f8647fff640 1 -- 172.21.15.3:0/1160954409 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f86480d2ee0 con 0x7f86300039b0 2024-05-09T06:36:18.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.184+0000 7f8647fff640 1 -- 172.21.15.3:0/1160954409 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f863000c060 con 0x7f86300051c0 2024-05-09T06:36:18.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.184+0000 7f8647fff640 1 -- 172.21.15.3:0/1160954409 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f863000c260 con 0x7f8630004800 2024-05-09T06:36:18.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.184+0000 7f864d691640 1 --2- 172.21.15.3:0/1160954409 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f86300039b0 0x7f8630003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:18.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.184+0000 7f864ce90640 1 --2- 172.21.15.3:0/1160954409 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8630004800 0x7f8630004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:18.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.184+0000 7f864de92640 1 --2- 172.21.15.3:0/1160954409 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f86300051c0 0x7f863000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:18.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.185+0000 7f864d691640 1 -- 172.21.15.3:0/1160954409 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8630004800 msgr2=0x7f8630004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:18.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.185+0000 7f864d691640 1 --2- 172.21.15.3:0/1160954409 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8630004800 0x7f8630004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:18.169 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.185+0000 7f864d691640 1 -- 172.21.15.3:0/1160954409 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f86300051c0 msgr2=0x7f863000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:18.169 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.185+0000 7f864d691640 1 --2- 172.21.15.3:0/1160954409 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f86300051c0 0x7f863000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:18.169 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.185+0000 7f864d691640 1 -- 172.21.15.3:0/1160954409 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8630003520 con 0x7f86300039b0 2024-05-09T06:36:18.169 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.185+0000 7f864ce90640 1 --2- 172.21.15.3:0/1160954409 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8630004800 0x7f8630004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:36:18.169 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.185+0000 7f864de92640 1 --2- 172.21.15.3:0/1160954409 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f86300051c0 0x7f863000b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:36:18.169 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.185+0000 7f864de92640 1 -- 172.21.15.3:0/1160954409 reap_dead start 2024-05-09T06:36:18.169 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.185+0000 7f864d691640 1 --2- 172.21.15.3:0/1160954409 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f86300039b0 0x7f8630003dd0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f864000c210 tx=0x7f8630005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=601d4df4475e0f5d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:18.169 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.185+0000 7f8647fff640 1 -- 172.21.15.3:0/1160954409 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8640007d30 con 0x7f86300039b0 2024-05-09T06:36:18.169 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.185+0000 7f8647fff640 1 -- 172.21.15.3:0/1160954409 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8640005ad0 con 0x7f86300039b0 2024-05-09T06:36:18.169 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.185+0000 7f8647fff640 1 -- 172.21.15.3:0/1160954409 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8640007070 con 0x7f86300039b0 2024-05-09T06:36:18.169 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.186+0000 7f864f8f5640 1 -- 172.21.15.3:0/1160954409 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f86300039b0 msgr2=0x7f8630003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:18.169 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.186+0000 7f864f8f5640 1 --2- 172.21.15.3:0/1160954409 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f86300039b0 0x7f8630003dd0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f864000c210 tx=0x7f8630005830 comp rx=0 tx=0).stop 2024-05-09T06:36:18.169 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.186+0000 7f864f8f5640 1 -- 172.21.15.3:0/1160954409 shutdown_connections 2024-05-09T06:36:18.169 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.186+0000 7f864f8f5640 1 --2- 172.21.15.3:0/1160954409 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f86300039b0 0x7f8630003dd0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:18.169 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.186+0000 7f864f8f5640 1 -- 172.21.15.3:0/1160954409 >> 172.21.15.3:0/1160954409 conn(0x7f864809d500 msgr2=0x7f86480d4220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:18.170 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.186+0000 7f864f8f5640 1 -- 172.21.15.3:0/1160954409 shutdown_connections 2024-05-09T06:36:18.170 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.186+0000 7f864f8f5640 1 -- 172.21.15.3:0/1160954409 wait complete. 2024-05-09T06:36:18.172 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.187+0000 7f864f8f5640 1 Processor -- start 2024-05-09T06:36:18.172 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.187+0000 7f864f8f5640 1 -- start start 2024-05-09T06:36:18.172 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.188+0000 7f864f8f5640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f86300039b0 0x7f86480cd920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:18.172 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.188+0000 7f864f8f5640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f86480cc1e0 0x7f8648151540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:18.173 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.188+0000 7f864f8f5640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8648159740 0x7f8648151a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:18.173 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.188+0000 7f864f8f5640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f86480d5090 con 0x7f8648159740 2024-05-09T06:36:18.173 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.188+0000 7f864f8f5640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f86480b42c0 con 0x7f86300039b0 2024-05-09T06:36:18.173 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.188+0000 7f864f8f5640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f86480d2800 con 0x7f86480cc1e0 2024-05-09T06:36:18.173 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.188+0000 7f864de92640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8648159740 0x7f8648151a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:18.173 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.188+0000 7f864de92640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8648159740 0x7f8648151a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49906/0 (socket says 172.21.15.3:49906) 2024-05-09T06:36:18.173 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.188+0000 7f864de92640 1 -- 172.21.15.3:0/1096470935 learned_addr learned my addr 172.21.15.3:0/1096470935 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:36:18.173 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.188+0000 7f864ce90640 1 --2- 172.21.15.3:0/1096470935 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f86480cc1e0 0x7f8648151540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:18.173 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.188+0000 7f864de92640 1 -- 172.21.15.3:0/1096470935 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f86480cc1e0 msgr2=0x7f8648151540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:18.173 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.188+0000 7f864de92640 1 --2- 172.21.15.3:0/1096470935 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f86480cc1e0 0x7f8648151540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:18.173 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.188+0000 7f864de92640 1 -- 172.21.15.3:0/1096470935 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f86300039b0 msgr2=0x7f86480cd920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:18.173 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.188+0000 7f864d691640 1 --2- 172.21.15.3:0/1096470935 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f86300039b0 0x7f86480cd920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:18.173 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.188+0000 7f864de92640 1 --2- 172.21.15.3:0/1096470935 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f86300039b0 0x7f86480cd920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:18.173 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.188+0000 7f864de92640 1 -- 172.21.15.3:0/1096470935 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86480cc6a0 con 0x7f8648159740 2024-05-09T06:36:18.174 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.189+0000 7f864de92640 1 --2- 172.21.15.3:0/1096470935 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8648159740 0x7f8648151a80 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f8648161ff0 tx=0x7f863800e960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:18.174 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.189+0000 7f8645ffb640 1 -- 172.21.15.3:0/1096470935 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8638019070 con 0x7f8648159740 2024-05-09T06:36:18.174 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.189+0000 7f8645ffb640 1 -- 172.21.15.3:0/1096470935 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8638003b80 con 0x7f8648159740 2024-05-09T06:36:18.174 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.189+0000 7f864f8f5640 1 -- 172.21.15.3:0/1096470935 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86480cc980 con 0x7f8648159740 2024-05-09T06:36:18.174 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.189+0000 7f8645ffb640 1 -- 172.21.15.3:0/1096470935 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86380050d0 con 0x7f8648159740 2024-05-09T06:36:18.174 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.189+0000 7f864f8f5640 1 -- 172.21.15.3:0/1096470935 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86480cde60 con 0x7f8648159740 2024-05-09T06:36:18.174 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.190+0000 7f864f8f5640 1 -- 172.21.15.3:0/1096470935 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f864815fbe0 con 0x7f8648159740 2024-05-09T06:36:18.174 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.190+0000 7f8645ffb640 1 -- 172.21.15.3:0/1096470935 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f863801c070 con 0x7f8648159740 2024-05-09T06:36:18.174 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.191+0000 7f8645ffb640 1 --2- 172.21.15.3:0/1096470935 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f863406be60 0x7f863406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:18.174 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.191+0000 7f8645ffb640 1 -- 172.21.15.3:0/1096470935 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f863808a4d0 con 0x7f8648159740 2024-05-09T06:36:18.174 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.191+0000 7f864d691640 1 --2- 172.21.15.3:0/1096470935 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f863406be60 0x7f863406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:18.174 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.192+0000 7f864d691640 1 --2- 172.21.15.3:0/1096470935 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f863406be60 0x7f863406e320 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f8640033000 tx=0x7f864002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:18.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.195+0000 7f8645ffb640 1 -- 172.21.15.3:0/1096470935 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f8638058e80 con 0x7f8648159740 2024-05-09T06:36:18.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.377+0000 7f864f8f5640 1 -- 172.21.15.3:0/1096470935 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f8648069550 con 0x7f8648159740 2024-05-09T06:36:18.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.377+0000 7f8645ffb640 1 -- 172.21.15.3:0/1096470935 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7f8638088e20 con 0x7f8648159740 2024-05-09T06:36:18.358 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 64 2024-05-09T06:36:18.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.379+0000 7f864f8f5640 1 -- 172.21.15.3:0/1096470935 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f863406be60 msgr2=0x7f863406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:18.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.379+0000 7f864f8f5640 1 --2- 172.21.15.3:0/1096470935 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f863406be60 0x7f863406e320 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f8640033000 tx=0x7f864002e000 comp rx=0 tx=0).stop 2024-05-09T06:36:18.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.379+0000 7f864f8f5640 1 -- 172.21.15.3:0/1096470935 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8648159740 msgr2=0x7f8648151a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:18.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.379+0000 7f864f8f5640 1 --2- 172.21.15.3:0/1096470935 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8648159740 0x7f8648151a80 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f8648161ff0 tx=0x7f863800e960 comp rx=0 tx=0).stop 2024-05-09T06:36:18.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.379+0000 7f864f8f5640 1 -- 172.21.15.3:0/1096470935 shutdown_connections 2024-05-09T06:36:18.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.379+0000 7f864f8f5640 1 --2- 172.21.15.3:0/1096470935 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f863406be60 0x7f863406e320 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:18.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.379+0000 7f864f8f5640 1 --2- 172.21.15.3:0/1096470935 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8648159740 0x7f8648151a80 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:18.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.380+0000 7f864f8f5640 1 --2- 172.21.15.3:0/1096470935 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f86480cc1e0 0x7f8648151540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:18.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.380+0000 7f864f8f5640 1 --2- 172.21.15.3:0/1096470935 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f86300039b0 0x7f86480cd920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:18.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.380+0000 7f864f8f5640 1 -- 172.21.15.3:0/1096470935 >> 172.21.15.3:0/1096470935 conn(0x7f864809d500 msgr2=0x7f86480b2e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:18.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.380+0000 7f864f8f5640 1 -- 172.21.15.3:0/1096470935 shutdown_connections 2024-05-09T06:36:18.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.380+0000 7f864f8f5640 1 -- 172.21.15.3:0/1096470935 wait complete. 2024-05-09T06:36:18.371 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T06:36:18.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.495+0000 7f26a62a5640 1 Processor -- start 2024-05-09T06:36:18.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.495+0000 7f26a62a5640 1 -- start start 2024-05-09T06:36:18.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.496+0000 7f26a62a5640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f26a00ae9c0 0x7f26a006c400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:18.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.496+0000 7f26a62a5640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f26a006cab0 con 0x7f26a006d8f0 2024-05-09T06:36:18.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.496+0000 7f26a62a5640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f26a00b4120 con 0x7f26a015e100 2024-05-09T06:36:18.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.496+0000 7f26a62a5640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f26a00b3ed0 con 0x7f26a00ae9c0 2024-05-09T06:36:18.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.496+0000 7f269f7fe640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f26a006d8f0 0x7f26a006bcf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:49326/0 (socket says 172.21.15.3:49326) 2024-05-09T06:36:18.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.496+0000 7f269f7fe640 1 -- 172.21.15.3:0/2114053259 learned_addr learned my addr 172.21.15.3:0/2114053259 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-05-09T06:36:18.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.496+0000 7f26a4842640 1 --2- 172.21.15.3:0/2114053259 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f26a00ae9c0 0x7f26a006c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:18.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.496+0000 7f269effd640 1 -- 172.21.15.3:0/2114053259 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3574401512 0 0) 0x7f26a006cab0 con 0x7f26a006d8f0 2024-05-09T06:36:18.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.496+0000 7f269effd640 1 -- 172.21.15.3:0/2114053259 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2688003520 con 0x7f26a006d8f0 2024-05-09T06:36:18.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.496+0000 7f269effd640 1 -- 172.21.15.3:0/2114053259 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1041071770 0 0) 0x7f26a00b4120 con 0x7f26a015e100 2024-05-09T06:36:18.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.496+0000 7f269effd640 1 -- 172.21.15.3:0/2114053259 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f26a006cab0 con 0x7f26a015e100 2024-05-09T06:36:18.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.496+0000 7f26a4842640 1 -- 172.21.15.3:0/2114053259 >> v1:172.21.15.190:6789/0 conn(0x7f26a015e100 legacy=0x7f26a00ae2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:18.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.496+0000 7f26a4842640 1 -- 172.21.15.3:0/2114053259 >> v1:172.21.15.3:6789/0 conn(0x7f26a006d8f0 legacy=0x7f26a006bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:18.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.496+0000 7f26a4842640 1 -- 172.21.15.3:0/2114053259 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26a00b4400 con 0x7f26a00ae9c0 2024-05-09T06:36:18.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.497+0000 7f26a4842640 1 --2- 172.21.15.3:0/2114053259 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f26a00ae9c0 0x7f26a006c400 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f26a00b4050 tx=0x7f269802a630 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f9c95f03bfd85583 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:18.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.497+0000 7f269effd640 1 -- 172.21.15.3:0/2114053259 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2698007740 con 0x7f26a00ae9c0 2024-05-09T06:36:18.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.497+0000 7f269effd640 1 -- 172.21.15.3:0/2114053259 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f269802ae50 con 0x7f26a00ae9c0 2024-05-09T06:36:18.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.497+0000 7f269effd640 1 -- 172.21.15.3:0/2114053259 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2698007b00 con 0x7f26a00ae9c0 2024-05-09T06:36:18.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.498+0000 7f26a62a5640 1 -- 172.21.15.3:0/2114053259 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f26a00ae9c0 msgr2=0x7f26a006c400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:18.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.498+0000 7f26a62a5640 1 --2- 172.21.15.3:0/2114053259 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f26a00ae9c0 0x7f26a006c400 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f26a00b4050 tx=0x7f269802a630 comp rx=0 tx=0).stop 2024-05-09T06:36:18.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.498+0000 7f26a62a5640 1 -- 172.21.15.3:0/2114053259 shutdown_connections 2024-05-09T06:36:18.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.498+0000 7f26a62a5640 1 --2- 172.21.15.3:0/2114053259 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f26a00ae9c0 0x7f26a006c400 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:18.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.498+0000 7f26a62a5640 1 -- 172.21.15.3:0/2114053259 >> 172.21.15.3:0/2114053259 conn(0x7f26a00d05d0 msgr2=0x7f26a00d09e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:18.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.498+0000 7f26a62a5640 1 -- 172.21.15.3:0/2114053259 shutdown_connections 2024-05-09T06:36:18.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.498+0000 7f26a62a5640 1 -- 172.21.15.3:0/2114053259 wait complete. 2024-05-09T06:36:18.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.499+0000 7f26a62a5640 1 Processor -- start 2024-05-09T06:36:18.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.499+0000 7f26a62a5640 1 -- start start 2024-05-09T06:36:18.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.500+0000 7f26a62a5640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f26a006d8f0 0x7f26a0147640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:18.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.500+0000 7f26a62a5640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f26a00ae9c0 0x7f26a0146250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:18.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.500+0000 7f26a62a5640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f26a015e100 0x7f26a006b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:18.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.500+0000 7f26a62a5640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f26a0169160 con 0x7f26a00ae9c0 2024-05-09T06:36:18.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.500+0000 7f26a62a5640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f26a00b4770 con 0x7f26a015e100 2024-05-09T06:36:18.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.500+0000 7f26a62a5640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f26a01692e0 con 0x7f26a006d8f0 2024-05-09T06:36:18.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.500+0000 7f269f7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f26a00ae9c0 0x7f26a0146250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:18.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.500+0000 7f269f7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f26a00ae9c0 0x7f26a0146250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49920/0 (socket says 172.21.15.3:49920) 2024-05-09T06:36:18.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.500+0000 7f269f7fe640 1 -- 172.21.15.3:0/3468009511 learned_addr learned my addr 172.21.15.3:0/3468009511 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:36:18.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.500+0000 7f269ffff640 1 --2- 172.21.15.3:0/3468009511 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f26a006d8f0 0x7f26a0147640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:18.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.500+0000 7f26a4842640 1 --2- 172.21.15.3:0/3468009511 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f26a015e100 0x7f26a006b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:18.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.500+0000 7f269f7fe640 1 -- 172.21.15.3:0/3468009511 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f26a006d8f0 msgr2=0x7f26a0147640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:18.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.500+0000 7f269f7fe640 1 --2- 172.21.15.3:0/3468009511 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f26a006d8f0 0x7f26a0147640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:18.486 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.500+0000 7f269f7fe640 1 -- 172.21.15.3:0/3468009511 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f26a015e100 msgr2=0x7f26a006b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:18.486 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.500+0000 7f269f7fe640 1 --2- 172.21.15.3:0/3468009511 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f26a015e100 0x7f26a006b310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:18.486 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.500+0000 7f269f7fe640 1 -- 172.21.15.3:0/3468009511 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26a0147e90 con 0x7f26a00ae9c0 2024-05-09T06:36:18.486 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.500+0000 7f269f7fe640 1 --2- 172.21.15.3:0/3468009511 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f26a00ae9c0 0x7f26a0146250 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f269400c450 tx=0x7f269400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:18.486 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.501+0000 7f269cff9640 1 -- 172.21.15.3:0/3468009511 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2694017070 con 0x7f26a00ae9c0 2024-05-09T06:36:18.486 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.501+0000 7f269cff9640 1 -- 172.21.15.3:0/3468009511 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2694005da0 con 0x7f26a00ae9c0 2024-05-09T06:36:18.486 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.501+0000 7f26a62a5640 1 -- 172.21.15.3:0/3468009511 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26a0148170 con 0x7f26a00ae9c0 2024-05-09T06:36:18.486 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.501+0000 7f269cff9640 1 -- 172.21.15.3:0/3468009511 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2694007290 con 0x7f26a00ae9c0 2024-05-09T06:36:18.486 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.501+0000 7f26a62a5640 1 -- 172.21.15.3:0/3468009511 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26a0148730 con 0x7f26a00ae9c0 2024-05-09T06:36:18.486 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.502+0000 7f26a62a5640 1 -- 172.21.15.3:0/3468009511 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26a004f660 con 0x7f26a00ae9c0 2024-05-09T06:36:18.486 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.502+0000 7f269cff9640 1 -- 172.21.15.3:0/3468009511 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f269401a070 con 0x7f26a00ae9c0 2024-05-09T06:36:18.486 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.503+0000 7f269cff9640 1 --2- 172.21.15.3:0/3468009511 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f268c06be30 0x7f268c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:18.487 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.503+0000 7f269cff9640 1 -- 172.21.15.3:0/3468009511 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f269408a160 con 0x7f26a00ae9c0 2024-05-09T06:36:18.487 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.503+0000 7f269ffff640 1 --2- 172.21.15.3:0/3468009511 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f268c06be30 0x7f268c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:18.487 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.503+0000 7f269ffff640 1 --2- 172.21.15.3:0/3468009511 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f268c06be30 0x7f268c06e2f0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f26900015b0 tx=0x7f2690009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:18.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.507+0000 7f269cff9640 1 -- 172.21.15.3:0/3468009511 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f2694058ba0 con 0x7f26a00ae9c0 2024-05-09T06:36:18.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.728+0000 7f26a62a5640 1 -- 172.21.15.3:0/3468009511 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f26a0156510 con 0x7f26a00ae9c0 2024-05-09T06:36:18.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.729+0000 7f269cff9640 1 -- 172.21.15.3:0/3468009511 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5172 (secure 0 0 0) 0x7f26940584a0 con 0x7f26a00ae9c0 2024-05-09T06:36:18.710 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-09T06:36:18.710 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4272,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6834/126273055","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":126273055},{"type":"v1","addr":"172.21.15.3:6834","nonce":126273055}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4273,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6836/3885692063","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":3885692063},{"type":"v1","addr":"172.21.15.190:6836","nonce":3885692063}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4275,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/3461228756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6833","nonce":3461228756},{"type":"v1","addr":"172.21.15.3:6835","nonce":3461228756}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-09T06:36:07.221126+0000","modified":"2024-05-09T06:36:10.231051+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4276},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4276":{"gid":4276,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.190:6837/4111472052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6835","nonce":4111472052},{"type":"v1","addr":"172.21.15.190:6837","nonce":4111472052}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4276,"qdb_cluster":[4276]},"id":1}]} 2024-05-09T06:36:18.710 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 8 2024-05-09T06:36:18.712 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.731+0000 7f26a62a5640 1 -- 172.21.15.3:0/3468009511 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f268c06be30 msgr2=0x7f268c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:18.712 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.731+0000 7f26a62a5640 1 --2- 172.21.15.3:0/3468009511 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f268c06be30 0x7f268c06e2f0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f26900015b0 tx=0x7f2690009040 comp rx=0 tx=0).stop 2024-05-09T06:36:18.712 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.731+0000 7f26a62a5640 1 -- 172.21.15.3:0/3468009511 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f26a00ae9c0 msgr2=0x7f26a0146250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:18.712 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.731+0000 7f26a62a5640 1 --2- 172.21.15.3:0/3468009511 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f26a00ae9c0 0x7f26a0146250 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f269400c450 tx=0x7f269400c910 comp rx=0 tx=0).stop 2024-05-09T06:36:18.712 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.731+0000 7f26a62a5640 1 -- 172.21.15.3:0/3468009511 shutdown_connections 2024-05-09T06:36:18.712 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.731+0000 7f26a62a5640 1 --2- 172.21.15.3:0/3468009511 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f268c06be30 0x7f268c06e2f0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:18.712 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.731+0000 7f26a62a5640 1 --2- 172.21.15.3:0/3468009511 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f26a015e100 0x7f26a006b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:18.713 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.731+0000 7f26a62a5640 1 --2- 172.21.15.3:0/3468009511 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f26a00ae9c0 0x7f26a0146250 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:18.713 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.731+0000 7f26a62a5640 1 --2- 172.21.15.3:0/3468009511 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f26a006d8f0 0x7f26a0147640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:18.713 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.731+0000 7f26a62a5640 1 -- 172.21.15.3:0/3468009511 >> 172.21.15.3:0/3468009511 conn(0x7f26a00d05d0 msgr2=0x7f26a00ae170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:18.713 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.731+0000 7f26a62a5640 1 -- 172.21.15.3:0/3468009511 shutdown_connections 2024-05-09T06:36:18.713 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.731+0000 7f26a62a5640 1 -- 172.21.15.3:0/3468009511 wait complete. 2024-05-09T06:36:18.723 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T06:36:18.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.848+0000 7fea39509640 1 Processor -- start 2024-05-09T06:36:18.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.848+0000 7fea39509640 1 -- start start 2024-05-09T06:36:18.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.849+0000 7fea39509640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fea34147d00 0x7fea340b3e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:18.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.849+0000 7fea39509640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fea340b43c0 0x7fea3415ed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:18.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.849+0000 7fea39509640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fea34162f50 con 0x7fea3415f270 2024-05-09T06:36:18.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.849+0000 7fea39509640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fea34162190 con 0x7fea340b43c0 2024-05-09T06:36:18.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.849+0000 7fea39509640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fea34162470 con 0x7fea34147d00 2024-05-09T06:36:18.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.849+0000 7fea2bfff640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fea340b43c0 0x7fea3415ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:18.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.849+0000 7fea337fe640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fea3415f270 0x7fea3406f5f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:49334/0 (socket says 172.21.15.3:49334) 2024-05-09T06:36:18.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.849+0000 7fea337fe640 1 -- 172.21.15.3:0/3004431188 learned_addr learned my addr 172.21.15.3:0/3004431188 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-05-09T06:36:18.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.849+0000 7fea32ffd640 1 --2- 172.21.15.3:0/3004431188 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fea34147d00 0x7fea340b3e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:18.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.849+0000 7fea2bfff640 1 -- 172.21.15.3:0/3004431188 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fea34147d00 msgr2=0x7fea340b3e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:18.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.849+0000 7fea2bfff640 1 --2- 172.21.15.3:0/3004431188 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fea34147d00 0x7fea340b3e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:18.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.849+0000 7fea2bfff640 1 -- 172.21.15.3:0/3004431188 >> v1:172.21.15.3:6789/0 conn(0x7fea3415f270 legacy=0x7fea3406f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:18.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.849+0000 7fea2bfff640 1 -- 172.21.15.3:0/3004431188 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea34162750 con 0x7fea340b43c0 2024-05-09T06:36:18.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.849+0000 7fea327fc640 1 -- 172.21.15.3:0/3004431188 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4178257481 0 0) 0x7fea34162f50 con 0x7fea3415f270 2024-05-09T06:36:18.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.850+0000 7fea2bfff640 1 --2- 172.21.15.3:0/3004431188 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fea340b43c0 0x7fea3415ed30 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fea1c0097a0 tx=0x7fea1c02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=888e11e7ac3cfb08 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:18.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.850+0000 7fea327fc640 1 -- 172.21.15.3:0/3004431188 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea1c037070 con 0x7fea340b43c0 2024-05-09T06:36:18.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.850+0000 7fea327fc640 1 -- 172.21.15.3:0/3004431188 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea1c032440 con 0x7fea340b43c0 2024-05-09T06:36:18.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.850+0000 7fea327fc640 1 -- 172.21.15.3:0/3004431188 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea1c032810 con 0x7fea340b43c0 2024-05-09T06:36:18.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.851+0000 7fea39509640 1 -- 172.21.15.3:0/3004431188 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fea340b43c0 msgr2=0x7fea3415ed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:18.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.851+0000 7fea39509640 1 --2- 172.21.15.3:0/3004431188 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fea340b43c0 0x7fea3415ed30 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fea1c0097a0 tx=0x7fea1c02a180 comp rx=0 tx=0).stop 2024-05-09T06:36:18.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.851+0000 7fea39509640 1 -- 172.21.15.3:0/3004431188 shutdown_connections 2024-05-09T06:36:18.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.851+0000 7fea39509640 1 --2- 172.21.15.3:0/3004431188 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fea340b43c0 0x7fea3415ed30 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:18.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.851+0000 7fea39509640 1 --2- 172.21.15.3:0/3004431188 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fea34147d00 0x7fea340b3e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:18.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.851+0000 7fea39509640 1 -- 172.21.15.3:0/3004431188 >> 172.21.15.3:0/3004431188 conn(0x7fea340671a0 msgr2=0x7fea340675b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:18.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.851+0000 7fea39509640 1 -- 172.21.15.3:0/3004431188 shutdown_connections 2024-05-09T06:36:18.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.851+0000 7fea39509640 1 -- 172.21.15.3:0/3004431188 wait complete. 2024-05-09T06:36:18.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.852+0000 7fea39509640 1 Processor -- start 2024-05-09T06:36:18.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.852+0000 7fea39509640 1 -- start start 2024-05-09T06:36:18.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.853+0000 7fea39509640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fea340b43c0 0x7fea340cecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:18.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.853+0000 7fea39509640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fea34147d00 0x7fea341465b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:18.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.853+0000 7fea39509640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fea3415f270 0x7fea34141070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:18.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.853+0000 7fea39509640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fea3406d880 con 0x7fea34147d00 2024-05-09T06:36:18.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.853+0000 7fea39509640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fea341480e0 con 0x7fea340b43c0 2024-05-09T06:36:18.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.853+0000 7fea39509640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fea3415f700 con 0x7fea3415f270 2024-05-09T06:36:18.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.853+0000 7fea2bfff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fea34147d00 0x7fea341465b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:18.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.853+0000 7fea2bfff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fea34147d00 0x7fea341465b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49942/0 (socket says 172.21.15.3:49942) 2024-05-09T06:36:18.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.853+0000 7fea2bfff640 1 -- 172.21.15.3:0/2765775371 learned_addr learned my addr 172.21.15.3:0/2765775371 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:36:18.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.853+0000 7fea32ffd640 1 --2- 172.21.15.3:0/2765775371 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fea340b43c0 0x7fea340cecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:18.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.853+0000 7fea337fe640 1 --2- 172.21.15.3:0/2765775371 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fea3415f270 0x7fea34141070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:18.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.853+0000 7fea2bfff640 1 -- 172.21.15.3:0/2765775371 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fea3415f270 msgr2=0x7fea34141070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:18.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.853+0000 7fea2bfff640 1 --2- 172.21.15.3:0/2765775371 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fea3415f270 0x7fea34141070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:18.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.853+0000 7fea2bfff640 1 -- 172.21.15.3:0/2765775371 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fea340b43c0 msgr2=0x7fea340cecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:18.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.853+0000 7fea2bfff640 1 --2- 172.21.15.3:0/2765775371 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fea340b43c0 0x7fea340cecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:18.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.853+0000 7fea2bfff640 1 -- 172.21.15.3:0/2765775371 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea340cb2c0 con 0x7fea34147d00 2024-05-09T06:36:18.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.853+0000 7fea337fe640 1 --2- 172.21.15.3:0/2765775371 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fea3415f270 0x7fea34141070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:36:18.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.853+0000 7fea32ffd640 1 --2- 172.21.15.3:0/2765775371 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fea340b43c0 0x7fea340cecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:36:18.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.853+0000 7fea2bfff640 1 --2- 172.21.15.3:0/2765775371 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fea34147d00 0x7fea341465b0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fea1c035040 tx=0x7fea1c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:18.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.853+0000 7fea2b7fe640 1 -- 172.21.15.3:0/2765775371 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea1c037040 con 0x7fea34147d00 2024-05-09T06:36:18.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.853+0000 7fea2b7fe640 1 -- 172.21.15.3:0/2765775371 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea1c030770 con 0x7fea34147d00 2024-05-09T06:36:18.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.853+0000 7fea2b7fe640 1 -- 172.21.15.3:0/2765775371 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea1c030b80 con 0x7fea34147d00 2024-05-09T06:36:18.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.853+0000 7fea39509640 1 -- 172.21.15.3:0/2765775371 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea340cb5a0 con 0x7fea34147d00 2024-05-09T06:36:18.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.854+0000 7fea39509640 1 -- 172.21.15.3:0/2765775371 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea340cbb60 con 0x7fea34147d00 2024-05-09T06:36:18.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.855+0000 7fea39509640 1 -- 172.21.15.3:0/2765775371 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea3406bd80 con 0x7fea34147d00 2024-05-09T06:36:18.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.855+0000 7fea2b7fe640 1 -- 172.21.15.3:0/2765775371 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fea1c044020 con 0x7fea34147d00 2024-05-09T06:36:18.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.855+0000 7fea2b7fe640 1 --2- 172.21.15.3:0/2765775371 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fea1406be30 0x7fea1406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:18.843 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.855+0000 7fea2b7fe640 1 -- 172.21.15.3:0/2765775371 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fea1c0ab440 con 0x7fea34147d00 2024-05-09T06:36:18.843 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.856+0000 7fea32ffd640 1 --2- 172.21.15.3:0/2765775371 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fea1406be30 0x7fea1406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:18.843 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.856+0000 7fea32ffd640 1 --2- 172.21.15.3:0/2765775371 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fea1406be30 0x7fea1406e2f0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fea24006fd0 tx=0x7fea24008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:18.844 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:18.860+0000 7fea2b7fe640 1 -- 172.21.15.3:0/2765775371 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fea3406bd80 con 0x7fea34147d00 2024-05-09T06:36:19.026 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:19.046+0000 7fea39509640 1 -- 172.21.15.3:0/2765775371 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fea3404f660 con 0x7fea34147d00 2024-05-09T06:36:19.027 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:19.047+0000 7fea2b7fe640 1 -- 172.21.15.3:0/2765775371 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16521 (secure 0 0 0) 0x7fea1c032e20 con 0x7fea34147d00 2024-05-09T06:36:19.028 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-09T06:36:19.028 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":20,"fsid":"16ed737d-2665-431a-8ac6-ef242be3975a","created":"2024-05-09T06:35:42.541368+0000","modified":"2024-05-09T06:36:08.856391+0000","last_up_change":"2024-05-09T06:35:53.778274+0000","last_in_change":"2024-05-09T06:35:44.780586+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T06:35:54.819999+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T06:36:04.931314+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T06:36:06.200433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fae3a8db-4826-4f3c-b33a-12c09bb5df6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":1897849827},{"type":"v1","addr":"172.21.15.3:6817","nonce":1897849827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":1897849827},{"type":"v1","addr":"172.21.15.3:6819","nonce":1897849827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":1897849827},{"type":"v1","addr":"172.21.15.3:6823","nonce":1897849827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":1897849827},{"type":"v1","addr":"172.21.15.3:6821","nonce":1897849827}]},"public_addr":"172.21.15.3:6817/1897849827","cluster_addr":"172.21.15.3:6819/1897849827","heartbeat_back_addr":"172.21.15.3:6823/1897849827","heartbeat_front_addr":"172.21.15.3:6821/1897849827","state":["exists","up"]},{"osd":1,"uuid":"646e882d-4ce4-4720-8c84-f71df563a35e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":926805498},{"type":"v1","addr":"172.21.15.3:6801","nonce":926805498}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":926805498},{"type":"v1","addr":"172.21.15.3:6803","nonce":926805498}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":926805498},{"type":"v1","addr":"172.21.15.3:6807","nonce":926805498}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":926805498},{"type":"v1","addr":"172.21.15.3:6805","nonce":926805498}]},"public_addr":"172.21.15.3:6801/926805498","cluster_addr":"172.21.15.3:6803/926805498","heartbeat_back_addr":"172.21.15.3:6807/926805498","heartbeat_front_addr":"172.21.15.3:6805/926805498","state":["exists","up"]},{"osd":2,"uuid":"1abbadb5-a570-4359-9643-acff3ede35a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":801295451},{"type":"v1","addr":"172.21.15.3:6809","nonce":801295451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":801295451},{"type":"v1","addr":"172.21.15.3:6811","nonce":801295451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":801295451},{"type":"v1","addr":"172.21.15.3:6815","nonce":801295451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":801295451},{"type":"v1","addr":"172.21.15.3:6813","nonce":801295451}]},"public_addr":"172.21.15.3:6809/801295451","cluster_addr":"172.21.15.3:6811/801295451","heartbeat_back_addr":"172.21.15.3:6815/801295451","heartbeat_front_addr":"172.21.15.3:6813/801295451","state":["exists","up"]},{"osd":3,"uuid":"62ac4dc1-2371-4dd2-acd3-e2dbb11aedf5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":1903389857},{"type":"v1","addr":"172.21.15.3:6825","nonce":1903389857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":1903389857},{"type":"v1","addr":"172.21.15.3:6827","nonce":1903389857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":1903389857},{"type":"v1","addr":"172.21.15.3:6831","nonce":1903389857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":1903389857},{"type":"v1","addr":"172.21.15.3:6829","nonce":1903389857}]},"public_addr":"172.21.15.3:6825/1903389857","cluster_addr":"172.21.15.3:6827/1903389857","heartbeat_back_addr":"172.21.15.3:6831/1903389857","heartbeat_front_addr":"172.21.15.3:6829/1903389857","state":["exists","up"]},{"osd":4,"uuid":"ae808398-8326-4d7c-9319-a2a0eafffb59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":2143624576},{"type":"v1","addr":"172.21.15.190:6801","nonce":2143624576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":2143624576},{"type":"v1","addr":"172.21.15.190:6803","nonce":2143624576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":2143624576},{"type":"v1","addr":"172.21.15.190:6807","nonce":2143624576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":2143624576},{"type":"v1","addr":"172.21.15.190:6805","nonce":2143624576}]},"public_addr":"172.21.15.190:6801/2143624576","cluster_addr":"172.21.15.190:6803/2143624576","heartbeat_back_addr":"172.21.15.190:6807/2143624576","heartbeat_front_addr":"172.21.15.190:6805/2143624576","state":["exists","up"]},{"osd":5,"uuid":"918adf50-c440-48ac-9a8d-4837562d7275","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":1766923126},{"type":"v1","addr":"172.21.15.190:6814","nonce":1766923126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":1766923126},{"type":"v1","addr":"172.21.15.190:6818","nonce":1766923126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":1766923126},{"type":"v1","addr":"172.21.15.190:6823","nonce":1766923126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":1766923126},{"type":"v1","addr":"172.21.15.190:6821","nonce":1766923126}]},"public_addr":"172.21.15.190:6814/1766923126","cluster_addr":"172.21.15.190:6818/1766923126","heartbeat_back_addr":"172.21.15.190:6823/1766923126","heartbeat_front_addr":"172.21.15.190:6821/1766923126","state":["exists","up"]},{"osd":6,"uuid":"ac2882bc-2a7c-463c-8f6c-76ff49828adc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":342414248},{"type":"v1","addr":"172.21.15.190:6825","nonce":342414248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":342414248},{"type":"v1","addr":"172.21.15.190:6827","nonce":342414248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":342414248},{"type":"v1","addr":"172.21.15.190:6831","nonce":342414248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":342414248},{"type":"v1","addr":"172.21.15.190:6829","nonce":342414248}]},"public_addr":"172.21.15.190:6825/342414248","cluster_addr":"172.21.15.190:6827/342414248","heartbeat_back_addr":"172.21.15.190:6831/342414248","heartbeat_front_addr":"172.21.15.190:6829/342414248","state":["exists","up"]},{"osd":7,"uuid":"dd4e767a-4730-4c70-8bd5-3f088140a5a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":537849891},{"type":"v1","addr":"172.21.15.190:6809","nonce":537849891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":537849891},{"type":"v1","addr":"172.21.15.190:6811","nonce":537849891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6817","nonce":537849891},{"type":"v1","addr":"172.21.15.190:6819","nonce":537849891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6813","nonce":537849891},{"type":"v1","addr":"172.21.15.190:6815","nonce":537849891}]},"public_addr":"172.21.15.190:6809/537849891","cluster_addr":"172.21.15.190:6811/537849891","heartbeat_back_addr":"172.21.15.190:6819/537849891","heartbeat_front_addr":"172.21.15.190:6815/537849891","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.889784+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.974002+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.973549+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.955396+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.928471+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.902418+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.958696+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.926474+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T06:36:19.030 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:19.049+0000 7fea39509640 1 -- 172.21.15.3:0/2765775371 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fea1406be30 msgr2=0x7fea1406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:19.030 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:19.049+0000 7fea39509640 1 --2- 172.21.15.3:0/2765775371 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fea1406be30 0x7fea1406e2f0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fea24006fd0 tx=0x7fea24008040 comp rx=0 tx=0).stop 2024-05-09T06:36:19.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:19.049+0000 7fea39509640 1 -- 172.21.15.3:0/2765775371 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fea34147d00 msgr2=0x7fea341465b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:19.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:19.049+0000 7fea39509640 1 --2- 172.21.15.3:0/2765775371 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fea34147d00 0x7fea341465b0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fea1c035040 tx=0x7fea1c0046c0 comp rx=0 tx=0).stop 2024-05-09T06:36:19.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:19.049+0000 7fea39509640 1 -- 172.21.15.3:0/2765775371 shutdown_connections 2024-05-09T06:36:19.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:19.049+0000 7fea39509640 1 --2- 172.21.15.3:0/2765775371 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fea1406be30 0x7fea1406e2f0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:19.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:19.049+0000 7fea39509640 1 --2- 172.21.15.3:0/2765775371 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fea3415f270 0x7fea34141070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:19.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:19.049+0000 7fea39509640 1 --2- 172.21.15.3:0/2765775371 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fea34147d00 0x7fea341465b0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:19.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:19.049+0000 7fea39509640 1 --2- 172.21.15.3:0/2765775371 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fea340b43c0 0x7fea340cecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:19.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:19.049+0000 7fea39509640 1 -- 172.21.15.3:0/2765775371 >> 172.21.15.3:0/2765775371 conn(0x7fea340671a0 msgr2=0x7fea341425d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:19.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:19.049+0000 7fea39509640 1 -- 172.21.15.3:0/2765775371 shutdown_connections 2024-05-09T06:36:19.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:19.049+0000 7fea39509640 1 -- 172.21.15.3:0/2765775371 wait complete. 2024-05-09T06:36:19.041 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-09T06:35:54.819999+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-09T06:36:04.931314+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-09T06:36:06.200433+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-09T06:36:19.042 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-09T06:36:19.148 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:19.168+0000 7fca78aa5640 1 Processor -- start 2024-05-09T06:36:19.149 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:19.168+0000 7fca78aa5640 1 -- start start 2024-05-09T06:36:19.150 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:19.168+0000 7fca78aa5640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fca7415e5c0 0x7fca740d2800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:19.150 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:19.168+0000 7fca78aa5640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fca740d1300 0x7fca740b47e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:19.150 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:19.169+0000 7fca78aa5640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fca7415e9a0 con 0x7fca740d4c40 2024-05-09T06:36:19.150 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:19.169+0000 7fca78aa5640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7fca740d31c0 con 0x7fca740d1300 2024-05-09T06:36:19.150 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:19.169+0000 7fca78aa5640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fca740d50a0 con 0x7fca7415e5c0 2024-05-09T06:36:19.150 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:19.169+0000 7fca7259c640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fca7415e5c0 0x7fca740d2800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:19.151 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:19.169+0000 7fca72d9d640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fca740d1300 0x7fca740b47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:19.151 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:19.169+0000 7fca72d9d640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fca740d1300 0x7fca740b47e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.3:48108/0 (socket says 172.21.15.3:48108) 2024-05-09T06:36:19.151 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:19.169+0000 7fca6bfff640 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7fca740d4c40 0x7fca740d0bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.3:58570/0 (socket says 172.21.15.3:58570) 2024-05-09T06:36:19.151 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:19.169+0000 7fca7259c640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fca7415e5c0 0x7fca740d2800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.3:45454/0 (socket says 172.21.15.3:45454) 2024-05-09T06:36:19.151 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:19.169+0000 7fca6bfff640 1 -- 172.21.15.3:0/2994911391 learned_addr learned my addr 172.21.15.3:0/2994911391 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-05-09T06:36:19.151 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:19.169+0000 7fca71d9b640 1 -- 172.21.15.3:0/2994911391 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1175150911 0 0) 0x7fca7415e9a0 con 0x7fca740d4c40 2024-05-09T06:36:19.151 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:19.169+0000 7fca71d9b640 1 -- 172.21.15.3:0/2994911391 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fca60003520 con 0x7fca740d4c40 2024-05-09T06:36:19.151 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:19.169+0000 7fca7259c640 1 -- 172.21.15.3:0/2994911391 >> v2:172.21.15.190:3300/0 conn(0x7fca740d1300 msgr2=0x7fca740b47e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:19.151 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:19.169+0000 7fca7259c640 1 --2- 172.21.15.3:0/2994911391 >> v2:172.21.15.190:3300/0 conn(0x7fca740d1300 0x7fca740b47e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:19.151 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:19.169+0000 7fca7259c640 1 -- 172.21.15.3:0/2994911391 >> v1:172.21.15.190:6789/0 conn(0x7fca740d4c40 legacy=0x7fca740d0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:19.151 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:19.169+0000 7fca7259c640 1 -- 172.21.15.3:0/2994911391 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca740d2ee0 con 0x7fca7415e5c0 2024-05-09T06:36:19.152 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:19.170+0000 7fca7259c640 1 --2- 172.21.15.3:0/2994911391 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fca7415e5c0 0x7fca740d2800 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fca640097a0 tx=0x7fca6402a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1b4f8507160fa856 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:19.153 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:19.170+0000 7fca71d9b640 1 -- 172.21.15.3:0/2994911391 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca64037070 con 0x7fca7415e5c0 2024-05-09T06:36:19.153 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:19.170+0000 7fca71d9b640 1 -- 172.21.15.3:0/2994911391 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca64032440 con 0x7fca7415e5c0 2024-05-09T06:36:19.153 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:19.170+0000 7fca71d9b640 1 -- 172.21.15.3:0/2994911391 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca64032810 con 0x7fca7415e5c0 2024-05-09T06:36:19.153 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:19.171+0000 7fca78aa5640 1 -- 172.21.15.3:0/2994911391 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fca7415e5c0 msgr2=0x7fca740d2800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:19.153 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:19.171+0000 7fca78aa5640 1 --2- 172.21.15.3:0/2994911391 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fca7415e5c0 0x7fca740d2800 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fca640097a0 tx=0x7fca6402a180 comp rx=0 tx=0).stop 2024-05-09T06:36:19.153 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:19.171+0000 7fca78aa5640 1 -- 172.21.15.3:0/2994911391 shutdown_connections 2024-05-09T06:36:19.153 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:19.171+0000 7fca78aa5640 1 --2- 172.21.15.3:0/2994911391 >> v2:172.21.15.190:3300/0 conn(0x7fca740d1300 0x7fca740b47e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:19.153 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:19.171+0000 7fca78aa5640 1 --2- 172.21.15.3:0/2994911391 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fca7415e5c0 0x7fca740d2800 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:19.153 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:19.171+0000 7fca78aa5640 1 -- 172.21.15.3:0/2994911391 >> 172.21.15.3:0/2994911391 conn(0x7fca7409d500 msgr2=0x7fca740d4220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:19.153 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:19.171+0000 7fca78aa5640 1 -- 172.21.15.3:0/2994911391 shutdown_connections 2024-05-09T06:36:19.153 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:19.171+0000 7fca78aa5640 1 -- 172.21.15.3:0/2994911391 wait complete. 2024-05-09T06:36:19.155 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:19.172+0000 7fca78aa5640 1 Processor -- start 2024-05-09T06:36:19.155 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:19.172+0000 7fca78aa5640 1 -- start start 2024-05-09T06:36:19.155 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:19.173+0000 7fca78aa5640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fca740d1300 0x7fca74160500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:19.155 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:19.173+0000 7fca78aa5640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fca740d4c40 0x7fca740cc5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:19.155 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:19.173+0000 7fca78aa5640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fca7415e5c0 0x7fca741612b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:19.155 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:19.173+0000 7fca78aa5640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fca740b42c0 con 0x7fca740d1300 2024-05-09T06:36:19.155 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:19.173+0000 7fca78aa5640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fca740b3fe0 con 0x7fca740d4c40 2024-05-09T06:36:19.155 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:19.173+0000 7fca78aa5640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fca740b4d20 con 0x7fca7415e5c0 2024-05-09T06:36:19.155 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:19.173+0000 7fca7259c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fca740d1300 0x7fca74160500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:19.157 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:19.173+0000 7fca7259c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fca740d1300 0x7fca74160500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49946/0 (socket says 172.21.15.3:49946) 2024-05-09T06:36:19.158 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:19.173+0000 7fca7259c640 1 -- 172.21.15.3:0/2069434706 learned_addr learned my addr 172.21.15.3:0/2069434706 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:36:19.158 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:19.174+0000 7fca72d9d640 1 --2- 172.21.15.3:0/2069434706 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fca7415e5c0 0x7fca741612b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:19.158 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:19.174+0000 7fca6bfff640 1 --2- 172.21.15.3:0/2069434706 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fca740d4c40 0x7fca740cc5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:19.158 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:19.174+0000 7fca7259c640 1 -- 172.21.15.3:0/2069434706 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fca7415e5c0 msgr2=0x7fca741612b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:19.158 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:19.174+0000 7fca7259c640 1 --2- 172.21.15.3:0/2069434706 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fca7415e5c0 0x7fca741612b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:19.158 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:19.174+0000 7fca7259c640 1 -- 172.21.15.3:0/2069434706 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fca740d4c40 msgr2=0x7fca740cc5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:19.158 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:19.174+0000 7fca7259c640 1 --2- 172.21.15.3:0/2069434706 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fca740d4c40 0x7fca740cc5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:19.158 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:19.174+0000 7fca7259c640 1 -- 172.21.15.3:0/2069434706 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca7415f740 con 0x7fca740d1300 2024-05-09T06:36:19.160 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:19.174+0000 7fca7259c640 1 --2- 172.21.15.3:0/2069434706 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fca740d1300 0x7fca74160500 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fca64000c00 tx=0x7fca640046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:19.160 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:19.174+0000 7fca6affd640 1 -- 172.21.15.3:0/2069434706 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca64041020 con 0x7fca740d1300 2024-05-09T06:36:19.160 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:19.174+0000 7fca6affd640 1 -- 172.21.15.3:0/2069434706 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca6403d640 con 0x7fca740d1300 2024-05-09T06:36:19.160 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:19.174+0000 7fca6affd640 1 -- 172.21.15.3:0/2069434706 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca64037040 con 0x7fca740d1300 2024-05-09T06:36:19.160 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:19.174+0000 7fca78aa5640 1 -- 172.21.15.3:0/2069434706 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca7415fa20 con 0x7fca740d1300 2024-05-09T06:36:19.160 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:19.174+0000 7fca78aa5640 1 -- 172.21.15.3:0/2069434706 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca7415ffe0 con 0x7fca740d1300 2024-05-09T06:36:19.160 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:19.175+0000 7fca78aa5640 1 -- 172.21.15.3:0/2069434706 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca74069550 con 0x7fca740d1300 2024-05-09T06:36:19.160 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:19.176+0000 7fca6affd640 1 -- 172.21.15.3:0/2069434706 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fca64044020 con 0x7fca740d1300 2024-05-09T06:36:19.160 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:19.176+0000 7fca6affd640 1 --2- 172.21.15.3:0/2069434706 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fca5406be30 0x7fca5406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:19.160 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:19.176+0000 7fca6affd640 1 -- 172.21.15.3:0/2069434706 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fca640ab4a0 con 0x7fca740d1300 2024-05-09T06:36:19.160 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:19.177+0000 7fca6bfff640 1 --2- 172.21.15.3:0/2069434706 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fca5406be30 0x7fca5406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:19.161 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:19.177+0000 7fca6bfff640 1 --2- 172.21.15.3:0/2069434706 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fca5406be30 0x7fca5406e2f0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fca740ba2b0 tx=0x7fca5c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:19.161 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:19.181+0000 7fca6affd640 1 -- 172.21.15.3:0/2069434706 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fca6404b030 con 0x7fca740d1300 2024-05-09T06:36:19.345 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:19.364+0000 7fca78aa5640 1 -- 172.21.15.3:0/2069434706 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fca74160220 con 0x7fca740d1300 2024-05-09T06:36:19.345 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:19.365+0000 7fca6affd640 1 -- 172.21.15.3:0/2069434706 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7fca64079810 con 0x7fca740d1300 2024-05-09T06:36:19.346 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 1 2024-05-09T06:36:19.348 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:19.367+0000 7fca78aa5640 1 -- 172.21.15.3:0/2069434706 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fca5406be30 msgr2=0x7fca5406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:19.348 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:19.367+0000 7fca78aa5640 1 --2- 172.21.15.3:0/2069434706 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fca5406be30 0x7fca5406e2f0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fca740ba2b0 tx=0x7fca5c009040 comp rx=0 tx=0).stop 2024-05-09T06:36:19.348 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:19.367+0000 7fca78aa5640 1 -- 172.21.15.3:0/2069434706 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fca740d1300 msgr2=0x7fca74160500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:19.348 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:19.367+0000 7fca78aa5640 1 --2- 172.21.15.3:0/2069434706 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fca740d1300 0x7fca74160500 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fca64000c00 tx=0x7fca640046c0 comp rx=0 tx=0).stop 2024-05-09T06:36:19.348 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:19.367+0000 7fca78aa5640 1 -- 172.21.15.3:0/2069434706 shutdown_connections 2024-05-09T06:36:19.348 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:19.367+0000 7fca78aa5640 1 --2- 172.21.15.3:0/2069434706 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fca5406be30 0x7fca5406e2f0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:19.348 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:19.367+0000 7fca78aa5640 1 --2- 172.21.15.3:0/2069434706 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fca7415e5c0 0x7fca741612b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:19.348 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:19.367+0000 7fca78aa5640 1 --2- 172.21.15.3:0/2069434706 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fca740d4c40 0x7fca740cc5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:19.348 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:19.367+0000 7fca78aa5640 1 --2- 172.21.15.3:0/2069434706 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fca740d1300 0x7fca74160500 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:19.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:19.367+0000 7fca78aa5640 1 -- 172.21.15.3:0/2069434706 >> 172.21.15.3:0/2069434706 conn(0x7fca7409d500 msgr2=0x7fca740b6010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:19.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:19.367+0000 7fca78aa5640 1 -- 172.21.15.3:0/2069434706 shutdown_connections 2024-05-09T06:36:19.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:19.367+0000 7fca78aa5640 1 -- 172.21.15.3:0/2069434706 wait complete. 2024-05-09T06:36:19.359 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-09T06:36:19.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:19.484+0000 7f6eadb5c640 1 Processor -- start 2024-05-09T06:36:19.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:19.485+0000 7f6eadb5c640 1 -- start start 2024-05-09T06:36:19.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:19.485+0000 7f6eadb5c640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6ea80afb00 0x7f6ea80cd240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:19.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:19.485+0000 7f6eadb5c640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f6ea8141cf0 0x7f6ea8160330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:19.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:19.485+0000 7f6eadb5c640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6ea8142650 con 0x7f6ea8141890 2024-05-09T06:36:19.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:19.485+0000 7f6eadb5c640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f6ea80cee90 con 0x7f6ea8141cf0 2024-05-09T06:36:19.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:19.485+0000 7f6eadb5c640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f6ea80cf130 con 0x7f6ea80afb00 2024-05-09T06:36:19.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:19.486+0000 7f6ea7fff640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f6ea8141cf0 0x7f6ea8160330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:19.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:19.486+0000 7f6ea6ffd640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f6ea8141890 0x7f6ea815fc20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:49340/0 (socket says 172.21.15.3:49340) 2024-05-09T06:36:19.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:19.486+0000 7f6ea6ffd640 1 -- 172.21.15.3:0/2875357779 learned_addr learned my addr 172.21.15.3:0/2875357779 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-05-09T06:36:19.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:19.486+0000 7f6ea77fe640 1 --2- 172.21.15.3:0/2875357779 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6ea80afb00 0x7f6ea80cd240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:19.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:19.486+0000 7f6ea7fff640 1 -- 172.21.15.3:0/2875357779 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6ea80afb00 msgr2=0x7f6ea80cd240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:19.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:19.486+0000 7f6ea7fff640 1 --2- 172.21.15.3:0/2875357779 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6ea80afb00 0x7f6ea80cd240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:19.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:19.486+0000 7f6ea7fff640 1 -- 172.21.15.3:0/2875357779 >> v1:172.21.15.3:6789/0 conn(0x7f6ea8141890 legacy=0x7f6ea815fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:19.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:19.486+0000 7f6ea7fff640 1 -- 172.21.15.3:0/2875357779 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ea80cf410 con 0x7f6ea8141cf0 2024-05-09T06:36:19.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:19.486+0000 7f6ea67fc640 1 -- 172.21.15.3:0/2875357779 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3166106450 0 0) 0x7f6ea8142650 con 0x7f6ea8141890 2024-05-09T06:36:19.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:19.486+0000 7f6ea77fe640 1 --2- 172.21.15.3:0/2875357779 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6ea80afb00 0x7f6ea80cd240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:36:19.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:19.487+0000 7f6ea7fff640 1 --2- 172.21.15.3:0/2875357779 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6ea8141cf0 0x7f6ea8160330 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f6ea0009d90 tx=0x7f6ea002a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3211cd7b154c0d36 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:19.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:19.487+0000 7f6ea67fc640 1 -- 172.21.15.3:0/2875357779 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ea0031600 con 0x7f6ea8141cf0 2024-05-09T06:36:19.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:19.487+0000 7f6ea67fc640 1 -- 172.21.15.3:0/2875357779 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ea00317a0 con 0x7f6ea8141cf0 2024-05-09T06:36:19.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:19.487+0000 7f6ea67fc640 1 -- 172.21.15.3:0/2875357779 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ea0031b70 con 0x7f6ea8141cf0 2024-05-09T06:36:19.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:19.487+0000 7f6eadb5c640 1 -- 172.21.15.3:0/2875357779 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6ea8141cf0 msgr2=0x7f6ea8160330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:19.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:19.487+0000 7f6eadb5c640 1 --2- 172.21.15.3:0/2875357779 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6ea8141cf0 0x7f6ea8160330 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f6ea0009d90 tx=0x7f6ea002a5d0 comp rx=0 tx=0).stop 2024-05-09T06:36:19.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:19.487+0000 7f6eadb5c640 1 -- 172.21.15.3:0/2875357779 shutdown_connections 2024-05-09T06:36:19.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:19.487+0000 7f6eadb5c640 1 --2- 172.21.15.3:0/2875357779 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6ea8141cf0 0x7f6ea8160330 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:19.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:19.487+0000 7f6eadb5c640 1 --2- 172.21.15.3:0/2875357779 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6ea80afb00 0x7f6ea80cd240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:19.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:19.487+0000 7f6eadb5c640 1 -- 172.21.15.3:0/2875357779 >> 172.21.15.3:0/2875357779 conn(0x7f6ea80d07b0 msgr2=0x7f6ea80d0bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:19.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:19.487+0000 7f6eadb5c640 1 -- 172.21.15.3:0/2875357779 shutdown_connections 2024-05-09T06:36:19.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:19.487+0000 7f6eadb5c640 1 -- 172.21.15.3:0/2875357779 wait complete. 2024-05-09T06:36:19.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:19.489+0000 7f6eadb5c640 1 Processor -- start 2024-05-09T06:36:19.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:19.489+0000 7f6eadb5c640 1 -- start start 2024-05-09T06:36:19.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:19.489+0000 7f6eadb5c640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6ea80afb00 0x7f6ea81339a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:19.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:19.489+0000 7f6eadb5c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6ea8141890 0x7f6ea80b39e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:19.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:19.489+0000 7f6eadb5c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6ea8141cf0 0x7f6ea8143cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:19.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:19.489+0000 7f6eadb5c640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f6ea80cfcb0 con 0x7f6ea8141890 2024-05-09T06:36:19.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:19.489+0000 7f6eadb5c640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f6ea80cf9d0 con 0x7f6ea8141cf0 2024-05-09T06:36:19.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:19.489+0000 7f6eadb5c640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f6ea8142100 con 0x7f6ea80afb00 2024-05-09T06:36:19.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:19.490+0000 7f6ea6ffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6ea8141890 0x7f6ea80b39e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:19.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:19.490+0000 7f6ea6ffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6ea8141890 0x7f6ea80b39e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49958/0 (socket says 172.21.15.3:49958) 2024-05-09T06:36:19.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:19.490+0000 7f6ea6ffd640 1 -- 172.21.15.3:0/3159457248 learned_addr learned my addr 172.21.15.3:0/3159457248 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:36:19.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:19.490+0000 7f6ea77fe640 1 --2- 172.21.15.3:0/3159457248 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6ea80afb00 0x7f6ea81339a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:19.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:19.490+0000 7f6ea7fff640 1 --2- 172.21.15.3:0/3159457248 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6ea8141cf0 0x7f6ea8143cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:19.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:19.490+0000 7f6ea6ffd640 1 -- 172.21.15.3:0/3159457248 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6ea80afb00 msgr2=0x7f6ea81339a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:19.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:19.490+0000 7f6ea6ffd640 1 --2- 172.21.15.3:0/3159457248 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6ea80afb00 0x7f6ea81339a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:19.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:19.490+0000 7f6ea6ffd640 1 -- 172.21.15.3:0/3159457248 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6ea8141cf0 msgr2=0x7f6ea8143cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:19.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:19.490+0000 7f6ea6ffd640 1 --2- 172.21.15.3:0/3159457248 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6ea8141cf0 0x7f6ea8143cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:19.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:19.490+0000 7f6ea6ffd640 1 -- 172.21.15.3:0/3159457248 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ea8134030 con 0x7f6ea8141890 2024-05-09T06:36:19.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:19.490+0000 7f6ea77fe640 1 --2- 172.21.15.3:0/3159457248 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6ea80afb00 0x7f6ea81339a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:36:19.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:19.490+0000 7f6ea6ffd640 1 --2- 172.21.15.3:0/3159457248 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6ea8141890 0x7f6ea80b39e0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f6e9c00c4d0 tx=0x7f6e9c00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:19.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:19.490+0000 7f6e8ffff640 1 -- 172.21.15.3:0/3159457248 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e9c018070 con 0x7f6ea8141890 2024-05-09T06:36:19.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:19.490+0000 7f6e8ffff640 1 -- 172.21.15.3:0/3159457248 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e9c007500 con 0x7f6ea8141890 2024-05-09T06:36:19.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:19.490+0000 7f6eadb5c640 1 -- 172.21.15.3:0/3159457248 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ea8134310 con 0x7f6ea8141890 2024-05-09T06:36:19.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:19.490+0000 7f6e8ffff640 1 -- 172.21.15.3:0/3159457248 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e9c004a20 con 0x7f6ea8141890 2024-05-09T06:36:19.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:19.490+0000 7f6eadb5c640 1 -- 172.21.15.3:0/3159457248 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ea81348d0 con 0x7f6ea8141890 2024-05-09T06:36:19.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:19.491+0000 7f6eadb5c640 1 -- 172.21.15.3:0/3159457248 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ea8131c00 con 0x7f6ea8141890 2024-05-09T06:36:19.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:19.492+0000 7f6e8ffff640 1 -- 172.21.15.3:0/3159457248 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6e9c01b070 con 0x7f6ea8141890 2024-05-09T06:36:19.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:19.492+0000 7f6e8ffff640 1 --2- 172.21.15.3:0/3159457248 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f6e9406be30 0x7f6e9406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:19.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:19.492+0000 7f6e8ffff640 1 -- 172.21.15.3:0/3159457248 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f6e9c08a1e0 con 0x7f6ea8141890 2024-05-09T06:36:19.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:19.492+0000 7f6ea77fe640 1 --2- 172.21.15.3:0/3159457248 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f6e9406be30 0x7f6e9406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:19.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:19.493+0000 7f6ea77fe640 1 --2- 172.21.15.3:0/3159457248 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f6e9406be30 0x7f6e9406e2f0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f6e9800aa30 tx=0x7f6e98008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:19.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:19.497+0000 7f6e8ffff640 1 -- 172.21.15.3:0/3159457248 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f6e9c058b90 con 0x7f6ea8141890 2024-05-09T06:36:19.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:19.680+0000 7f6eadb5c640 1 -- 172.21.15.3:0/3159457248 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f6ea8069550 con 0x7f6ea8141890 2024-05-09T06:36:19.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:19.680+0000 7f6e8ffff640 1 -- 172.21.15.3:0/3159457248 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7f6e9c058490 con 0x7f6ea8141890 2024-05-09T06:36:19.662 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 32 2024-05-09T06:36:19.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:19.682+0000 7f6eadb5c640 1 -- 172.21.15.3:0/3159457248 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f6e9406be30 msgr2=0x7f6e9406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:19.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:19.682+0000 7f6eadb5c640 1 --2- 172.21.15.3:0/3159457248 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f6e9406be30 0x7f6e9406e2f0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f6e9800aa30 tx=0x7f6e98008040 comp rx=0 tx=0).stop 2024-05-09T06:36:19.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:19.682+0000 7f6eadb5c640 1 -- 172.21.15.3:0/3159457248 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6ea8141890 msgr2=0x7f6ea80b39e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:19.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:19.682+0000 7f6eadb5c640 1 --2- 172.21.15.3:0/3159457248 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6ea8141890 0x7f6ea80b39e0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f6e9c00c4d0 tx=0x7f6e9c00c990 comp rx=0 tx=0).stop 2024-05-09T06:36:19.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:19.683+0000 7f6eadb5c640 1 -- 172.21.15.3:0/3159457248 shutdown_connections 2024-05-09T06:36:19.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:19.683+0000 7f6eadb5c640 1 --2- 172.21.15.3:0/3159457248 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f6e9406be30 0x7f6e9406e2f0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:19.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:19.683+0000 7f6eadb5c640 1 --2- 172.21.15.3:0/3159457248 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6ea8141cf0 0x7f6ea8143cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:19.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:19.683+0000 7f6eadb5c640 1 --2- 172.21.15.3:0/3159457248 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6ea8141890 0x7f6ea80b39e0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:19.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:19.683+0000 7f6eadb5c640 1 --2- 172.21.15.3:0/3159457248 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6ea80afb00 0x7f6ea81339a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:19.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:19.683+0000 7f6eadb5c640 1 -- 172.21.15.3:0/3159457248 >> 172.21.15.3:0/3159457248 conn(0x7f6ea80d07b0 msgr2=0x7f6ea80d0b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:19.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:19.683+0000 7f6eadb5c640 1 -- 172.21.15.3:0/3159457248 shutdown_connections 2024-05-09T06:36:19.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:19.683+0000 7f6eadb5c640 1 -- 172.21.15.3:0/3159457248 wait complete. 2024-05-09T06:36:19.675 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-09T06:36:19.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:19.801+0000 7f3432da9640 1 Processor -- start 2024-05-09T06:36:19.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:19.801+0000 7f3432da9640 1 -- start start 2024-05-09T06:36:19.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:19.802+0000 7f3432da9640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f342c0cfbd0 0x7f342c0cd940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:19.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:19.802+0000 7f3432da9640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f342c0d05c0 0x7f342c0d0a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:19.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:19.802+0000 7f3432da9640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f342c0d0200 con 0x7f342c15c310 2024-05-09T06:36:19.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:19.802+0000 7f3432da9640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f342c0d0f70 con 0x7f342c0cfbd0 2024-05-09T06:36:19.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:19.802+0000 7f3432da9640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f342c0cce60 con 0x7f342c0d05c0 2024-05-09T06:36:19.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:19.802+0000 7f342bfff640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f342c0cfbd0 0x7f342c0cd940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:19.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:19.802+0000 7f342bfff640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f342c0cfbd0 0x7f342c0cd940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49974/0 (socket says 172.21.15.3:49974) 2024-05-09T06:36:19.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:19.802+0000 7f342bfff640 1 -- 172.21.15.3:0/4061636137 learned_addr learned my addr 172.21.15.3:0/4061636137 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:36:19.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:19.802+0000 7f3431346640 1 --2- 172.21.15.3:0/4061636137 >> v2:172.21.15.190:3300/0 conn(0x7f342c0d05c0 0x7f342c0d0a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:19.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:19.802+0000 7f342bfff640 1 -- 172.21.15.3:0/4061636137 >> v2:172.21.15.190:3300/0 conn(0x7f342c0d05c0 msgr2=0x7f342c0d0a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:19.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:19.802+0000 7f342bfff640 1 --2- 172.21.15.3:0/4061636137 >> v2:172.21.15.190:3300/0 conn(0x7f342c0d05c0 0x7f342c0d0a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:19.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:19.802+0000 7f342bfff640 1 -- 172.21.15.3:0/4061636137 >> v1:172.21.15.190:6789/0 conn(0x7f342c15c310 legacy=0x7f342c09b3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:19.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:19.803+0000 7f342bfff640 1 -- 172.21.15.3:0/4061636137 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f342c0cd140 con 0x7f342c0cfbd0 2024-05-09T06:36:19.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:19.803+0000 7f3431346640 1 --2- 172.21.15.3:0/4061636137 >> v2:172.21.15.190:3300/0 conn(0x7f342c0d05c0 0x7f342c0d0a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:36:19.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:19.803+0000 7f342bfff640 1 --2- 172.21.15.3:0/4061636137 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f342c0cfbd0 0x7f342c0cd940 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f342c0cfb80 tx=0x7f342002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e8f35ceddef3e4e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:19.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:19.803+0000 7f342b7fe640 1 -- 172.21.15.3:0/4061636137 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3420037070 con 0x7f342c0cfbd0 2024-05-09T06:36:19.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:19.803+0000 7f342b7fe640 1 -- 172.21.15.3:0/4061636137 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3420032440 con 0x7f342c0cfbd0 2024-05-09T06:36:19.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:19.803+0000 7f342b7fe640 1 -- 172.21.15.3:0/4061636137 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3420032810 con 0x7f342c0cfbd0 2024-05-09T06:36:19.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:19.804+0000 7f3432da9640 1 -- 172.21.15.3:0/4061636137 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f342c0cfbd0 msgr2=0x7f342c0cd940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:19.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:19.804+0000 7f3432da9640 1 --2- 172.21.15.3:0/4061636137 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f342c0cfbd0 0x7f342c0cd940 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f342c0cfb80 tx=0x7f342002a1e0 comp rx=0 tx=0).stop 2024-05-09T06:36:19.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:19.804+0000 7f3432da9640 1 -- 172.21.15.3:0/4061636137 shutdown_connections 2024-05-09T06:36:19.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:19.804+0000 7f3432da9640 1 --2- 172.21.15.3:0/4061636137 >> v2:172.21.15.190:3300/0 conn(0x7f342c0d05c0 0x7f342c0d0a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:19.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:19.804+0000 7f3432da9640 1 --2- 172.21.15.3:0/4061636137 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f342c0cfbd0 0x7f342c0cd940 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:19.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:19.804+0000 7f3432da9640 1 -- 172.21.15.3:0/4061636137 >> 172.21.15.3:0/4061636137 conn(0x7f342c0b3af0 msgr2=0x7f342c0b3f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:19.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:19.804+0000 7f3432da9640 1 -- 172.21.15.3:0/4061636137 shutdown_connections 2024-05-09T06:36:19.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:19.804+0000 7f3432da9640 1 -- 172.21.15.3:0/4061636137 wait complete. 2024-05-09T06:36:19.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:19.805+0000 7f3432da9640 1 Processor -- start 2024-05-09T06:36:19.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:19.805+0000 7f3432da9640 1 -- start start 2024-05-09T06:36:19.789 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:19.806+0000 7f3432da9640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f342c0cfbd0 0x7f342c151750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:19.789 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:19.806+0000 7f3432da9640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f342c0d05c0 0x7f342c06cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:19.789 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:19.806+0000 7f3432da9640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f342c15c310 0x7f342c1f7c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:19.789 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:19.806+0000 7f3432da9640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f342c0ccb10 con 0x7f342c15c310 2024-05-09T06:36:19.789 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:19.806+0000 7f3432da9640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f342c15c6f0 con 0x7f342c0cfbd0 2024-05-09T06:36:19.789 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:19.806+0000 7f3432da9640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f342c0cde80 con 0x7f342c0d05c0 2024-05-09T06:36:19.789 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:19.806+0000 7f3431346640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f342c15c310 0x7f342c1f7c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:19.789 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:19.806+0000 7f3431346640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f342c15c310 0x7f342c1f7c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49980/0 (socket says 172.21.15.3:49980) 2024-05-09T06:36:19.789 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:19.806+0000 7f3431346640 1 -- 172.21.15.3:0/3061591157 learned_addr learned my addr 172.21.15.3:0/3061591157 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:36:19.789 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:19.806+0000 7f3431346640 1 -- 172.21.15.3:0/3061591157 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f342c0d05c0 msgr2=0x7f342c06cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:19.790 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:19.806+0000 7f3431346640 1 --2- 172.21.15.3:0/3061591157 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f342c0d05c0 0x7f342c06cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:19.790 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:19.806+0000 7f3431346640 1 -- 172.21.15.3:0/3061591157 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f342c0cfbd0 msgr2=0x7f342c151750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:19.790 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:19.806+0000 7f3431346640 1 --2- 172.21.15.3:0/3061591157 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f342c0cfbd0 0x7f342c151750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:19.790 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:19.806+0000 7f3431346640 1 -- 172.21.15.3:0/3061591157 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f342c15ed40 con 0x7f342c15c310 2024-05-09T06:36:19.790 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:19.807+0000 7f3431346640 1 --2- 172.21.15.3:0/3061591157 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f342c15c310 0x7f342c1f7c80 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f3424007df0 tx=0x7f342400d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:19.790 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:19.807+0000 7f34297fa640 1 -- 172.21.15.3:0/3061591157 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f342401a070 con 0x7f342c15c310 2024-05-09T06:36:19.790 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:19.807+0000 7f34297fa640 1 -- 172.21.15.3:0/3061591157 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3424003990 con 0x7f342c15c310 2024-05-09T06:36:19.790 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:19.807+0000 7f3432da9640 1 -- 172.21.15.3:0/3061591157 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f342c15f020 con 0x7f342c15c310 2024-05-09T06:36:19.790 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:19.807+0000 7f34297fa640 1 -- 172.21.15.3:0/3061591157 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3424003d80 con 0x7f342c15c310 2024-05-09T06:36:19.790 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:19.807+0000 7f3432da9640 1 -- 172.21.15.3:0/3061591157 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f342c0c9550 con 0x7f342c15c310 2024-05-09T06:36:19.790 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:19.808+0000 7f3432da9640 1 -- 172.21.15.3:0/3061591157 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f342c06b710 con 0x7f342c15c310 2024-05-09T06:36:19.790 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:19.808+0000 7f34297fa640 1 -- 172.21.15.3:0/3061591157 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f342401d070 con 0x7f342c15c310 2024-05-09T06:36:19.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:19.809+0000 7f34297fa640 1 --2- 172.21.15.3:0/3061591157 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f341806be60 0x7f341806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:19.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:19.809+0000 7f34297fa640 1 -- 172.21.15.3:0/3061591157 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f342408b0f0 con 0x7f342c15c310 2024-05-09T06:36:19.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:19.809+0000 7f3430b45640 1 --2- 172.21.15.3:0/3061591157 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f341806be60 0x7f341806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:19.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:19.810+0000 7f3430b45640 1 --2- 172.21.15.3:0/3061591157 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f341806be60 0x7f341806e320 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f341c000f60 tx=0x7f341c0079b0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:19.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:19.813+0000 7f34297fa640 1 -- 172.21.15.3:0/3061591157 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f342c06b710 con 0x7f342c15c310 2024-05-09T06:36:19.979 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:19.998+0000 7f3432da9640 1 -- 172.21.15.3:0/3061591157 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f342c069550 con 0x7f342c15c310 2024-05-09T06:36:19.979 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:19.998+0000 7f34297fa640 1 -- 172.21.15.3:0/3061591157 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7f3424010070 con 0x7f342c15c310 2024-05-09T06:36:19.979 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 64 2024-05-09T06:36:19.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:20.000+0000 7f3432da9640 1 -- 172.21.15.3:0/3061591157 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f341806be60 msgr2=0x7f341806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:19.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:20.000+0000 7f3432da9640 1 --2- 172.21.15.3:0/3061591157 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f341806be60 0x7f341806e320 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f341c000f60 tx=0x7f341c0079b0 comp rx=0 tx=0).stop 2024-05-09T06:36:19.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:20.000+0000 7f3432da9640 1 -- 172.21.15.3:0/3061591157 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f342c15c310 msgr2=0x7f342c1f7c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:19.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:20.000+0000 7f3432da9640 1 --2- 172.21.15.3:0/3061591157 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f342c15c310 0x7f342c1f7c80 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f3424007df0 tx=0x7f342400d4a0 comp rx=0 tx=0).stop 2024-05-09T06:36:19.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:20.000+0000 7f3432da9640 1 -- 172.21.15.3:0/3061591157 shutdown_connections 2024-05-09T06:36:19.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:20.000+0000 7f3432da9640 1 --2- 172.21.15.3:0/3061591157 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f341806be60 0x7f341806e320 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:19.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:20.000+0000 7f3432da9640 1 --2- 172.21.15.3:0/3061591157 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f342c15c310 0x7f342c1f7c80 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:19.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:20.000+0000 7f3432da9640 1 --2- 172.21.15.3:0/3061591157 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f342c0d05c0 0x7f342c06cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:19.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:20.000+0000 7f3432da9640 1 --2- 172.21.15.3:0/3061591157 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f342c0cfbd0 0x7f342c151750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:19.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:20.000+0000 7f3432da9640 1 -- 172.21.15.3:0/3061591157 >> 172.21.15.3:0/3061591157 conn(0x7f342c0b3af0 msgr2=0x7f342c0c7c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:19.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:20.000+0000 7f3432da9640 1 -- 172.21.15.3:0/3061591157 shutdown_connections 2024-05-09T06:36:19.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:20.000+0000 7f3432da9640 1 -- 172.21.15.3:0/3061591157 wait complete. 2024-05-09T06:36:19.992 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T06:36:20.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:20.115+0000 7f1d041a9640 1 Processor -- start 2024-05-09T06:36:20.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:20.116+0000 7f1d041a9640 1 -- start start 2024-05-09T06:36:20.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:20.116+0000 7f1d041a9640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f1cfc15e480 0x7f1cfc0d1dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:20.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:20.116+0000 7f1d041a9640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1cfc0d4e50 con 0x7f1cfc0d2310 2024-05-09T06:36:20.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:20.116+0000 7f1d041a9640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1cfc0d0270 con 0x7f1cfc0d2ed0 2024-05-09T06:36:20.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:20.116+0000 7f1d041a9640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f1cfc0cf1d0 con 0x7f1cfc15e480 2024-05-09T06:36:20.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:20.117+0000 7f1d01f45640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f1cfc15e480 0x7f1cfc0d1dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:20.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:20.117+0000 7f1d01f45640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f1cfc15e480 0x7f1cfc0d1dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49992/0 (socket says 172.21.15.3:49992) 2024-05-09T06:36:20.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:20.117+0000 7f1d01744640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f1cfc0d2310 0x7f1cfc0d27c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:49342/0 (socket says 172.21.15.3:49342) 2024-05-09T06:36:20.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:20.117+0000 7f1d01f45640 1 -- 172.21.15.3:0/1793343495 learned_addr learned my addr 172.21.15.3:0/1793343495 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:36:20.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:20.117+0000 7f1d01f45640 1 -- 172.21.15.3:0/1793343495 >> v1:172.21.15.190:6789/0 conn(0x7f1cfc0d2ed0 legacy=0x7f1cfc1683a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:20.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:20.117+0000 7f1d01f45640 1 -- 172.21.15.3:0/1793343495 >> v1:172.21.15.3:6789/0 conn(0x7f1cfc0d2310 legacy=0x7f1cfc0d27c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:20.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:20.117+0000 7f1d01f45640 1 -- 172.21.15.3:0/1793343495 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cfc0cf4b0 con 0x7f1cfc15e480 2024-05-09T06:36:20.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:20.117+0000 7f1d00f43640 1 -- 172.21.15.3:0/1793343495 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1080738193 0 0) 0x7f1cf0001110 con 0x7f1cfc0d2310 2024-05-09T06:36:20.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:20.118+0000 7f1d01f45640 1 --2- 172.21.15.3:0/1793343495 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1cfc15e480 0x7f1cfc0d1dd0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f1cf4009870 tx=0x7f1cf402a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de2a0144b6ce2152 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:20.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:20.118+0000 7f1d00f43640 1 -- 172.21.15.3:0/1793343495 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cf4037070 con 0x7f1cfc15e480 2024-05-09T06:36:20.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:20.118+0000 7f1d00f43640 1 -- 172.21.15.3:0/1793343495 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1cf402ae50 con 0x7f1cfc15e480 2024-05-09T06:36:20.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:20.118+0000 7f1d00f43640 1 -- 172.21.15.3:0/1793343495 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cf4032860 con 0x7f1cfc15e480 2024-05-09T06:36:20.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:20.118+0000 7f1d041a9640 1 -- 172.21.15.3:0/1793343495 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1cfc15e480 msgr2=0x7f1cfc0d1dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:20.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:20.118+0000 7f1d041a9640 1 --2- 172.21.15.3:0/1793343495 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1cfc15e480 0x7f1cfc0d1dd0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f1cf4009870 tx=0x7f1cf402a300 comp rx=0 tx=0).stop 2024-05-09T06:36:20.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:20.118+0000 7f1d041a9640 1 -- 172.21.15.3:0/1793343495 shutdown_connections 2024-05-09T06:36:20.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:20.118+0000 7f1d041a9640 1 --2- 172.21.15.3:0/1793343495 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1cfc15e480 0x7f1cfc0d1dd0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:20.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:20.118+0000 7f1d041a9640 1 -- 172.21.15.3:0/1793343495 >> 172.21.15.3:0/1793343495 conn(0x7f1cfc09d4d0 msgr2=0x7f1cfc0d4200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:20.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:20.118+0000 7f1d041a9640 1 -- 172.21.15.3:0/1793343495 shutdown_connections 2024-05-09T06:36:20.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:20.118+0000 7f1d041a9640 1 -- 172.21.15.3:0/1793343495 wait complete. 2024-05-09T06:36:20.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:20.120+0000 7f1d041a9640 1 Processor -- start 2024-05-09T06:36:20.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:20.120+0000 7f1d041a9640 1 -- start start 2024-05-09T06:36:20.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:20.120+0000 7f1d041a9640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1cfc0d2310 0x7f1cfc1f7bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:20.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:20.120+0000 7f1d041a9640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1cfc0d2ed0 0x7f1cfc152df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:20.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:20.120+0000 7f1d041a9640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1cfc15e480 0x7f1cfc1f3d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:20.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:20.120+0000 7f1d041a9640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f1cfc0cfd50 con 0x7f1cfc0d2ed0 2024-05-09T06:36:20.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:20.120+0000 7f1d041a9640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f1cfc0cfa70 con 0x7f1cfc0d2310 2024-05-09T06:36:20.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:20.120+0000 7f1d041a9640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f1cfc0b5ab0 con 0x7f1cfc15e480 2024-05-09T06:36:20.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:20.121+0000 7f1d01744640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1cfc0d2ed0 0x7f1cfc152df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:20.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:20.121+0000 7f1d01744640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1cfc0d2ed0 0x7f1cfc152df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50002/0 (socket says 172.21.15.3:50002) 2024-05-09T06:36:20.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:20.121+0000 7f1d01744640 1 -- 172.21.15.3:0/25258975 learned_addr learned my addr 172.21.15.3:0/25258975 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:36:20.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:20.121+0000 7f1d01f45640 1 --2- 172.21.15.3:0/25258975 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1cfc0d2310 0x7f1cfc1f7bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:20.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:20.121+0000 7f1d02746640 1 --2- 172.21.15.3:0/25258975 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1cfc15e480 0x7f1cfc1f3d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:20.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:20.121+0000 7f1d01744640 1 -- 172.21.15.3:0/25258975 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1cfc15e480 msgr2=0x7f1cfc1f3d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:20.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:20.121+0000 7f1d01744640 1 --2- 172.21.15.3:0/25258975 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1cfc15e480 0x7f1cfc1f3d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:20.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:20.121+0000 7f1d01744640 1 -- 172.21.15.3:0/25258975 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1cfc0d2310 msgr2=0x7f1cfc1f7bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:20.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:20.121+0000 7f1d01744640 1 --2- 172.21.15.3:0/25258975 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1cfc0d2310 0x7f1cfc1f7bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:20.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:20.121+0000 7f1d01744640 1 -- 172.21.15.3:0/25258975 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cfc0c9760 con 0x7f1cfc0d2ed0 2024-05-09T06:36:20.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:20.121+0000 7f1d01744640 1 --2- 172.21.15.3:0/25258975 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1cfc0d2ed0 0x7f1cfc152df0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f1cf000c3c0 tx=0x7f1cf000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:20.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:20.121+0000 7f1cfa7fc640 1 -- 172.21.15.3:0/25258975 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cf00074e0 con 0x7f1cfc0d2ed0 2024-05-09T06:36:20.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:20.121+0000 7f1cfa7fc640 1 -- 172.21.15.3:0/25258975 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1cf00047e0 con 0x7f1cfc0d2ed0 2024-05-09T06:36:20.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:20.121+0000 7f1cfa7fc640 1 -- 172.21.15.3:0/25258975 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cf0004bd0 con 0x7f1cfc0d2ed0 2024-05-09T06:36:20.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:20.121+0000 7f1d041a9640 1 -- 172.21.15.3:0/25258975 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1cfc1f81b0 con 0x7f1cfc0d2ed0 2024-05-09T06:36:20.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:20.121+0000 7f1d041a9640 1 -- 172.21.15.3:0/25258975 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1cfc1f8770 con 0x7f1cfc0d2ed0 2024-05-09T06:36:20.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:20.122+0000 7f1d041a9640 1 -- 172.21.15.3:0/25258975 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1cfc04f660 con 0x7f1cfc0d2ed0 2024-05-09T06:36:20.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:20.123+0000 7f1cfa7fc640 1 -- 172.21.15.3:0/25258975 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1cf0011070 con 0x7f1cfc0d2ed0 2024-05-09T06:36:20.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:20.123+0000 7f1cfa7fc640 1 --2- 172.21.15.3:0/25258975 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f1ce806be30 0x7f1ce806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:20.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:20.124+0000 7f1cfa7fc640 1 -- 172.21.15.3:0/25258975 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f1cf008a9b0 con 0x7f1cfc0d2ed0 2024-05-09T06:36:20.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:20.124+0000 7f1d01f45640 1 --2- 172.21.15.3:0/25258975 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f1ce806be30 0x7f1ce806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:20.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:20.124+0000 7f1d01f45640 1 --2- 172.21.15.3:0/25258975 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f1ce806be30 0x7f1ce806e2f0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f1cf4034000 tx=0x7f1cf402e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:20.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:20.128+0000 7f1cfa7fc640 1 -- 172.21.15.3:0/25258975 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f1cfc0d4e50 con 0x7f1cfc0d2ed0 2024-05-09T06:36:20.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:20.348+0000 7f1d041a9640 1 -- 172.21.15.3:0/25258975 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1cfc0ba080 con 0x7f1cfc0d2ed0 2024-05-09T06:36:20.330 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:20.349+0000 7f1cfa7fc640 1 -- 172.21.15.3:0/25258975 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5172 (secure 0 0 0) 0x7f1cf0058c80 con 0x7f1cfc0d2ed0 2024-05-09T06:36:20.330 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-09T06:36:20.330 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4272,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6834/126273055","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":126273055},{"type":"v1","addr":"172.21.15.3:6834","nonce":126273055}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4273,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6836/3885692063","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":3885692063},{"type":"v1","addr":"172.21.15.190:6836","nonce":3885692063}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4275,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/3461228756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6833","nonce":3461228756},{"type":"v1","addr":"172.21.15.3:6835","nonce":3461228756}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-09T06:36:07.221126+0000","modified":"2024-05-09T06:36:10.231051+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4276},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4276":{"gid":4276,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.190:6837/4111472052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6835","nonce":4111472052},{"type":"v1","addr":"172.21.15.190:6837","nonce":4111472052}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4276,"qdb_cluster":[4276]},"id":1}]} 2024-05-09T06:36:20.331 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 8 2024-05-09T06:36:20.332 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:20.351+0000 7f1d041a9640 1 -- 172.21.15.3:0/25258975 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f1ce806be30 msgr2=0x7f1ce806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:20.332 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:20.351+0000 7f1d041a9640 1 --2- 172.21.15.3:0/25258975 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f1ce806be30 0x7f1ce806e2f0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f1cf4034000 tx=0x7f1cf402e000 comp rx=0 tx=0).stop 2024-05-09T06:36:20.332 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:20.351+0000 7f1d041a9640 1 -- 172.21.15.3:0/25258975 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1cfc0d2ed0 msgr2=0x7f1cfc152df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:20.332 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:20.351+0000 7f1d041a9640 1 --2- 172.21.15.3:0/25258975 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1cfc0d2ed0 0x7f1cfc152df0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f1cf000c3c0 tx=0x7f1cf000c880 comp rx=0 tx=0).stop 2024-05-09T06:36:20.332 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:20.351+0000 7f1d041a9640 1 -- 172.21.15.3:0/25258975 shutdown_connections 2024-05-09T06:36:20.332 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:20.351+0000 7f1d041a9640 1 --2- 172.21.15.3:0/25258975 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f1ce806be30 0x7f1ce806e2f0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:20.332 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:20.351+0000 7f1d041a9640 1 --2- 172.21.15.3:0/25258975 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1cfc15e480 0x7f1cfc1f3d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:20.332 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:20.351+0000 7f1d041a9640 1 --2- 172.21.15.3:0/25258975 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1cfc0d2ed0 0x7f1cfc152df0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:20.332 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:20.351+0000 7f1d041a9640 1 --2- 172.21.15.3:0/25258975 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1cfc0d2310 0x7f1cfc1f7bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:20.332 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:20.351+0000 7f1d041a9640 1 -- 172.21.15.3:0/25258975 >> 172.21.15.3:0/25258975 conn(0x7f1cfc09d4d0 msgr2=0x7f1cfc0b8e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:20.333 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:20.351+0000 7f1d041a9640 1 -- 172.21.15.3:0/25258975 shutdown_connections 2024-05-09T06:36:20.333 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:20.351+0000 7f1d041a9640 1 -- 172.21.15.3:0/25258975 wait complete. 2024-05-09T06:36:20.343 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T06:36:20.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:20.467+0000 7f20407c8640 1 Processor -- start 2024-05-09T06:36:20.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:20.467+0000 7f20407c8640 1 -- start start 2024-05-09T06:36:20.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:20.468+0000 7f20407c8640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f203813e0c0 0x7f20380cf720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:20.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:20.468+0000 7f20407c8640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f203813e4a0 con 0x7f20381408f0 2024-05-09T06:36:20.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:20.468+0000 7f20407c8640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f20380ce960 con 0x7f20380cc870 2024-05-09T06:36:20.450 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:20.468+0000 7f20407c8640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f20380cec40 con 0x7f203813e0c0 2024-05-09T06:36:20.450 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:20.468+0000 7f203ed65640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f20381408f0 0x7f2038140d40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:49346/0 (socket says 172.21.15.3:49346) 2024-05-09T06:36:20.450 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:20.468+0000 7f203ed65640 1 -- 172.21.15.3:0/1863858747 learned_addr learned my addr 172.21.15.3:0/1863858747 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-05-09T06:36:20.450 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:20.468+0000 7f203e564640 1 --2- 172.21.15.3:0/1863858747 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f203813e0c0 0x7f20380cf720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:20.450 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:20.468+0000 7f203d562640 1 -- 172.21.15.3:0/1863858747 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 61680730 0 0) 0x7f2030001110 con 0x7f20381408f0 2024-05-09T06:36:20.450 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:20.468+0000 7f203d562640 1 -- 172.21.15.3:0/1863858747 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2024003520 con 0x7f20381408f0 2024-05-09T06:36:20.450 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:20.469+0000 7f203d562640 1 -- 172.21.15.3:0/1863858747 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3714135253 0 0) 0x7f20380ce960 con 0x7f20380cc870 2024-05-09T06:36:20.450 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:20.469+0000 7f203e564640 1 -- 172.21.15.3:0/1863858747 >> v1:172.21.15.190:6789/0 conn(0x7f20380cc870 legacy=0x7f20380cccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:20.450 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:20.469+0000 7f203e564640 1 -- 172.21.15.3:0/1863858747 >> v1:172.21.15.3:6789/0 conn(0x7f20381408f0 legacy=0x7f2038140d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:20.450 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:20.469+0000 7f203e564640 1 -- 172.21.15.3:0/1863858747 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20380cef20 con 0x7f203813e0c0 2024-05-09T06:36:20.452 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:20.469+0000 7f203e564640 1 --2- 172.21.15.3:0/1863858747 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f203813e0c0 0x7f20380cf720 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f2034009dc0 tx=0x7f203402a640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3d603b7e7c43b052 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:20.452 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:20.470+0000 7f203d562640 1 -- 172.21.15.3:0/1863858747 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2034035020 con 0x7f203813e0c0 2024-05-09T06:36:20.452 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:20.470+0000 7f203d562640 1 -- 172.21.15.3:0/1863858747 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2034007540 con 0x7f203813e0c0 2024-05-09T06:36:20.452 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:20.470+0000 7f203d562640 1 -- 172.21.15.3:0/1863858747 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2030001840 con 0x7f203813e0c0 2024-05-09T06:36:20.453 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:20.470+0000 7f20407c8640 1 -- 172.21.15.3:0/1863858747 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f203813e0c0 msgr2=0x7f20380cf720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:20.453 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:20.470+0000 7f20407c8640 1 --2- 172.21.15.3:0/1863858747 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f203813e0c0 0x7f20380cf720 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f2034009dc0 tx=0x7f203402a640 comp rx=0 tx=0).stop 2024-05-09T06:36:20.453 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:20.470+0000 7f20407c8640 1 -- 172.21.15.3:0/1863858747 shutdown_connections 2024-05-09T06:36:20.453 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:20.470+0000 7f20407c8640 1 --2- 172.21.15.3:0/1863858747 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f203813e0c0 0x7f20380cf720 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:20.453 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:20.470+0000 7f20407c8640 1 -- 172.21.15.3:0/1863858747 >> 172.21.15.3:0/1863858747 conn(0x7f203806b430 msgr2=0x7f203806b810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:20.453 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:20.470+0000 7f20407c8640 1 -- 172.21.15.3:0/1863858747 shutdown_connections 2024-05-09T06:36:20.453 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:20.470+0000 7f20407c8640 1 -- 172.21.15.3:0/1863858747 wait complete. 2024-05-09T06:36:20.453 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:20.471+0000 7f20407c8640 1 Processor -- start 2024-05-09T06:36:20.453 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:20.471+0000 7f20407c8640 1 -- start start 2024-05-09T06:36:20.455 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:20.472+0000 7f20407c8640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f20380cc870 0x7f203813d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:20.455 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:20.472+0000 7f20407c8640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f203813e0c0 0x7f2038154e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:20.455 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:20.472+0000 7f20407c8640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f20381408f0 0x7f20381553a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:20.456 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:20.472+0000 7f20407c8640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f2038071080 con 0x7f20381408f0 2024-05-09T06:36:20.456 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:20.472+0000 7f20407c8640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f2038141660 con 0x7f203813e0c0 2024-05-09T06:36:20.456 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:20.472+0000 7f20407c8640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f2038071200 con 0x7f20380cc870 2024-05-09T06:36:20.456 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:20.472+0000 7f203ed65640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f20381408f0 0x7f20381553a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:20.456 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:20.472+0000 7f203ed65640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f20381408f0 0x7f20381553a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50012/0 (socket says 172.21.15.3:50012) 2024-05-09T06:36:20.456 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:20.472+0000 7f203ed65640 1 -- 172.21.15.3:0/3739429118 learned_addr learned my addr 172.21.15.3:0/3739429118 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:36:20.456 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:20.473+0000 7f203e564640 1 --2- 172.21.15.3:0/3739429118 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f20380cc870 0x7f203813d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:20.456 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:20.473+0000 7f203dd63640 1 --2- 172.21.15.3:0/3739429118 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f203813e0c0 0x7f2038154e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:20.456 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:20.473+0000 7f203ed65640 1 -- 172.21.15.3:0/3739429118 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f20380cc870 msgr2=0x7f203813d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:20.456 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:20.473+0000 7f203ed65640 1 --2- 172.21.15.3:0/3739429118 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f20380cc870 0x7f203813d600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:20.456 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:20.473+0000 7f203ed65640 1 -- 172.21.15.3:0/3739429118 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f203813e0c0 msgr2=0x7f2038154e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:20.456 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:20.473+0000 7f203ed65640 1 --2- 172.21.15.3:0/3739429118 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f203813e0c0 0x7f2038154e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:20.456 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:20.473+0000 7f203ed65640 1 -- 172.21.15.3:0/3739429118 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f203813db40 con 0x7f20381408f0 2024-05-09T06:36:20.456 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:20.473+0000 7f203ed65640 1 --2- 172.21.15.3:0/3739429118 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f20381408f0 0x7f20381553a0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f20300013b0 tx=0x7f203000cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:20.456 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:20.473+0000 7f202effd640 1 -- 172.21.15.3:0/3739429118 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2030018070 con 0x7f20381408f0 2024-05-09T06:36:20.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:20.473+0000 7f202effd640 1 -- 172.21.15.3:0/3739429118 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f20300072d0 con 0x7f20381408f0 2024-05-09T06:36:20.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:20.473+0000 7f20407c8640 1 -- 172.21.15.3:0/3739429118 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f203813ea70 con 0x7f20381408f0 2024-05-09T06:36:20.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:20.473+0000 7f202effd640 1 -- 172.21.15.3:0/3739429118 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20300047e0 con 0x7f20381408f0 2024-05-09T06:36:20.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:20.473+0000 7f20407c8640 1 -- 172.21.15.3:0/3739429118 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f203813f030 con 0x7f20381408f0 2024-05-09T06:36:20.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:20.474+0000 7f20407c8640 1 -- 172.21.15.3:0/3739429118 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f203804f660 con 0x7f20381408f0 2024-05-09T06:36:20.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:20.475+0000 7f202effd640 1 -- 172.21.15.3:0/3739429118 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2030004980 con 0x7f20381408f0 2024-05-09T06:36:20.458 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:20.475+0000 7f202effd640 1 --2- 172.21.15.3:0/3739429118 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f201c06be30 0x7f201c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:20.458 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:20.475+0000 7f202effd640 1 -- 172.21.15.3:0/3739429118 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f203008a310 con 0x7f20381408f0 2024-05-09T06:36:20.458 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:20.475+0000 7f203e564640 1 --2- 172.21.15.3:0/3739429118 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f201c06be30 0x7f201c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:20.458 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:20.476+0000 7f203e564640 1 --2- 172.21.15.3:0/3739429118 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f201c06be30 0x7f201c06e2f0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f2034033000 tx=0x7f203402f000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:20.460 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:20.480+0000 7f202effd640 1 -- 172.21.15.3:0/3739429118 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f203813e4a0 con 0x7f20381408f0 2024-05-09T06:36:20.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:20.666+0000 7f20407c8640 1 -- 172.21.15.3:0/3739429118 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f203815d980 con 0x7f20381408f0 2024-05-09T06:36:20.648 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:20.667+0000 7f202effd640 1 -- 172.21.15.3:0/3739429118 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16521 (secure 0 0 0) 0x7f2030058c40 con 0x7f20381408f0 2024-05-09T06:36:20.648 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-09T06:36:20.648 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":20,"fsid":"16ed737d-2665-431a-8ac6-ef242be3975a","created":"2024-05-09T06:35:42.541368+0000","modified":"2024-05-09T06:36:08.856391+0000","last_up_change":"2024-05-09T06:35:53.778274+0000","last_in_change":"2024-05-09T06:35:44.780586+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T06:35:54.819999+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T06:36:04.931314+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T06:36:06.200433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fae3a8db-4826-4f3c-b33a-12c09bb5df6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":1897849827},{"type":"v1","addr":"172.21.15.3:6817","nonce":1897849827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":1897849827},{"type":"v1","addr":"172.21.15.3:6819","nonce":1897849827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":1897849827},{"type":"v1","addr":"172.21.15.3:6823","nonce":1897849827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":1897849827},{"type":"v1","addr":"172.21.15.3:6821","nonce":1897849827}]},"public_addr":"172.21.15.3:6817/1897849827","cluster_addr":"172.21.15.3:6819/1897849827","heartbeat_back_addr":"172.21.15.3:6823/1897849827","heartbeat_front_addr":"172.21.15.3:6821/1897849827","state":["exists","up"]},{"osd":1,"uuid":"646e882d-4ce4-4720-8c84-f71df563a35e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":926805498},{"type":"v1","addr":"172.21.15.3:6801","nonce":926805498}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":926805498},{"type":"v1","addr":"172.21.15.3:6803","nonce":926805498}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":926805498},{"type":"v1","addr":"172.21.15.3:6807","nonce":926805498}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":926805498},{"type":"v1","addr":"172.21.15.3:6805","nonce":926805498}]},"public_addr":"172.21.15.3:6801/926805498","cluster_addr":"172.21.15.3:6803/926805498","heartbeat_back_addr":"172.21.15.3:6807/926805498","heartbeat_front_addr":"172.21.15.3:6805/926805498","state":["exists","up"]},{"osd":2,"uuid":"1abbadb5-a570-4359-9643-acff3ede35a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":801295451},{"type":"v1","addr":"172.21.15.3:6809","nonce":801295451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":801295451},{"type":"v1","addr":"172.21.15.3:6811","nonce":801295451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":801295451},{"type":"v1","addr":"172.21.15.3:6815","nonce":801295451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":801295451},{"type":"v1","addr":"172.21.15.3:6813","nonce":801295451}]},"public_addr":"172.21.15.3:6809/801295451","cluster_addr":"172.21.15.3:6811/801295451","heartbeat_back_addr":"172.21.15.3:6815/801295451","heartbeat_front_addr":"172.21.15.3:6813/801295451","state":["exists","up"]},{"osd":3,"uuid":"62ac4dc1-2371-4dd2-acd3-e2dbb11aedf5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":1903389857},{"type":"v1","addr":"172.21.15.3:6825","nonce":1903389857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":1903389857},{"type":"v1","addr":"172.21.15.3:6827","nonce":1903389857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":1903389857},{"type":"v1","addr":"172.21.15.3:6831","nonce":1903389857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":1903389857},{"type":"v1","addr":"172.21.15.3:6829","nonce":1903389857}]},"public_addr":"172.21.15.3:6825/1903389857","cluster_addr":"172.21.15.3:6827/1903389857","heartbeat_back_addr":"172.21.15.3:6831/1903389857","heartbeat_front_addr":"172.21.15.3:6829/1903389857","state":["exists","up"]},{"osd":4,"uuid":"ae808398-8326-4d7c-9319-a2a0eafffb59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":2143624576},{"type":"v1","addr":"172.21.15.190:6801","nonce":2143624576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":2143624576},{"type":"v1","addr":"172.21.15.190:6803","nonce":2143624576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":2143624576},{"type":"v1","addr":"172.21.15.190:6807","nonce":2143624576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":2143624576},{"type":"v1","addr":"172.21.15.190:6805","nonce":2143624576}]},"public_addr":"172.21.15.190:6801/2143624576","cluster_addr":"172.21.15.190:6803/2143624576","heartbeat_back_addr":"172.21.15.190:6807/2143624576","heartbeat_front_addr":"172.21.15.190:6805/2143624576","state":["exists","up"]},{"osd":5,"uuid":"918adf50-c440-48ac-9a8d-4837562d7275","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":1766923126},{"type":"v1","addr":"172.21.15.190:6814","nonce":1766923126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":1766923126},{"type":"v1","addr":"172.21.15.190:6818","nonce":1766923126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":1766923126},{"type":"v1","addr":"172.21.15.190:6823","nonce":1766923126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":1766923126},{"type":"v1","addr":"172.21.15.190:6821","nonce":1766923126}]},"public_addr":"172.21.15.190:6814/1766923126","cluster_addr":"172.21.15.190:6818/1766923126","heartbeat_back_addr":"172.21.15.190:6823/1766923126","heartbeat_front_addr":"172.21.15.190:6821/1766923126","state":["exists","up"]},{"osd":6,"uuid":"ac2882bc-2a7c-463c-8f6c-76ff49828adc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":342414248},{"type":"v1","addr":"172.21.15.190:6825","nonce":342414248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":342414248},{"type":"v1","addr":"172.21.15.190:6827","nonce":342414248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":342414248},{"type":"v1","addr":"172.21.15.190:6831","nonce":342414248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":342414248},{"type":"v1","addr":"172.21.15.190:6829","nonce":342414248}]},"public_addr":"172.21.15.190:6825/342414248","cluster_addr":"172.21.15.190:6827/342414248","heartbeat_back_addr":"172.21.15.190:6831/342414248","heartbeat_front_addr":"172.21.15.190:6829/342414248","state":["exists","up"]},{"osd":7,"uuid":"dd4e767a-4730-4c70-8bd5-3f088140a5a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":537849891},{"type":"v1","addr":"172.21.15.190:6809","nonce":537849891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":537849891},{"type":"v1","addr":"172.21.15.190:6811","nonce":537849891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6817","nonce":537849891},{"type":"v1","addr":"172.21.15.190:6819","nonce":537849891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6813","nonce":537849891},{"type":"v1","addr":"172.21.15.190:6815","nonce":537849891}]},"public_addr":"172.21.15.190:6809/537849891","cluster_addr":"172.21.15.190:6811/537849891","heartbeat_back_addr":"172.21.15.190:6819/537849891","heartbeat_front_addr":"172.21.15.190:6815/537849891","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.889784+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.974002+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.973549+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.955396+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.928471+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.902418+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.958696+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.926474+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T06:36:20.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:20.669+0000 7f20407c8640 1 -- 172.21.15.3:0/3739429118 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f201c06be30 msgr2=0x7f201c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:20.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:20.669+0000 7f20407c8640 1 --2- 172.21.15.3:0/3739429118 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f201c06be30 0x7f201c06e2f0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f2034033000 tx=0x7f203402f000 comp rx=0 tx=0).stop 2024-05-09T06:36:20.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:20.669+0000 7f20407c8640 1 -- 172.21.15.3:0/3739429118 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f20381408f0 msgr2=0x7f20381553a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:20.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:20.669+0000 7f20407c8640 1 --2- 172.21.15.3:0/3739429118 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f20381408f0 0x7f20381553a0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f20300013b0 tx=0x7f203000cbe0 comp rx=0 tx=0).stop 2024-05-09T06:36:20.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:20.669+0000 7f20407c8640 1 -- 172.21.15.3:0/3739429118 shutdown_connections 2024-05-09T06:36:20.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:20.669+0000 7f20407c8640 1 --2- 172.21.15.3:0/3739429118 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f201c06be30 0x7f201c06e2f0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:20.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:20.669+0000 7f20407c8640 1 --2- 172.21.15.3:0/3739429118 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f20381408f0 0x7f20381553a0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:20.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:20.669+0000 7f20407c8640 1 --2- 172.21.15.3:0/3739429118 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f203813e0c0 0x7f2038154e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:20.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:20.669+0000 7f20407c8640 1 --2- 172.21.15.3:0/3739429118 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f20380cc870 0x7f203813d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:20.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:20.669+0000 7f20407c8640 1 -- 172.21.15.3:0/3739429118 >> 172.21.15.3:0/3739429118 conn(0x7f203806b430 msgr2=0x7f203815de80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:20.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:20.669+0000 7f20407c8640 1 -- 172.21.15.3:0/3739429118 shutdown_connections 2024-05-09T06:36:20.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:20.669+0000 7f20407c8640 1 -- 172.21.15.3:0/3739429118 wait complete. 2024-05-09T06:36:20.662 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T06:36:20.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:20.785+0000 7f2d24310640 1 Processor -- start 2024-05-09T06:36:20.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:20.785+0000 7f2d24310640 1 -- start start 2024-05-09T06:36:20.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:20.786+0000 7f2d24310640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2d1c15e420 0x7f2d1c06b250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:20.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:20.786+0000 7f2d24310640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f2d1c06b790 0x7f2d1c0b4640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:20.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:20.786+0000 7f2d24310640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2d1c0b5010 con 0x7f2d1c0b4b80 2024-05-09T06:36:20.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:20.786+0000 7f2d24310640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f2d1c06bbf0 con 0x7f2d1c06b790 2024-05-09T06:36:20.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:20.786+0000 7f2d24310640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f2d1c15e800 con 0x7f2d1c15e420 2024-05-09T06:36:20.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:20.786+0000 7f2d228ad640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f2d1c0b4b80 0x7f2d1c06fc80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:47838/0 (socket says 172.21.15.3:47838) 2024-05-09T06:36:20.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:20.786+0000 7f2d228ad640 1 -- 172.21.15.3:0/425623404 learned_addr learned my addr 172.21.15.3:0/425623404 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-05-09T06:36:20.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:20.787+0000 7f2d218ab640 1 --2- 172.21.15.3:0/425623404 >> v2:172.21.15.190:3300/0 conn(0x7f2d1c06b790 0x7f2d1c0b4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:20.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:20.787+0000 7f2d220ac640 1 --2- 172.21.15.3:0/425623404 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2d1c15e420 0x7f2d1c06b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:20.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:20.787+0000 7f2d210aa640 1 -- 172.21.15.3:0/425623404 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2841119791 0 0) 0x7f2d0c001490 con 0x7f2d1c0b4b80 2024-05-09T06:36:20.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:20.787+0000 7f2d210aa640 1 -- 172.21.15.3:0/425623404 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2d04003520 con 0x7f2d1c0b4b80 2024-05-09T06:36:20.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:20.787+0000 7f2d210aa640 1 -- 172.21.15.3:0/425623404 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 779846411 0 0) 0x7f2d04003520 con 0x7f2d1c0b4b80 2024-05-09T06:36:20.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:20.787+0000 7f2d210aa640 1 -- 172.21.15.3:0/425623404 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2d1c15e420 msgr2=0x7f2d1c06b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:20.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:20.787+0000 7f2d210aa640 1 --2- 172.21.15.3:0/425623404 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2d1c15e420 0x7f2d1c06b250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:20.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:20.787+0000 7f2d210aa640 1 -- 172.21.15.3:0/425623404 >> v2:172.21.15.190:3300/0 conn(0x7f2d1c06b790 msgr2=0x7f2d1c0b4640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:20.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:20.787+0000 7f2d210aa640 1 --2- 172.21.15.3:0/425623404 >> v2:172.21.15.190:3300/0 conn(0x7f2d1c06b790 0x7f2d1c0b4640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:20.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:20.787+0000 7f2d210aa640 1 -- 172.21.15.3:0/425623404 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d1c0ae600 con 0x7f2d1c0b4b80 2024-05-09T06:36:20.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:20.787+0000 7f2d210aa640 1 -- 172.21.15.3:0/425623404 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 563312031 0 0) 0x7f2d0c001bc0 con 0x7f2d1c0b4b80 2024-05-09T06:36:20.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:20.787+0000 7f2d210aa640 1 -- 172.21.15.3:0/425623404 >> v1:172.21.15.3:6789/0 conn(0x7f2d1c0b4b80 legacy=0x7f2d1c06fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:20.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:20.787+0000 7f2d210aa640 1 --2- 172.21.15.3:0/425623404 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2d040039b0 0x7f2d04003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:20.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:20.787+0000 7f2d210aa640 1 --2- 172.21.15.3:0/425623404 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2d1c15e420 0x7f2d04004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:20.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:20.787+0000 7f2d210aa640 1 --2- 172.21.15.3:0/425623404 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2d04004d40 0x7f2d040051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:20.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:20.787+0000 7f2d210aa640 1 -- 172.21.15.3:0/425623404 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f2d1c15e800 con 0x7f2d04004d40 2024-05-09T06:36:20.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:20.787+0000 7f2d210aa640 1 -- 172.21.15.3:0/425623404 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f2d0400b9c0 con 0x7f2d040039b0 2024-05-09T06:36:20.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:20.787+0000 7f2d210aa640 1 -- 172.21.15.3:0/425623404 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f2d0400bb40 con 0x7f2d1c15e420 2024-05-09T06:36:20.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:20.787+0000 7f2d228ad640 1 --2- 172.21.15.3:0/425623404 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2d04004d40 0x7f2d040051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:20.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:20.787+0000 7f2d220ac640 1 --2- 172.21.15.3:0/425623404 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2d040039b0 0x7f2d04003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:20.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:20.787+0000 7f2d218ab640 1 --2- 172.21.15.3:0/425623404 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2d1c15e420 0x7f2d04004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:20.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:20.787+0000 7f2d228ad640 1 -- 172.21.15.3:0/425623404 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2d1c15e420 msgr2=0x7f2d04004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:20.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:20.787+0000 7f2d228ad640 1 --2- 172.21.15.3:0/425623404 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2d1c15e420 0x7f2d04004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:20.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:20.787+0000 7f2d228ad640 1 -- 172.21.15.3:0/425623404 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2d040039b0 msgr2=0x7f2d04003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:20.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:20.787+0000 7f2d228ad640 1 --2- 172.21.15.3:0/425623404 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2d040039b0 0x7f2d04003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:20.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:20.787+0000 7f2d228ad640 1 -- 172.21.15.3:0/425623404 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d04003520 con 0x7f2d04004d40 2024-05-09T06:36:20.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:20.788+0000 7f2d228ad640 1 --2- 172.21.15.3:0/425623404 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2d04004d40 0x7f2d040051c0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f2d0c00aa70 tx=0x7f2d04005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa02a6908002d2d9 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:20.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:20.788+0000 7f2d210aa640 1 -- 172.21.15.3:0/425623404 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d0c00a690 con 0x7f2d04004d40 2024-05-09T06:36:20.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:20.788+0000 7f2d210aa640 1 -- 172.21.15.3:0/425623404 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d0c02f070 con 0x7f2d04004d40 2024-05-09T06:36:20.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:20.788+0000 7f2d210aa640 1 -- 172.21.15.3:0/425623404 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d0c034470 con 0x7f2d04004d40 2024-05-09T06:36:20.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:20.789+0000 7f2d24310640 1 -- 172.21.15.3:0/425623404 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2d04004d40 msgr2=0x7f2d040051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:20.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:20.789+0000 7f2d24310640 1 --2- 172.21.15.3:0/425623404 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2d04004d40 0x7f2d040051c0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f2d0c00aa70 tx=0x7f2d04005830 comp rx=0 tx=0).stop 2024-05-09T06:36:20.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:20.789+0000 7f2d228ad640 1 -- 172.21.15.3:0/425623404 reap_dead start 2024-05-09T06:36:20.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:20.789+0000 7f2d24310640 1 -- 172.21.15.3:0/425623404 shutdown_connections 2024-05-09T06:36:20.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:20.789+0000 7f2d24310640 1 -- 172.21.15.3:0/425623404 >> 172.21.15.3:0/425623404 conn(0x7f2d1c06e3d0 msgr2=0x7f2d1c06e7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:20.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:20.789+0000 7f2d24310640 1 -- 172.21.15.3:0/425623404 shutdown_connections 2024-05-09T06:36:20.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:20.789+0000 7f2d24310640 1 -- 172.21.15.3:0/425623404 wait complete. 2024-05-09T06:36:20.776 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:20.790+0000 7f2d24310640 1 Processor -- start 2024-05-09T06:36:20.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:20.790+0000 7f2d24310640 1 -- start start 2024-05-09T06:36:20.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:20.791+0000 7f2d24310640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2d04004d40 0x7f2d1c0b3340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:20.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:20.791+0000 7f2d24310640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2d1c0d05a0 0x7f2d1c0b0d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:20.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:20.791+0000 7f2d24310640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2d1c1f7460 0x7f2d1c1f78e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:20.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:20.791+0000 7f2d24310640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f2d1c0b5850 con 0x7f2d1c1f7460 2024-05-09T06:36:20.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:20.791+0000 7f2d24310640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f2d1c0b5570 con 0x7f2d04004d40 2024-05-09T06:36:20.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:20.791+0000 7f2d24310640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f2d1c0b5b30 con 0x7f2d1c0d05a0 2024-05-09T06:36:20.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:20.791+0000 7f2d228ad640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2d1c1f7460 0x7f2d1c1f78e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:20.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:20.791+0000 7f2d228ad640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2d1c1f7460 0x7f2d1c1f78e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50060/0 (socket says 172.21.15.3:50060) 2024-05-09T06:36:20.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:20.791+0000 7f2d228ad640 1 -- 172.21.15.3:0/3582651903 learned_addr learned my addr 172.21.15.3:0/3582651903 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:36:20.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:20.791+0000 7f2d218ab640 1 --2- 172.21.15.3:0/3582651903 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2d1c0d05a0 0x7f2d1c0b0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:20.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:20.791+0000 7f2d220ac640 1 --2- 172.21.15.3:0/3582651903 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2d04004d40 0x7f2d1c0b3340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:20.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:20.791+0000 7f2d228ad640 1 -- 172.21.15.3:0/3582651903 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2d1c0d05a0 msgr2=0x7f2d1c0b0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:20.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:20.791+0000 7f2d228ad640 1 --2- 172.21.15.3:0/3582651903 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2d1c0d05a0 0x7f2d1c0b0d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:20.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:20.791+0000 7f2d228ad640 1 -- 172.21.15.3:0/3582651903 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2d04004d40 msgr2=0x7f2d1c0b3340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:20.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:20.791+0000 7f2d228ad640 1 --2- 172.21.15.3:0/3582651903 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2d04004d40 0x7f2d1c0b3340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:20.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:20.791+0000 7f2d228ad640 1 -- 172.21.15.3:0/3582651903 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d1c156320 con 0x7f2d1c1f7460 2024-05-09T06:36:20.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:20.791+0000 7f2d228ad640 1 --2- 172.21.15.3:0/3582651903 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2d1c1f7460 0x7f2d1c1f78e0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f2d0c001ec0 tx=0x7f2d0c001f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:20.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:20.792+0000 7f2d1affd640 1 -- 172.21.15.3:0/3582651903 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d0c039070 con 0x7f2d1c1f7460 2024-05-09T06:36:20.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:20.792+0000 7f2d1affd640 1 -- 172.21.15.3:0/3582651903 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d0c001bb0 con 0x7f2d1c1f7460 2024-05-09T06:36:20.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:20.792+0000 7f2d24310640 1 -- 172.21.15.3:0/3582651903 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d1c0d0a20 con 0x7f2d1c1f7460 2024-05-09T06:36:20.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:20.792+0000 7f2d1affd640 1 -- 172.21.15.3:0/3582651903 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d0c00a690 con 0x7f2d1c1f7460 2024-05-09T06:36:20.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:20.792+0000 7f2d24310640 1 -- 172.21.15.3:0/3582651903 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d1c0b3c00 con 0x7f2d1c1f7460 2024-05-09T06:36:20.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:20.793+0000 7f2d24310640 1 -- 172.21.15.3:0/3582651903 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d1c04f660 con 0x7f2d1c1f7460 2024-05-09T06:36:20.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:20.793+0000 7f2d1affd640 1 -- 172.21.15.3:0/3582651903 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2d0c044060 con 0x7f2d1c1f7460 2024-05-09T06:36:20.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:20.794+0000 7f2d1affd640 1 --2- 172.21.15.3:0/3582651903 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f2d0806be60 0x7f2d0806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:20.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:20.794+0000 7f2d1affd640 1 -- 172.21.15.3:0/3582651903 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f2d0c0aaaa0 con 0x7f2d1c1f7460 2024-05-09T06:36:20.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:20.794+0000 7f2d220ac640 1 --2- 172.21.15.3:0/3582651903 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f2d0806be60 0x7f2d0806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:20.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:20.795+0000 7f2d220ac640 1 --2- 172.21.15.3:0/3582651903 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f2d0806be60 0x7f2d0806e320 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f2d14009b00 tx=0x7f2d14008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:20.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:20.798+0000 7f2d1affd640 1 -- 172.21.15.3:0/3582651903 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f2d1c0b5010 con 0x7f2d1c1f7460 2024-05-09T06:36:21.002 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.021+0000 7f2d24310640 1 -- 172.21.15.3:0/3582651903 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2d1c0b3e40 con 0x7f2d1c1f7460 2024-05-09T06:36:21.002 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.022+0000 7f2d1affd640 1 -- 172.21.15.3:0/3582651903 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5172 (secure 0 0 0) 0x7f2d0c0793b0 con 0x7f2d1c1f7460 2024-05-09T06:36:21.003 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-09T06:36:21.003 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4272,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6834/126273055","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":126273055},{"type":"v1","addr":"172.21.15.3:6834","nonce":126273055}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4273,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6836/3885692063","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":3885692063},{"type":"v1","addr":"172.21.15.190:6836","nonce":3885692063}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4275,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/3461228756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6833","nonce":3461228756},{"type":"v1","addr":"172.21.15.3:6835","nonce":3461228756}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-09T06:36:07.221126+0000","modified":"2024-05-09T06:36:10.231051+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4276},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4276":{"gid":4276,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.190:6837/4111472052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6835","nonce":4111472052},{"type":"v1","addr":"172.21.15.190:6837","nonce":4111472052}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4276,"qdb_cluster":[4276]},"id":1}]} 2024-05-09T06:36:21.003 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 8 2024-05-09T06:36:21.005 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.023+0000 7f2d24310640 1 -- 172.21.15.3:0/3582651903 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f2d0806be60 msgr2=0x7f2d0806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:21.005 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.023+0000 7f2d24310640 1 --2- 172.21.15.3:0/3582651903 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f2d0806be60 0x7f2d0806e320 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f2d14009b00 tx=0x7f2d14008040 comp rx=0 tx=0).stop 2024-05-09T06:36:21.005 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.023+0000 7f2d24310640 1 -- 172.21.15.3:0/3582651903 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2d1c1f7460 msgr2=0x7f2d1c1f78e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:21.005 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.023+0000 7f2d24310640 1 --2- 172.21.15.3:0/3582651903 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2d1c1f7460 0x7f2d1c1f78e0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f2d0c001ec0 tx=0x7f2d0c001f90 comp rx=0 tx=0).stop 2024-05-09T06:36:21.005 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.024+0000 7f2d24310640 1 -- 172.21.15.3:0/3582651903 shutdown_connections 2024-05-09T06:36:21.005 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.024+0000 7f2d24310640 1 --2- 172.21.15.3:0/3582651903 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f2d0806be60 0x7f2d0806e320 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:21.005 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.024+0000 7f2d24310640 1 --2- 172.21.15.3:0/3582651903 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2d1c1f7460 0x7f2d1c1f78e0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:21.005 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.024+0000 7f2d24310640 1 --2- 172.21.15.3:0/3582651903 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2d1c0d05a0 0x7f2d1c0b0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:21.005 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.024+0000 7f2d24310640 1 --2- 172.21.15.3:0/3582651903 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2d04004d40 0x7f2d1c0b3340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:21.005 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.024+0000 7f2d24310640 1 -- 172.21.15.3:0/3582651903 >> 172.21.15.3:0/3582651903 conn(0x7f2d1c06e3d0 msgr2=0x7f2d1c15ea10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:21.005 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.024+0000 7f2d24310640 1 -- 172.21.15.3:0/3582651903 shutdown_connections 2024-05-09T06:36:21.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.024+0000 7f2d24310640 1 -- 172.21.15.3:0/3582651903 wait complete. 2024-05-09T06:36:21.418 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 8, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-09T06:36:07.221126+0000', 'modified': '2024-05-09T06:36:10.231051+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4276}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4276': {'gid': 4276, 'name': 'd', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.190:6837/4111472052', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.190:6835', 'nonce': 4111472052}, {'type': 'v1', 'addr': '172.21.15.190:6837', 'nonce': 4111472052}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4276, 'qdb_cluster': [4276]} 2024-05-09T06:36:21.419 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-09T06:36:21.419 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-05-09T06:36:21.529 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.549+0000 7fdd8e534640 1 Processor -- start 2024-05-09T06:36:21.530 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.549+0000 7fdd8e534640 1 -- start start 2024-05-09T06:36:21.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.549+0000 7fdd8e534640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fdd8815c0f0 0x7fdd88160730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:21.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.549+0000 7fdd8e534640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdd88160c70 con 0x7fdd8806e960 2024-05-09T06:36:21.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.549+0000 7fdd8e534640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdd880b40f0 con 0x7fdd880b4e30 2024-05-09T06:36:21.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.549+0000 7fdd8e534640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7fdd880b3050 con 0x7fdd8815c0f0 2024-05-09T06:36:21.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.550+0000 7fdd8cad1640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fdd8806e960 0x7fdd8815c4d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:47840/0 (socket says 172.21.15.3:47840) 2024-05-09T06:36:21.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.550+0000 7fdd8cad1640 1 -- 172.21.15.3:0/1295914674 learned_addr learned my addr 172.21.15.3:0/1295914674 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-05-09T06:36:21.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.550+0000 7fdd87fff640 1 --2- 172.21.15.3:0/1295914674 >> v2:172.21.15.190:3300/0 conn(0x7fdd8815c0f0 0x7fdd88160730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:21.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.550+0000 7fdd86ffd640 1 -- 172.21.15.3:0/1295914674 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1500408878 0 0) 0x7fdd80001490 con 0x7fdd8806e960 2024-05-09T06:36:21.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.550+0000 7fdd86ffd640 1 -- 172.21.15.3:0/1295914674 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdd70003520 con 0x7fdd8806e960 2024-05-09T06:36:21.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.550+0000 7fdd86ffd640 1 -- 172.21.15.3:0/1295914674 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3965170681 0 0) 0x7fdd70003520 con 0x7fdd8806e960 2024-05-09T06:36:21.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.550+0000 7fdd87fff640 1 -- 172.21.15.3:0/1295914674 >> v1:172.21.15.190:6789/0 conn(0x7fdd880b4e30 legacy=0x7fdd8806e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:21.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.550+0000 7fdd87fff640 1 -- 172.21.15.3:0/1295914674 >> v1:172.21.15.3:6789/0 conn(0x7fdd8806e960 legacy=0x7fdd8815c4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:21.533 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.550+0000 7fdd87fff640 1 -- 172.21.15.3:0/1295914674 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd880b3330 con 0x7fdd8815c0f0 2024-05-09T06:36:21.533 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.551+0000 7fdd87fff640 1 --2- 172.21.15.3:0/1295914674 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdd8815c0f0 0x7fdd88160730 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fdd780097a0 tx=0x7fdd7802a130 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a88cadd0b5a0757 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:21.533 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.551+0000 7fdd86ffd640 1 -- 172.21.15.3:0/1295914674 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd78037070 con 0x7fdd8815c0f0 2024-05-09T06:36:21.533 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.551+0000 7fdd86ffd640 1 -- 172.21.15.3:0/1295914674 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd7802abc0 con 0x7fdd8815c0f0 2024-05-09T06:36:21.533 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.551+0000 7fdd86ffd640 1 -- 172.21.15.3:0/1295914674 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd80001be0 con 0x7fdd8815c0f0 2024-05-09T06:36:21.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.552+0000 7fdd8e534640 1 -- 172.21.15.3:0/1295914674 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdd8815c0f0 msgr2=0x7fdd88160730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:21.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.552+0000 7fdd8e534640 1 --2- 172.21.15.3:0/1295914674 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdd8815c0f0 0x7fdd88160730 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fdd780097a0 tx=0x7fdd7802a130 comp rx=0 tx=0).stop 2024-05-09T06:36:21.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.552+0000 7fdd8e534640 1 -- 172.21.15.3:0/1295914674 shutdown_connections 2024-05-09T06:36:21.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.552+0000 7fdd8e534640 1 --2- 172.21.15.3:0/1295914674 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdd8815c0f0 0x7fdd88160730 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:21.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.552+0000 7fdd8e534640 1 -- 172.21.15.3:0/1295914674 >> 172.21.15.3:0/1295914674 conn(0x7fdd880b1c20 msgr2=0x7fdd880b2030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:21.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.552+0000 7fdd8e534640 1 -- 172.21.15.3:0/1295914674 shutdown_connections 2024-05-09T06:36:21.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.552+0000 7fdd8e534640 1 -- 172.21.15.3:0/1295914674 wait complete. 2024-05-09T06:36:21.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.553+0000 7fdd8e534640 1 Processor -- start 2024-05-09T06:36:21.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.553+0000 7fdd8e534640 1 -- start start 2024-05-09T06:36:21.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.554+0000 7fdd8e534640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdd8806e960 0x7fdd8815ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:21.537 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.554+0000 7fdd8e534640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdd880b4e30 0x7fdd88139dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:21.537 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.554+0000 7fdd8e534640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdd8815c0f0 0x7fdd8814c120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:21.537 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.554+0000 7fdd8e534640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fdd880b3bd0 con 0x7fdd8806e960 2024-05-09T06:36:21.537 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.554+0000 7fdd8e534640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fdd880b38f0 con 0x7fdd8815c0f0 2024-05-09T06:36:21.537 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.554+0000 7fdd8e534640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fdd8815cbe0 con 0x7fdd880b4e30 2024-05-09T06:36:21.537 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.554+0000 7fdd87fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdd8806e960 0x7fdd8815ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:21.537 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.554+0000 7fdd87fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdd8806e960 0x7fdd8815ed90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50072/0 (socket says 172.21.15.3:50072) 2024-05-09T06:36:21.537 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.554+0000 7fdd87fff640 1 -- 172.21.15.3:0/588047185 learned_addr learned my addr 172.21.15.3:0/588047185 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:36:21.537 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.554+0000 7fdd877fe640 1 --2- 172.21.15.3:0/588047185 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdd880b4e30 0x7fdd88139dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:21.537 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.554+0000 7fdd8cad1640 1 --2- 172.21.15.3:0/588047185 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdd8815c0f0 0x7fdd8814c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:21.537 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.554+0000 7fdd87fff640 1 -- 172.21.15.3:0/588047185 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdd880b4e30 msgr2=0x7fdd88139dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:21.537 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.554+0000 7fdd87fff640 1 --2- 172.21.15.3:0/588047185 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdd880b4e30 0x7fdd88139dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:21.538 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.554+0000 7fdd87fff640 1 -- 172.21.15.3:0/588047185 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdd8815c0f0 msgr2=0x7fdd8814c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:21.538 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.554+0000 7fdd87fff640 1 --2- 172.21.15.3:0/588047185 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdd8815c0f0 0x7fdd8814c120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:21.538 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.554+0000 7fdd87fff640 1 -- 172.21.15.3:0/588047185 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd8813a310 con 0x7fdd8806e960 2024-05-09T06:36:21.538 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.554+0000 7fdd877fe640 1 --2- 172.21.15.3:0/588047185 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdd880b4e30 0x7fdd88139dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:36:21.538 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.554+0000 7fdd8cad1640 1 --2- 172.21.15.3:0/588047185 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdd8815c0f0 0x7fdd8814c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:36:21.538 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.554+0000 7fdd87fff640 1 --2- 172.21.15.3:0/588047185 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdd8806e960 0x7fdd8815ed90 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fdd78035040 tx=0x7fdd780046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:21.538 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.555+0000 7fdd84ff9640 1 -- 172.21.15.3:0/588047185 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd78041070 con 0x7fdd8806e960 2024-05-09T06:36:21.538 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.555+0000 7fdd84ff9640 1 -- 172.21.15.3:0/588047185 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd7803c660 con 0x7fdd8806e960 2024-05-09T06:36:21.538 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.555+0000 7fdd8e534640 1 -- 172.21.15.3:0/588047185 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd8813a800 con 0x7fdd8806e960 2024-05-09T06:36:21.538 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.555+0000 7fdd84ff9640 1 -- 172.21.15.3:0/588047185 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd78037040 con 0x7fdd8806e960 2024-05-09T06:36:21.538 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.555+0000 7fdd8e534640 1 -- 172.21.15.3:0/588047185 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd8813adc0 con 0x7fdd8806e960 2024-05-09T06:36:21.545 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.556+0000 7fdd84ff9640 1 -- 172.21.15.3:0/588047185 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fdd78044070 con 0x7fdd8806e960 2024-05-09T06:36:21.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.557+0000 7fdd84ff9640 1 --2- 172.21.15.3:0/588047185 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fdd7406be30 0x7fdd7406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:21.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.557+0000 7fdd84ff9640 1 -- 172.21.15.3:0/588047185 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fdd780ab610 con 0x7fdd8806e960 2024-05-09T06:36:21.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.557+0000 7fdd877fe640 1 --2- 172.21.15.3:0/588047185 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fdd7406be30 0x7fdd7406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:21.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.558+0000 7fdd91242740 1 Processor -- start 2024-05-09T06:36:21.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.558+0000 7fdd91242740 1 -- start start 2024-05-09T06:36:21.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.558+0000 7fdd877fe640 1 --2- 172.21.15.3:0/588047185 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fdd7406be30 0x7fdd7406e2f0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fdd880b13b0 tx=0x7fdd7c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:21.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.558+0000 7fdd91242740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55bcb7cea670 0x55bcb7ceaa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:21.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.558+0000 7fdd91242740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bcb7d07f10 0x55bcb7d08460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:21.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.558+0000 7fdd91242740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55bcb7cc0350 0x55bcb7d065d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:21.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.558+0000 7fdd91242740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55bcb7c234d0 con 0x55bcb7cc0350 2024-05-09T06:36:21.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.558+0000 7fdd91242740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55bcb7bcb3e0 con 0x55bcb7d07f10 2024-05-09T06:36:21.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.558+0000 7fdd91242740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x55bcb7c1f490 con 0x55bcb7cea670 2024-05-09T06:36:21.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.558+0000 7fdd8cad1640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55bcb7cc0350 0x55bcb7d065d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:21.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.558+0000 7fdd8cad1640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55bcb7cc0350 0x55bcb7d065d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50076/0 (socket says 172.21.15.3:50076) 2024-05-09T06:36:21.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.558+0000 7fdd8cad1640 1 -- 172.21.15.3:0/3458863049 learned_addr learned my addr 172.21.15.3:0/3458863049 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:36:21.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.559+0000 7fdd877fe640 1 --2- 172.21.15.3:0/3458863049 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bcb7d07f10 0x55bcb7d08460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:21.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.559+0000 7fdd87fff640 1 --2- 172.21.15.3:0/3458863049 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55bcb7cea670 0x55bcb7ceaa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:21.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.559+0000 7fdd8cad1640 1 -- 172.21.15.3:0/3458863049 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55bcb7cea670 msgr2=0x55bcb7ceaa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:21.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.559+0000 7fdd8cad1640 1 --2- 172.21.15.3:0/3458863049 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55bcb7cea670 0x55bcb7ceaa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:21.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.559+0000 7fdd8cad1640 1 -- 172.21.15.3:0/3458863049 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bcb7d07f10 msgr2=0x55bcb7d08460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:21.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.559+0000 7fdd8cad1640 1 --2- 172.21.15.3:0/3458863049 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bcb7d07f10 0x55bcb7d08460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:21.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.559+0000 7fdd8cad1640 1 -- 172.21.15.3:0/3458863049 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bcb79372d0 con 0x55bcb7cc0350 2024-05-09T06:36:21.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.559+0000 7fdd87fff640 1 --2- 172.21.15.3:0/3458863049 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55bcb7cea670 0x55bcb7ceaa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:36:21.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.559+0000 7fdd8cad1640 1 --2- 172.21.15.3:0/3458863049 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55bcb7cc0350 0x55bcb7d065d0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fdd80001b70 tx=0x7fdd8000d520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44aec23147e48236 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:21.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.559+0000 7fdd6dffb640 1 -- 172.21.15.3:0/3458863049 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd80004d60 con 0x55bcb7cc0350 2024-05-09T06:36:21.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.559+0000 7fdd6dffb640 1 -- 172.21.15.3:0/3458863049 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd8000de40 con 0x55bcb7cc0350 2024-05-09T06:36:21.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.559+0000 7fdd6dffb640 1 -- 172.21.15.3:0/3458863049 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd800072d0 con 0x55bcb7cc0350 2024-05-09T06:36:21.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.559+0000 7fdd91242740 1 -- 172.21.15.3:0/3458863049 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55bcb7cc0350 msgr2=0x55bcb7d065d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:21.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.559+0000 7fdd91242740 1 --2- 172.21.15.3:0/3458863049 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55bcb7cc0350 0x55bcb7d065d0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fdd80001b70 tx=0x7fdd8000d520 comp rx=0 tx=0).stop 2024-05-09T06:36:21.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.560+0000 7fdd91242740 1 -- 172.21.15.3:0/3458863049 shutdown_connections 2024-05-09T06:36:21.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.560+0000 7fdd91242740 1 --2- 172.21.15.3:0/3458863049 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55bcb7cc0350 0x55bcb7d065d0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:21.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.560+0000 7fdd91242740 1 --2- 172.21.15.3:0/3458863049 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bcb7d07f10 0x55bcb7d08460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:21.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.560+0000 7fdd91242740 1 --2- 172.21.15.3:0/3458863049 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55bcb7cea670 0x55bcb7ceaa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:21.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.560+0000 7fdd91242740 1 -- 172.21.15.3:0/3458863049 >> 172.21.15.3:0/3458863049 conn(0x55bcb7cbcc80 msgr2=0x55bcb7d05480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:21.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.560+0000 7fdd91242740 1 -- 172.21.15.3:0/3458863049 shutdown_connections 2024-05-09T06:36:21.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.560+0000 7fdd91242740 1 -- 172.21.15.3:0/3458863049 wait complete. 2024-05-09T06:36:21.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.560+0000 7fdd91242740 1 -- start start 2024-05-09T06:36:21.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.560+0000 7fdd91242740 1 Processor -- start 2024-05-09T06:36:21.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.561+0000 7fdd91242740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bcb7cea670 0x55bcb7cd5ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:21.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.561+0000 7fdd91242740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55bcb7d07f10 0x55bcb7cd9960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:21.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.561+0000 7fdd91242740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55bcb7d06400 0x55bcb7d06880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:21.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.561+0000 7fdd91242740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55bcb7cc10f0 con 0x55bcb7d07f10 2024-05-09T06:36:21.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.561+0000 7fdd91242740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55bcb7bde8e0 con 0x55bcb7cea670 2024-05-09T06:36:21.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.561+0000 7fdd91242740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x55bcb7cbabb0 con 0x55bcb7d06400 2024-05-09T06:36:21.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.561+0000 7fdd91242740 10 client.0 _ll_register_callbacks cb 0x55bcb7bab9c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T06:36:21.550 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.561+0000 7fdd877fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55bcb7d07f10 0x55bcb7cd9960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:21.550 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.561+0000 7fdd877fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55bcb7d07f10 0x55bcb7cd9960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50092/0 (socket says 172.21.15.3:50092) 2024-05-09T06:36:21.550 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.561+0000 7fdd877fe640 1 -- 172.21.15.3:0/4167459610 learned_addr learned my addr 172.21.15.3:0/4167459610 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:36:21.550 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.561+0000 7fdd87fff640 1 --2- 172.21.15.3:0/4167459610 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bcb7cea670 0x55bcb7cd5ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:21.550 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.561+0000 7fdd8cad1640 1 --2- 172.21.15.3:0/4167459610 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55bcb7d06400 0x55bcb7d06880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:21.550 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.562+0000 7fdd877fe640 1 -- 172.21.15.3:0/4167459610 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55bcb7d06400 msgr2=0x55bcb7d06880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:21.550 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.562+0000 7fdd877fe640 1 --2- 172.21.15.3:0/4167459610 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55bcb7d06400 0x55bcb7d06880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:21.550 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.562+0000 7fdd877fe640 1 -- 172.21.15.3:0/4167459610 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bcb7cea670 msgr2=0x55bcb7cd5ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:21.550 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.562+0000 7fdd877fe640 1 --2- 172.21.15.3:0/4167459610 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bcb7cea670 0x55bcb7cd5ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:21.550 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.562+0000 7fdd877fe640 1 -- 172.21.15.3:0/4167459610 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bcb7a016b0 con 0x55bcb7d07f10 2024-05-09T06:36:21.551 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.562+0000 7fdd8cad1640 1 --2- 172.21.15.3:0/4167459610 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55bcb7d06400 0x55bcb7d06880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:36:21.551 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.562+0000 7fdd877fe640 1 --2- 172.21.15.3:0/4167459610 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55bcb7d07f10 0x55bcb7cd9960 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fdd7c00de50 tx=0x7fdd7c000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bd1962d4a70166f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:21.551 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.562+0000 7fdd66ffd640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-05-09T06:36:21.551 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.562+0000 7fdd66ffd640 1 -- 172.21.15.3:0/4167459610 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd7c014e30 con 0x55bcb7d07f10 2024-05-09T06:36:21.551 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.562+0000 7fdd66ffd640 1 -- 172.21.15.3:0/4167459610 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fdd7c0125e0 con 0x55bcb7d07f10 2024-05-09T06:36:21.551 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.562+0000 7fdd91242740 1 -- 172.21.15.3:0/4167459610 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55bcb7c30c00 con 0x55bcb7d07f10 2024-05-09T06:36:21.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.562+0000 7fdd66ffd640 1 -- 172.21.15.3:0/4167459610 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) ==== 24+0+0 (secure 0 0 0) 0x7fdd7c02d050 con 0x55bcb7d07f10 2024-05-09T06:36:21.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.562+0000 7fdd91242740 10 client.4656 fetch_fsmap learned FSMap version 8 2024-05-09T06:36:21.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.562+0000 7fdd91242740 1 -- 172.21.15.3:0/4167459610 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) -- 0x55bcb7cdade0 con 0x55bcb7d07f10 2024-05-09T06:36:21.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.562+0000 7fdd66ffd640 1 -- 172.21.15.3:0/4167459610 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55bcb7c30c00 con 0x55bcb7d07f10 2024-05-09T06:36:21.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.562+0000 7fdd66ffd640 1 -- 172.21.15.3:0/4167459610 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 8) ==== 3494+0+0 (secure 0 0 0) 0x7fdd7c030020 con 0x55bcb7d07f10 2024-05-09T06:36:21.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.562+0000 7fdd91242740 10 client.4656 fetch_fsmap finished waiting for FSMap version 8 2024-05-09T06:36:21.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.562+0000 7fdd91242740 10 client.4656 resolve_mds: resolved name 'd' to daemon mds.d 2024-05-09T06:36:21.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.562+0000 7fdd91242740 20 client.4656 populate_metadata read hostname 'smithi003' 2024-05-09T06:36:21.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.562+0000 7fdd91242740 1 --2- 172.21.15.3:0/4167459610 >> [v2:172.21.15.190:6835/4111472052,v1:172.21.15.190:6837/4111472052] conn(0x55bcb7cdba10 0x55bcb7cdbdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:21.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.562+0000 7fdd91242740 4 client.4656 mds_command: new command op to 4276 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T06:36:21.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.562+0000 7fdd91242740 1 -- 172.21.15.3:0/4167459610 --> [v2:172.21.15.190:6835/4111472052,v1:172.21.15.190:6837/4111472052] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bcb7c2af50 con 0x55bcb7cdba10 2024-05-09T06:36:21.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.563+0000 7fdd87fff640 1 --2- 172.21.15.3:0/4167459610 >> [v2:172.21.15.190:6835/4111472052,v1:172.21.15.190:6837/4111472052] conn(0x55bcb7cdba10 0x55bcb7cdbdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:21.554 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.563+0000 7fdd87fff640 1 --2- 172.21.15.3:0/4167459610 >> [v2:172.21.15.190:6835/4111472052,v1:172.21.15.190:6837/4111472052] conn(0x55bcb7cdba10 0x55bcb7cdbdf0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=df742e530b4d9456 server_cookie=c69368a3552b044b in_seq=0 out_seq=0 2024-05-09T06:36:21.554 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.563+0000 7fdd66ffd640 10 client.4656 ms_handle_connect on v2:172.21.15.190:6835/4111472052 2024-05-09T06:36:21.554 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.564+0000 7fdd66ffd640 1 -- 172.21.15.3:0/4167459610 <== mds.0 v2:172.21.15.190:6835/4111472052 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55bcb7c2af50 con 0x55bcb7cdba10 2024-05-09T06:36:21.554 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.564+0000 7fdd66ffd640 10 client.4656 handle_command_reply: tid=0 2024-05-09T06:36:21.554 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.564+0000 7fdd91242740 1 client.4656 shutdown 2024-05-09T06:36:21.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.564+0000 7fdd91242740 1 -- 172.21.15.3:0/4167459610 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55bcb7d07f10 msgr2=0x55bcb7cd9960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:21.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.564+0000 7fdd91242740 1 --2- 172.21.15.3:0/4167459610 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55bcb7d07f10 0x55bcb7cd9960 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fdd7c00de50 tx=0x7fdd7c000ed0 comp rx=0 tx=0).stop 2024-05-09T06:36:21.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.564+0000 7fdd91242740 1 -- 172.21.15.3:0/4167459610 shutdown_connections 2024-05-09T06:36:21.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.564+0000 7fdd91242740 1 --2- 172.21.15.3:0/4167459610 >> [v2:172.21.15.190:6835/4111472052,v1:172.21.15.190:6837/4111472052] conn(0x55bcb7cdba10 0x55bcb7cdbdf0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:21.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.564+0000 7fdd91242740 1 --2- 172.21.15.3:0/4167459610 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55bcb7d06400 0x55bcb7d06880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:21.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.564+0000 7fdd91242740 1 --2- 172.21.15.3:0/4167459610 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55bcb7d07f10 0x55bcb7cd9960 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:21.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.564+0000 7fdd66ffd640 0 client.4656 ms_handle_reset on v2:172.21.15.190:6835/4111472052 2024-05-09T06:36:21.556 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.564+0000 7fdd91242740 1 --2- 172.21.15.3:0/4167459610 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bcb7cea670 0x55bcb7cd5ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:21.556 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.564+0000 7fdd91242740 1 -- 172.21.15.3:0/4167459610 >> 172.21.15.3:0/4167459610 conn(0x55bcb7cc0350 msgr2=0x55bcb7d08a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:21.556 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.564+0000 7fdd91242740 1 -- 172.21.15.3:0/4167459610 shutdown_connections 2024-05-09T06:36:21.556 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.564+0000 7fdd91242740 1 -- 172.21.15.3:0/4167459610 wait complete. 2024-05-09T06:36:21.556 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.564+0000 7fdd91242740 20 client.4656 trim_cache size 0 max 16384 2024-05-09T06:36:21.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.582+0000 7fdd91242740 1 Processor -- start 2024-05-09T06:36:21.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.582+0000 7fdd91242740 1 -- start start 2024-05-09T06:36:21.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.583+0000 7fdd91242740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bcb7cc0350 0x55bcb7d14e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:21.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.583+0000 7fdd91242740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55bcb7cea670 0x55bcb7cbfcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:21.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.583+0000 7fdd91242740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55bcb7d06400 0x55bcb7cdbdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:21.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.583+0000 7fdd91242740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fdd881fd840 con 0x55bcb7d06400 2024-05-09T06:36:21.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.583+0000 7fdd91242740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fdd8813cb10 con 0x55bcb7cc0350 2024-05-09T06:36:21.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.583+0000 7fdd91242740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fdd8813d160 con 0x55bcb7cea670 2024-05-09T06:36:21.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.583+0000 7fdd8cad1640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55bcb7d06400 0x55bcb7cdbdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:21.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.583+0000 7fdd8cad1640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55bcb7d06400 0x55bcb7cdbdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50108/0 (socket says 172.21.15.3:50108) 2024-05-09T06:36:21.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.583+0000 7fdd8cad1640 1 -- 172.21.15.3:0/31691887 learned_addr learned my addr 172.21.15.3:0/31691887 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:36:21.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.583+0000 7fdd877fe640 1 --2- 172.21.15.3:0/31691887 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55bcb7cea670 0x55bcb7cbfcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:21.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.583+0000 7fdd8cad1640 1 -- 172.21.15.3:0/31691887 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55bcb7cea670 msgr2=0x55bcb7cbfcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:21.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.583+0000 7fdd8cad1640 1 --2- 172.21.15.3:0/31691887 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55bcb7cea670 0x55bcb7cbfcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:21.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.583+0000 7fdd8cad1640 1 -- 172.21.15.3:0/31691887 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bcb7cc0350 msgr2=0x55bcb7d14e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:21.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.583+0000 7fdd87fff640 1 --2- 172.21.15.3:0/31691887 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bcb7cc0350 0x55bcb7d14e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:21.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.583+0000 7fdd8cad1640 1 --2- 172.21.15.3:0/31691887 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bcb7cc0350 0x55bcb7d14e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:21.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.583+0000 7fdd8cad1640 1 -- 172.21.15.3:0/31691887 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bcb7cbc730 con 0x55bcb7d06400 2024-05-09T06:36:21.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.583+0000 7fdd877fe640 1 --2- 172.21.15.3:0/31691887 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55bcb7cea670 0x55bcb7cbfcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:36:21.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.583+0000 7fdd87fff640 1 --2- 172.21.15.3:0/31691887 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bcb7cc0350 0x55bcb7d14e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:36:21.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.584+0000 7fdd8cad1640 1 --2- 172.21.15.3:0/31691887 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55bcb7d06400 0x55bcb7cdbdf0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fdd8000da20 tx=0x7fdd80006000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea230de21e406ba1 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:21.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.584+0000 7fdd6d7fa640 1 -- 172.21.15.3:0/31691887 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd80007de0 con 0x55bcb7d06400 2024-05-09T06:36:21.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.584+0000 7fdd6d7fa640 1 -- 172.21.15.3:0/31691887 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd80025070 con 0x55bcb7d06400 2024-05-09T06:36:21.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.584+0000 7fdd6d7fa640 1 -- 172.21.15.3:0/31691887 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd80004d80 con 0x55bcb7d06400 2024-05-09T06:36:21.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.584+0000 7fdd91242740 1 -- 172.21.15.3:0/31691887 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55bcb7d06400 msgr2=0x55bcb7cdbdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:21.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.584+0000 7fdd91242740 1 --2- 172.21.15.3:0/31691887 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55bcb7d06400 0x55bcb7cdbdf0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fdd8000da20 tx=0x7fdd80006000 comp rx=0 tx=0).stop 2024-05-09T06:36:21.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.584+0000 7fdd91242740 1 -- 172.21.15.3:0/31691887 shutdown_connections 2024-05-09T06:36:21.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.584+0000 7fdd91242740 1 --2- 172.21.15.3:0/31691887 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55bcb7d06400 0x55bcb7cdbdf0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:21.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.584+0000 7fdd91242740 1 --2- 172.21.15.3:0/31691887 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55bcb7cea670 0x55bcb7cbfcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:21.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.584+0000 7fdd91242740 1 --2- 172.21.15.3:0/31691887 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bcb7cc0350 0x55bcb7d14e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:21.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.584+0000 7fdd91242740 1 -- 172.21.15.3:0/31691887 >> 172.21.15.3:0/31691887 conn(0x55bcb7d05070 msgr2=0x55bcb7cd5cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:21.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.584+0000 7fdd91242740 1 -- 172.21.15.3:0/31691887 shutdown_connections 2024-05-09T06:36:21.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.584+0000 7fdd91242740 1 -- 172.21.15.3:0/31691887 wait complete. 2024-05-09T06:36:21.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.585+0000 7fdd91242740 1 -- start start 2024-05-09T06:36:21.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.585+0000 7fdd91242740 1 Processor -- start 2024-05-09T06:36:21.578 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.586+0000 7fdd91242740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bcb7cc0350 0x55bcb7d152d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:21.578 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.586+0000 7fdd91242740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55bcb7cea670 0x55bcb7cd8580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:21.578 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.586+0000 7fdd91242740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55bcb7d06400 0x55bcb7d825a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:21.578 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.586+0000 7fdd91242740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55bcb7cd9ef0 con 0x55bcb7d06400 2024-05-09T06:36:21.578 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.586+0000 7fdd91242740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55bcb7cdb0c0 con 0x55bcb7cc0350 2024-05-09T06:36:21.578 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.586+0000 7fdd91242740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x55bcb7d19590 con 0x55bcb7cea670 2024-05-09T06:36:21.578 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.586+0000 7fdd91242740 10 client.0 _ll_register_callbacks cb 0x7fdd7c0013e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T06:36:21.578 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.586+0000 7fdd8cad1640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55bcb7d06400 0x55bcb7d825a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:21.578 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.586+0000 7fdd8cad1640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55bcb7d06400 0x55bcb7d825a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50116/0 (socket says 172.21.15.3:50116) 2024-05-09T06:36:21.578 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.586+0000 7fdd8cad1640 1 -- 172.21.15.3:0/2096787880 learned_addr learned my addr 172.21.15.3:0/2096787880 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:36:21.578 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.586+0000 7fdd877fe640 1 --2- 172.21.15.3:0/2096787880 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55bcb7cea670 0x55bcb7cd8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:21.578 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.586+0000 7fdd87fff640 1 --2- 172.21.15.3:0/2096787880 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bcb7cc0350 0x55bcb7d152d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:21.579 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.586+0000 7fdd8cad1640 1 -- 172.21.15.3:0/2096787880 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55bcb7cea670 msgr2=0x55bcb7cd8580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:21.579 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.586+0000 7fdd8cad1640 1 --2- 172.21.15.3:0/2096787880 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55bcb7cea670 0x55bcb7cd8580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:21.579 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.586+0000 7fdd8cad1640 1 -- 172.21.15.3:0/2096787880 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bcb7cc0350 msgr2=0x55bcb7d152d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:21.579 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.586+0000 7fdd8cad1640 1 --2- 172.21.15.3:0/2096787880 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bcb7cc0350 0x55bcb7d152d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:21.579 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.586+0000 7fdd8cad1640 1 -- 172.21.15.3:0/2096787880 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bcb7d05890 con 0x55bcb7d06400 2024-05-09T06:36:21.579 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.587+0000 7fdd8cad1640 1 --2- 172.21.15.3:0/2096787880 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55bcb7d06400 0x55bcb7d825a0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fdd80016040 tx=0x7fdd80001250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5deb34ba81fa1458 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:21.579 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.587+0000 7fdd66ffd640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-05-09T06:36:21.579 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.587+0000 7fdd66ffd640 1 -- 172.21.15.3:0/2096787880 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd8002d070 con 0x55bcb7d06400 2024-05-09T06:36:21.579 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.587+0000 7fdd66ffd640 1 -- 172.21.15.3:0/2096787880 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fdd80007590 con 0x55bcb7d06400 2024-05-09T06:36:21.579 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.587+0000 7fdd91242740 1 -- 172.21.15.3:0/2096787880 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdd80025070 con 0x55bcb7d06400 2024-05-09T06:36:21.579 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.587+0000 7fdd66ffd640 1 -- 172.21.15.3:0/2096787880 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) ==== 24+0+0 (secure 0 0 0) 0x7fdd8003a050 con 0x55bcb7d06400 2024-05-09T06:36:21.579 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.587+0000 7fdd91242740 10 client.4662 fetch_fsmap learned FSMap version 8 2024-05-09T06:36:21.579 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.587+0000 7fdd91242740 1 -- 172.21.15.3:0/2096787880 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) -- 0x55bcb7ce96c0 con 0x55bcb7d06400 2024-05-09T06:36:21.579 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.587+0000 7fdd66ffd640 1 -- 172.21.15.3:0/2096787880 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd80025070 con 0x55bcb7d06400 2024-05-09T06:36:21.579 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.587+0000 7fdd66ffd640 1 -- 172.21.15.3:0/2096787880 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 8) ==== 3494+0+0 (secure 0 0 0) 0x7fdd8003d020 con 0x55bcb7d06400 2024-05-09T06:36:21.580 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.587+0000 7fdd91242740 10 client.4662 fetch_fsmap finished waiting for FSMap version 8 2024-05-09T06:36:21.580 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.587+0000 7fdd91242740 10 client.4662 resolve_mds: resolved name 'd' to daemon mds.d 2024-05-09T06:36:21.580 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.587+0000 7fdd91242740 20 client.4662 populate_metadata read hostname 'smithi003' 2024-05-09T06:36:21.580 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.587+0000 7fdd91242740 1 --2- 172.21.15.3:0/2096787880 >> [v2:172.21.15.190:6835/4111472052,v1:172.21.15.190:6837/4111472052] conn(0x55bcb7d07f10 0x55bcb7ced3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:21.580 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.587+0000 7fdd91242740 4 client.4662 mds_command: new command op to 4276 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-09T06:36:21.580 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.587+0000 7fdd91242740 1 -- 172.21.15.3:0/2096787880 --> [v2:172.21.15.190:6835/4111472052,v1:172.21.15.190:6837/4111472052] -- command(tid 0: {"prefix": "status"}) -- 0x55bcb7ba8b80 con 0x55bcb7d07f10 2024-05-09T06:36:21.580 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.587+0000 7fdd87fff640 1 --2- 172.21.15.3:0/2096787880 >> [v2:172.21.15.190:6835/4111472052,v1:172.21.15.190:6837/4111472052] conn(0x55bcb7d07f10 0x55bcb7ced3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:21.580 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.588+0000 7fdd87fff640 1 --2- 172.21.15.3:0/2096787880 >> [v2:172.21.15.190:6835/4111472052,v1:172.21.15.190:6837/4111472052] conn(0x55bcb7d07f10 0x55bcb7ced3b0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b73bf3a32503e0e8 server_cookie=c36817c740450d73 in_seq=0 out_seq=0 2024-05-09T06:36:21.580 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.588+0000 7fdd66ffd640 10 client.4662 ms_handle_connect on v2:172.21.15.190:6835/4111472052 2024-05-09T06:36:21.580 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.588+0000 7fdd66ffd640 1 -- 172.21.15.3:0/2096787880 <== mds.0 v2:172.21.15.190:6835/4111472052 1 ==== command_reply(tid 0: 0 ) ==== 8+0+324 (crc 0 0 0) 0x55bcb7ba8b80 con 0x55bcb7d07f10 2024-05-09T06:36:21.580 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.588+0000 7fdd66ffd640 10 client.4662 handle_command_reply: tid=0 2024-05-09T06:36:21.580 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.588+0000 7fdd91242740 1 client.4662 shutdown 2024-05-09T06:36:21.580 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.588+0000 7fdd91242740 1 -- 172.21.15.3:0/2096787880 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55bcb7d06400 msgr2=0x55bcb7d825a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:21.580 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.588+0000 7fdd91242740 1 --2- 172.21.15.3:0/2096787880 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55bcb7d06400 0x55bcb7d825a0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fdd80016040 tx=0x7fdd80001250 comp rx=0 tx=0).stop 2024-05-09T06:36:21.580 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.588+0000 7fdd91242740 1 -- 172.21.15.3:0/2096787880 shutdown_connections 2024-05-09T06:36:21.581 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.588+0000 7fdd91242740 1 --2- 172.21.15.3:0/2096787880 >> [v2:172.21.15.190:6835/4111472052,v1:172.21.15.190:6837/4111472052] conn(0x55bcb7d07f10 0x55bcb7ced3b0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:21.581 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.588+0000 7fdd91242740 1 --2- 172.21.15.3:0/2096787880 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55bcb7d06400 0x55bcb7d825a0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:21.581 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.588+0000 7fdd91242740 1 --2- 172.21.15.3:0/2096787880 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55bcb7cea670 0x55bcb7cd8580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:21.581 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.588+0000 7fdd66ffd640 0 client.4662 ms_handle_reset on v2:172.21.15.190:6835/4111472052 2024-05-09T06:36:21.581 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.588+0000 7fdd91242740 1 --2- 172.21.15.3:0/2096787880 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bcb7cc0350 0x55bcb7d152d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:21.581 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.588+0000 7fdd91242740 1 -- 172.21.15.3:0/2096787880 >> 172.21.15.3:0/2096787880 conn(0x55bcb7d05070 msgr2=0x55bcb7cc0940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:21.581 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.588+0000 7fdd91242740 1 -- 172.21.15.3:0/2096787880 shutdown_connections 2024-05-09T06:36:21.581 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.588+0000 7fdd91242740 1 -- 172.21.15.3:0/2096787880 wait complete. 2024-05-09T06:36:21.581 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.588+0000 7fdd91242740 20 client.4662 trim_cache size 0 max 16384 2024-05-09T06:36:21.581 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-05-09T06:36:21.581 INFO:teuthology.orchestra.run.smithi003.stdout: "cluster_fsid": "16ed737d-2665-431a-8ac6-ef242be3975a", 2024-05-09T06:36:21.582 INFO:teuthology.orchestra.run.smithi003.stdout: "whoami": 0, 2024-05-09T06:36:21.582 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 4276, 2024-05-09T06:36:21.582 INFO:teuthology.orchestra.run.smithi003.stdout: "want_state": "up:active", 2024-05-09T06:36:21.582 INFO:teuthology.orchestra.run.smithi003.stdout: "state": "up:active", 2024-05-09T06:36:21.582 INFO:teuthology.orchestra.run.smithi003.stdout: "fs_name": "cephfs", 2024-05-09T06:36:21.582 INFO:teuthology.orchestra.run.smithi003.stdout: "rank_uptime": 13.742025645, 2024-05-09T06:36:21.582 INFO:teuthology.orchestra.run.smithi003.stdout: "mdsmap_epoch": 8, 2024-05-09T06:36:21.582 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch": 20, 2024-05-09T06:36:21.582 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch_barrier": 20, 2024-05-09T06:36:21.582 INFO:teuthology.orchestra.run.smithi003.stdout: "uptime": 21.210725479000001 2024-05-09T06:36:21.582 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-05-09T06:36:21.584 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.589+0000 7fdd8e534640 1 -- 172.21.15.3:0/588047185 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fdd7406be30 msgr2=0x7fdd7406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:21.584 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.589+0000 7fdd8e534640 1 --2- 172.21.15.3:0/588047185 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fdd7406be30 0x7fdd7406e2f0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fdd880b13b0 tx=0x7fdd7c009040 comp rx=0 tx=0).stop 2024-05-09T06:36:21.584 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.589+0000 7fdd8e534640 1 -- 172.21.15.3:0/588047185 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdd8806e960 msgr2=0x7fdd8815ed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:21.584 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.589+0000 7fdd8e534640 1 --2- 172.21.15.3:0/588047185 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdd8806e960 0x7fdd8815ed90 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fdd78035040 tx=0x7fdd780046c0 comp rx=0 tx=0).stop 2024-05-09T06:36:21.584 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.589+0000 7fdd8e534640 1 -- 172.21.15.3:0/588047185 shutdown_connections 2024-05-09T06:36:21.584 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.589+0000 7fdd8e534640 1 --2- 172.21.15.3:0/588047185 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fdd7406be30 0x7fdd7406e2f0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:21.584 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.589+0000 7fdd8e534640 1 --2- 172.21.15.3:0/588047185 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdd8815c0f0 0x7fdd8814c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:21.584 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.589+0000 7fdd8e534640 1 --2- 172.21.15.3:0/588047185 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdd880b4e30 0x7fdd88139dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:21.584 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.589+0000 7fdd8e534640 1 --2- 172.21.15.3:0/588047185 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdd8806e960 0x7fdd8815ed90 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:21.585 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.589+0000 7fdd8e534640 1 -- 172.21.15.3:0/588047185 >> 172.21.15.3:0/588047185 conn(0x7fdd880b1c20 msgr2=0x7fdd8806d240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:21.585 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.589+0000 7fdd8e534640 1 -- 172.21.15.3:0/588047185 shutdown_connections 2024-05-09T06:36:21.585 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:21.589+0000 7fdd8e534640 1 -- 172.21.15.3:0/588047185 wait complete. 2024-05-09T06:36:21.585 INFO:teuthology.run_tasks:Running task sequential... 2024-05-09T06:36:21.595 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-05-09T06:36:21.595 INFO:teuthology.task.sequential:In sequential, running task print... 2024-05-09T06:36:21.595 INFO:teuthology.task.print:Enabling mgr modules 2024-05-09T06:36:21.595 INFO:teuthology.run_tasks:Running task kclient... 2024-05-09T06:36:21.631 INFO:tasks.kclient:Mounting kernel clients... 2024-05-09T06:36:21.631 INFO:tasks.kclient:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2024-05-09T06:36:21.631 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:36:21.631 DEBUG:teuthology.orchestra.run.smithi003:> ip netns list 2024-05-09T06:36:21.648 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:36:21.649 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip link delete ceph-brx 2024-05-09T06:36:21.717 INFO:teuthology.orchestra.run.smithi003.stderr:Cannot find device "ceph-brx" 2024-05-09T06:36:21.718 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T06:36:21.719 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:36:21.719 DEBUG:teuthology.orchestra.run.smithi003:> ip netns list 2024-05-09T06:36:21.776 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:36:21.777 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip link delete ceph-brx 2024-05-09T06:36:21.843 INFO:teuthology.orchestra.run.smithi003.stderr:Cannot find device "ceph-brx" 2024-05-09T06:36:21.844 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T06:36:21.845 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:36:21.845 DEBUG:teuthology.orchestra.run.smithi190:> ip netns list 2024-05-09T06:36:21.864 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:36:21.864 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip link delete ceph-brx 2024-05-09T06:36:21.937 INFO:teuthology.orchestra.run.smithi190.stderr:Cannot find device "ceph-brx" 2024-05-09T06:36:21.938 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T06:36:21.939 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:36:21.939 DEBUG:teuthology.orchestra.run.smithi190:> ip netns list 2024-05-09T06:36:21.998 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:36:21.998 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip link delete ceph-brx 2024-05-09T06:36:22.067 INFO:teuthology.orchestra.run.smithi190.stderr:Cannot find device "ceph-brx" 2024-05-09T06:36:22.069 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T06:36:22.069 INFO:tasks.kclient:client.0 config is {'syntax': 'v2'} 2024-05-09T06:36:22.069 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-05-09T06:36:22.070 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-09T06:36:22.070 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-09T06:36:22.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:22.194+0000 7fad8886b640 1 Processor -- start 2024-05-09T06:36:22.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:22.194+0000 7fad8886b640 1 -- start start 2024-05-09T06:36:22.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:22.195+0000 7fad8886b640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fad840d27c0 0x7fad840d2c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:22.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:22.195+0000 7fad8886b640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fad84166390 0x7fad84168780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:22.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:22.195+0000 7fad8886b640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fad840d4e50 con 0x7fad8415e480 2024-05-09T06:36:22.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:22.195+0000 7fad8886b640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fad840d3160 con 0x7fad84166390 2024-05-09T06:36:22.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:22.195+0000 7fad8886b640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7fad840d0270 con 0x7fad840d27c0 2024-05-09T06:36:22.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:22.195+0000 7fad82d9d640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fad84166390 0x7fad84168780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:22.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:22.195+0000 7fad82d9d640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fad84166390 0x7fad84168780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50120/0 (socket says 172.21.15.3:50120) 2024-05-09T06:36:22.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:22.195+0000 7fad82d9d640 1 -- 172.21.15.3:0/673811187 learned_addr learned my addr 172.21.15.3:0/673811187 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:36:22.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:22.195+0000 7fad8259c640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fad8415e480 0x7fad840d1dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:47842/0 (socket says 172.21.15.3:47842) 2024-05-09T06:36:22.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:22.196+0000 7fad7bfff640 1 --2- 172.21.15.3:0/673811187 >> v2:172.21.15.190:3300/0 conn(0x7fad840d27c0 0x7fad840d2c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:22.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:22.196+0000 7fad82d9d640 1 -- 172.21.15.3:0/673811187 >> v2:172.21.15.190:3300/0 conn(0x7fad840d27c0 msgr2=0x7fad840d2c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:22.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:22.196+0000 7fad82d9d640 1 --2- 172.21.15.3:0/673811187 >> v2:172.21.15.190:3300/0 conn(0x7fad840d27c0 0x7fad840d2c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:22.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:22.196+0000 7fad82d9d640 1 -- 172.21.15.3:0/673811187 >> v1:172.21.15.3:6789/0 conn(0x7fad8415e480 legacy=0x7fad840d1dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:22.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:22.196+0000 7fad81d9b640 1 -- 172.21.15.3:0/673811187 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3743361075 0 0) 0x7fad74001110 con 0x7fad8415e480 2024-05-09T06:36:22.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:22.196+0000 7fad82d9d640 1 -- 172.21.15.3:0/673811187 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad840cf1d0 con 0x7fad84166390 2024-05-09T06:36:22.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:22.196+0000 7fad82d9d640 1 --2- 172.21.15.3:0/673811187 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fad84166390 0x7fad84168780 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fad840d18c0 tx=0x7fad7c02a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d548534fa996287 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:22.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:22.197+0000 7fad81d9b640 1 -- 172.21.15.3:0/673811187 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad7c036070 con 0x7fad84166390 2024-05-09T06:36:22.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:22.197+0000 7fad81d9b640 1 -- 172.21.15.3:0/673811187 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad7c031600 con 0x7fad84166390 2024-05-09T06:36:22.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:22.197+0000 7fad81d9b640 1 -- 172.21.15.3:0/673811187 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad7c0319d0 con 0x7fad84166390 2024-05-09T06:36:22.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:22.197+0000 7fad8886b640 1 -- 172.21.15.3:0/673811187 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fad84166390 msgr2=0x7fad84168780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:22.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:22.197+0000 7fad8886b640 1 --2- 172.21.15.3:0/673811187 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fad84166390 0x7fad84168780 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fad840d18c0 tx=0x7fad7c02a730 comp rx=0 tx=0).stop 2024-05-09T06:36:22.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:22.197+0000 7fad8886b640 1 -- 172.21.15.3:0/673811187 shutdown_connections 2024-05-09T06:36:22.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:22.197+0000 7fad8886b640 1 --2- 172.21.15.3:0/673811187 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fad84166390 0x7fad84168780 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:22.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:22.197+0000 7fad8886b640 1 --2- 172.21.15.3:0/673811187 >> v2:172.21.15.190:3300/0 conn(0x7fad840d27c0 0x7fad840d2c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:22.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:22.197+0000 7fad8886b640 1 -- 172.21.15.3:0/673811187 >> 172.21.15.3:0/673811187 conn(0x7fad8409d4d0 msgr2=0x7fad840d4200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:22.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:22.197+0000 7fad8886b640 1 -- 172.21.15.3:0/673811187 shutdown_connections 2024-05-09T06:36:22.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:22.197+0000 7fad8886b640 1 -- 172.21.15.3:0/673811187 wait complete. 2024-05-09T06:36:22.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:22.199+0000 7fad8886b640 1 Processor -- start 2024-05-09T06:36:22.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:22.199+0000 7fad8886b640 1 -- start start 2024-05-09T06:36:22.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:22.199+0000 7fad8886b640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fad840d27c0 0x7fad84158080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:22.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:22.199+0000 7fad8886b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fad8415e480 0x7fad84154020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:22.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:22.199+0000 7fad8886b640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fad84166390 0x7fad840b9220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:22.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:22.199+0000 7fad8886b640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fad840cfa70 con 0x7fad8415e480 2024-05-09T06:36:22.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:22.199+0000 7fad8886b640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fad840cf790 con 0x7fad840d27c0 2024-05-09T06:36:22.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:22.199+0000 7fad8886b640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fad840cfd50 con 0x7fad84166390 2024-05-09T06:36:22.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:22.200+0000 7fad7bfff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fad8415e480 0x7fad84154020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:22.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:22.200+0000 7fad7bfff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fad8415e480 0x7fad84154020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50124/0 (socket says 172.21.15.3:50124) 2024-05-09T06:36:22.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:22.200+0000 7fad7bfff640 1 -- 172.21.15.3:0/1637489631 learned_addr learned my addr 172.21.15.3:0/1637489631 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:36:22.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:22.200+0000 7fad8259c640 1 --2- 172.21.15.3:0/1637489631 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fad840d27c0 0x7fad84158080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:22.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:22.200+0000 7fad82d9d640 1 --2- 172.21.15.3:0/1637489631 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fad84166390 0x7fad840b9220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:22.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:22.200+0000 7fad7bfff640 1 -- 172.21.15.3:0/1637489631 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fad84166390 msgr2=0x7fad840b9220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:22.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:22.200+0000 7fad7bfff640 1 --2- 172.21.15.3:0/1637489631 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fad84166390 0x7fad840b9220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:22.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:22.200+0000 7fad7bfff640 1 -- 172.21.15.3:0/1637489631 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fad840d27c0 msgr2=0x7fad84158080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:22.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:22.200+0000 7fad7bfff640 1 --2- 172.21.15.3:0/1637489631 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fad840d27c0 0x7fad84158080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:22.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:22.200+0000 7fad7bfff640 1 -- 172.21.15.3:0/1637489631 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad840caa00 con 0x7fad8415e480 2024-05-09T06:36:22.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:22.200+0000 7fad7bfff640 1 --2- 172.21.15.3:0/1637489631 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fad8415e480 0x7fad84154020 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fad6c00d5d0 tx=0x7fad6c00da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:22.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:22.200+0000 7fad7affd640 1 -- 172.21.15.3:0/1637489631 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad6c004240 con 0x7fad8415e480 2024-05-09T06:36:22.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:22.200+0000 7fad7affd640 1 -- 172.21.15.3:0/1637489631 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad6c0043e0 con 0x7fad8415e480 2024-05-09T06:36:22.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:22.200+0000 7fad8886b640 1 -- 172.21.15.3:0/1637489631 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad840b8740 con 0x7fad8415e480 2024-05-09T06:36:22.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:22.200+0000 7fad7affd640 1 -- 172.21.15.3:0/1637489631 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad6c003830 con 0x7fad8415e480 2024-05-09T06:36:22.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:22.200+0000 7fad8886b640 1 -- 172.21.15.3:0/1637489631 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad840b8d00 con 0x7fad8415e480 2024-05-09T06:36:22.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:22.201+0000 7fad8886b640 1 -- 172.21.15.3:0/1637489631 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad841545c0 con 0x7fad8415e480 2024-05-09T06:36:22.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:22.202+0000 7fad7affd640 1 -- 172.21.15.3:0/1637489631 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fad6c0039d0 con 0x7fad8415e480 2024-05-09T06:36:22.185 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:22.202+0000 7fad7affd640 1 --2- 172.21.15.3:0/1637489631 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fad6406be30 0x7fad6406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:22.185 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:22.202+0000 7fad7affd640 1 -- 172.21.15.3:0/1637489631 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fad6c08afc0 con 0x7fad8415e480 2024-05-09T06:36:22.185 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:22.203+0000 7fad8259c640 1 --2- 172.21.15.3:0/1637489631 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fad6406be30 0x7fad6406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:22.185 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:22.203+0000 7fad8259c640 1 --2- 172.21.15.3:0/1637489631 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fad6406be30 0x7fad6406e2f0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fad74000f60 tx=0x7fad74007b50 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:22.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:22.207+0000 7fad7affd640 1 -- 172.21.15.3:0/1637489631 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fad6c059320 con 0x7fad8415e480 2024-05-09T06:36:22.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:22.429+0000 7fad8886b640 1 -- 172.21.15.3:0/1637489631 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fad840d18f0 con 0x7fad8415e480 2024-05-09T06:36:22.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:22.429+0000 7fad7affd640 1 -- 172.21.15.3:0/1637489631 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) ==== 53+0+73 (secure 0 0 0) 0x7fad6c05d620 con 0x7fad8415e480 2024-05-09T06:36:22.410 INFO:teuthology.orchestra.run.smithi003.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-09T06:36:22.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:22.431+0000 7fad8886b640 1 -- 172.21.15.3:0/1637489631 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fad6406be30 msgr2=0x7fad6406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:22.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:22.431+0000 7fad8886b640 1 --2- 172.21.15.3:0/1637489631 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fad6406be30 0x7fad6406e2f0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fad74000f60 tx=0x7fad74007b50 comp rx=0 tx=0).stop 2024-05-09T06:36:22.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:22.431+0000 7fad8886b640 1 -- 172.21.15.3:0/1637489631 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fad8415e480 msgr2=0x7fad84154020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:22.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:22.431+0000 7fad8886b640 1 --2- 172.21.15.3:0/1637489631 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fad8415e480 0x7fad84154020 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fad6c00d5d0 tx=0x7fad6c00da90 comp rx=0 tx=0).stop 2024-05-09T06:36:22.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:22.431+0000 7fad8886b640 1 -- 172.21.15.3:0/1637489631 shutdown_connections 2024-05-09T06:36:22.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:22.431+0000 7fad8886b640 1 --2- 172.21.15.3:0/1637489631 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fad6406be30 0x7fad6406e2f0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:22.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:22.431+0000 7fad8886b640 1 --2- 172.21.15.3:0/1637489631 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fad84166390 0x7fad840b9220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:22.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:22.431+0000 7fad8886b640 1 --2- 172.21.15.3:0/1637489631 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fad8415e480 0x7fad84154020 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:22.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:22.431+0000 7fad8886b640 1 --2- 172.21.15.3:0/1637489631 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fad840d27c0 0x7fad84158080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:22.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:22.431+0000 7fad8886b640 1 -- 172.21.15.3:0/1637489631 >> 172.21.15.3:0/1637489631 conn(0x7fad8409d4d0 msgr2=0x7fad840d4200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:22.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:22.431+0000 7fad8886b640 1 -- 172.21.15.3:0/1637489631 shutdown_connections 2024-05-09T06:36:22.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:22.431+0000 7fad8886b640 1 -- 172.21.15.3:0/1637489631 wait complete. 2024-05-09T06:36:22.423 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-09T06:36:22.423 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-09T06:36:22.423 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi003.front.sepia.ceph.com 2024-05-09T06:36:22.423 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-09T06:36:22.423 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-09T06:36:22.423 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-09T06:36:22.423 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-09T06:36:22.423 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-09T06:36:22.423 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-09T06:36:22.424 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:36:22.424 DEBUG:teuthology.orchestra.run.smithi003:> ip addr 2024-05-09T06:36:22.440 INFO:teuthology.orchestra.run.smithi003.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-09T06:36:22.440 INFO:teuthology.orchestra.run.smithi003.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-09T06:36:22.440 INFO:teuthology.orchestra.run.smithi003.stdout: inet 127.0.0.1/8 scope host lo 2024-05-09T06:36:22.440 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-05-09T06:36:22.440 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 ::1/128 scope host 2024-05-09T06:36:22.440 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-05-09T06:36:22.440 INFO:teuthology.orchestra.run.smithi003.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-09T06:36:22.440 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:6b:fd:ac brd ff:ff:ff:ff:ff:ff 2024-05-09T06:36:22.440 INFO:teuthology.orchestra.run.smithi003.stdout: altname enp7s0f0 2024-05-09T06:36:22.440 INFO:teuthology.orchestra.run.smithi003.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-09T06:36:22.441 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:6b:fd:ad brd ff:ff:ff:ff:ff:ff 2024-05-09T06:36:22.441 INFO:teuthology.orchestra.run.smithi003.stdout: altname enp7s0f1 2024-05-09T06:36:22.441 INFO:teuthology.orchestra.run.smithi003.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-09T06:36:22.441 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:bd:15:dc brd ff:ff:ff:ff:ff:ff 2024-05-09T06:36:22.441 INFO:teuthology.orchestra.run.smithi003.stdout: altname enp1s0f0 2024-05-09T06:36:22.441 INFO:teuthology.orchestra.run.smithi003.stdout: inet 172.21.15.3/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-05-09T06:36:22.441 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft 42694sec preferred_lft 42694sec 2024-05-09T06:36:22.441 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 fe80::ec4:7aff:febd:15dc/64 scope link noprefixroute 2024-05-09T06:36:22.441 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-05-09T06:36:22.441 INFO:teuthology.orchestra.run.smithi003.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-09T06:36:22.441 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:bd:15:dd brd ff:ff:ff:ff:ff:ff 2024-05-09T06:36:22.441 INFO:teuthology.orchestra.run.smithi003.stdout: altname enp1s0f1 2024-05-09T06:36:22.442 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-05-09T06:36:22.442 DEBUG:teuthology.orchestra.run.smithi003:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-09T06:36:22.442 DEBUG:teuthology.orchestra.run.smithi003:> set -e 2024-05-09T06:36:22.442 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip link add name ceph-brx type bridge 2024-05-09T06:36:22.442 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip addr flush dev ceph-brx 2024-05-09T06:36:22.442 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip link set ceph-brx up 2024-05-09T06:36:22.442 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-05-09T06:36:22.442 DEBUG:teuthology.orchestra.run.smithi003:> ') 2024-05-09T06:36:22.531 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:22 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-09T06:36:22.621 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T06:36:22.628 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:36:22.628 DEBUG:teuthology.orchestra.run.smithi003:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-09T06:36:22.654 INFO:teuthology.orchestra.run.smithi003.stdout:1 2024-05-09T06:36:22.656 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:36:22.656 DEBUG:teuthology.orchestra.run.smithi003:> route 2024-05-09T06:36:22.731 INFO:teuthology.orchestra.run.smithi003.stdout:Kernel IP routing table 2024-05-09T06:36:22.731 INFO:teuthology.orchestra.run.smithi003.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-09T06:36:22.731 INFO:teuthology.orchestra.run.smithi003.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-05-09T06:36:22.731 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-05-09T06:36:22.731 INFO:teuthology.orchestra.run.smithi003.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-09T06:36:22.731 DEBUG:teuthology.orchestra.run.smithi003:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-09T06:36:22.732 DEBUG:teuthology.orchestra.run.smithi003:> set -e 2024-05-09T06:36:22.732 DEBUG:teuthology.orchestra.run.smithi003:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-05-09T06:36:22.732 DEBUG:teuthology.orchestra.run.smithi003:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-05-09T06:36:22.732 DEBUG:teuthology.orchestra.run.smithi003:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-05-09T06:36:22.732 DEBUG:teuthology.orchestra.run.smithi003:> ') 2024-05-09T06:36:22.821 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:22 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-09T06:36:23.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:23 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T06:36:23.014 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:36:23.014 DEBUG:teuthology.orchestra.run.smithi003:> ip netns list 2024-05-09T06:36:23.031 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:36:23.031 DEBUG:teuthology.orchestra.run.smithi003:> ip netns list-id 2024-05-09T06:36:23.089 DEBUG:teuthology.orchestra.run.smithi003:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-09T06:36:23.089 DEBUG:teuthology.orchestra.run.smithi003:> set -e 2024-05-09T06:36:23.089 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-09T06:36:23.090 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-05-09T06:36:23.090 DEBUG:teuthology.orchestra.run.smithi003:> ') 2024-05-09T06:36:23.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:23 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-09T06:36:23.207 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:23 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T06:36:23.213 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-05-09T06:36:23.214 DEBUG:teuthology.orchestra.run.smithi003:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-09T06:36:23.214 DEBUG:teuthology.orchestra.run.smithi003:> set -e 2024-05-09T06:36:23.214 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-05-09T06:36:23.214 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-05-09T06:36:23.214 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-05-09T06:36:23.214 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-05-09T06:36:23.214 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-05-09T06:36:23.214 DEBUG:teuthology.orchestra.run.smithi003:> ') 2024-05-09T06:36:23.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:23 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-09T06:36:23.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:23 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T06:36:23.354 DEBUG:teuthology.orchestra.run.smithi003:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-09T06:36:23.354 DEBUG:teuthology.orchestra.run.smithi003:> set -e 2024-05-09T06:36:23.354 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip link set brx.0 up 2024-05-09T06:36:23.354 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip link set dev brx.0 master ceph-brx 2024-05-09T06:36:23.354 DEBUG:teuthology.orchestra.run.smithi003:> ') 2024-05-09T06:36:23.404 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:23 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-09T06:36:23.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:23 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T06:36:23.434 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-09T06:36:23.435 DEBUG:teuthology.orchestra.run.smithi003:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-09T06:36:23.450 INFO:teuthology.orchestra.run.smithi003.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-09T06:36:23.451 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-09T06:36:23.451 DEBUG:teuthology.orchestra.run.smithi003:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-09T06:36:23.506 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-05-09T06:36:23.506 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:36:23.507 DEBUG:teuthology.orchestra.run.smithi003:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-05-09T06:36:23.598 INFO:teuthology.orchestra.run.smithi003.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-05-09T06:36:23.599 INFO:teuthology.orchestra.run.smithi003.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-05-09T06:36:23.730 INFO:teuthology.orchestra.run.smithi003.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-05-09T06:36:23.730 INFO:teuthology.orchestra.run.smithi003.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-05-09T06:36:23.731 INFO:teuthology.orchestra.run.smithi003.stdout:mount.ceph: resolved to: "172.21.15.3:3300,172.21.15.190:3300,172.21.15.190:3301" 2024-05-09T06:36:23.731 INFO:teuthology.orchestra.run.smithi003.stdout:mount.ceph: trying mount with new device syntax: 0@16ed737d-2665-431a-8ac6-ef242be3975a.cephfs=/ 2024-05-09T06:36:23.731 INFO:teuthology.orchestra.run.smithi003.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.3:3300/172.21.15.190:3300/172.21.15.190:3301" will pass to kernel 2024-05-09T06:36:23.731 INFO:teuthology.orchestra.run.smithi003.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-05-09T06:36:23.731 INFO:teuthology.orchestra.run.smithi003.stdout: You just mounted a file system that supports labels which does not 2024-05-09T06:36:23.731 INFO:teuthology.orchestra.run.smithi003.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-05-09T06:36:23.731 INFO:teuthology.orchestra.run.smithi003.stdout: applications will generate AVC messages and not be allowed access to 2024-05-09T06:36:23.731 INFO:teuthology.orchestra.run.smithi003.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-05-09T06:36:23.733 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-09T06:36:23.733 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:36:23.733 DEBUG:teuthology.orchestra.run.smithi003:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-09T06:36:23.760 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-09T06:36:23.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:23 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-09T06:36:23.860 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:23 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T06:36:23.867 INFO:teuthology.orchestra.run.smithi003.stdout:client4618 2024-05-09T06:36:23.867 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-05-09T06:36:23.867 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T06:36:23.884 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-09T06:36:23.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:23 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-09T06:36:23.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T06:36:23.989 INFO:teuthology.orchestra.run.smithi003.stdout:client4618 2024-05-09T06:36:23.989 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-09T06:36:24.041 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:24 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-09T06:36:24.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T06:36:24.053 INFO:teuthology.orchestra.run.smithi003.stdout:client4618 2024-05-09T06:36:24.054 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/16ed737d-2665-431a-8ac6-ef242be3975a.client4618/status') 2024-05-09T06:36:24.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:24 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-09T06:36:24.119 INFO:teuthology.orchestra.run.smithi003.stdout:instance: client.4618 (3)172.21.15.3:0/3907783175 2024-05-09T06:36:24.119 INFO:teuthology.orchestra.run.smithi003.stdout:blocklisted: false 2024-05-09T06:36:24.119 INFO:teuthology.orchestra.run.smithi003.stderr:0+1 records in 2024-05-09T06:36:24.119 INFO:teuthology.orchestra.run.smithi003.stderr:0+1 records out 2024-05-09T06:36:24.119 INFO:teuthology.orchestra.run.smithi003.stderr:69 bytes copied, 2.7149e-05 s, 2.5 MB/s 2024-05-09T06:36:24.120 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T06:36:24.126 INFO:tasks.kclient:client.2 config is {'syntax': 'v2'} 2024-05-09T06:36:24.127 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-05-09T06:36:24.127 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-05-09T06:36:24.127 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-09T06:36:24.231 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:24.250+0000 7f2dd8b9e640 1 Processor -- start 2024-05-09T06:36:24.231 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:24.250+0000 7f2dd8b9e640 1 -- start start 2024-05-09T06:36:24.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:24.251+0000 7f2dd8b9e640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2dd4161c60 0x7f2dd40d24e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:24.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:24.251+0000 7f2dd8b9e640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f2dd40b76c0 0x7f2dd40b7b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:24.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:24.251+0000 7f2dd8b9e640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2dd40cf6a0 con 0x7f2dd4161880 2024-05-09T06:36:24.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:24.251+0000 7f2dd8b9e640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f2dd40b6ee0 con 0x7f2dd40b76c0 2024-05-09T06:36:24.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:24.251+0000 7f2dd8b9e640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f2dd40cf940 con 0x7f2dd4161c60 2024-05-09T06:36:24.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:24.251+0000 7f2dd3fff640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f2dd40b76c0 0x7f2dd40b7b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:24.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:24.251+0000 7f2dd37fe640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f2dd4161880 0x7f2dd40d1dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:47844/0 (socket says 172.21.15.3:47844) 2024-05-09T06:36:24.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:24.252+0000 7f2dd37fe640 1 -- 172.21.15.3:0/3481379770 learned_addr learned my addr 172.21.15.3:0/3481379770 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-05-09T06:36:24.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:24.252+0000 7f2dd2ffd640 1 --2- 172.21.15.3:0/3481379770 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2dd4161c60 0x7f2dd40d24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:24.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:24.252+0000 7f2dd3fff640 1 -- 172.21.15.3:0/3481379770 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2dd4161c60 msgr2=0x7f2dd40d24e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:24.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:24.252+0000 7f2dd3fff640 1 --2- 172.21.15.3:0/3481379770 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2dd4161c60 0x7f2dd40d24e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:24.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:24.252+0000 7f2dd3fff640 1 -- 172.21.15.3:0/3481379770 >> v1:172.21.15.3:6789/0 conn(0x7f2dd4161880 legacy=0x7f2dd40d1dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:24.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:24.252+0000 7f2dd3fff640 1 -- 172.21.15.3:0/3481379770 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2dd40cfc20 con 0x7f2dd40b76c0 2024-05-09T06:36:24.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:24.252+0000 7f2dd27fc640 1 -- 172.21.15.3:0/3481379770 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1995302982 0 0) 0x7f2dd40cf6a0 con 0x7f2dd4161880 2024-05-09T06:36:24.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:24.253+0000 7f2dd3fff640 1 --2- 172.21.15.3:0/3481379770 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2dd40b76c0 0x7f2dd40b7b50 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f2dcc009d90 tx=0x7f2dcc02a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab450ee1d896b033 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:24.235 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:24.253+0000 7f2dd27fc640 1 -- 172.21.15.3:0/3481379770 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dcc007740 con 0x7f2dd40b76c0 2024-05-09T06:36:24.235 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:24.253+0000 7f2dd27fc640 1 -- 172.21.15.3:0/3481379770 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2dcc0078e0 con 0x7f2dd40b76c0 2024-05-09T06:36:24.235 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:24.253+0000 7f2dd27fc640 1 -- 172.21.15.3:0/3481379770 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dcc007cb0 con 0x7f2dd40b76c0 2024-05-09T06:36:24.235 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:24.253+0000 7f2dd8b9e640 1 -- 172.21.15.3:0/3481379770 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2dd40b76c0 msgr2=0x7f2dd40b7b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:24.236 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:24.253+0000 7f2dd8b9e640 1 --2- 172.21.15.3:0/3481379770 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2dd40b76c0 0x7f2dd40b7b50 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f2dcc009d90 tx=0x7f2dcc02a660 comp rx=0 tx=0).stop 2024-05-09T06:36:24.236 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:24.253+0000 7f2dd8b9e640 1 -- 172.21.15.3:0/3481379770 shutdown_connections 2024-05-09T06:36:24.236 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:24.253+0000 7f2dd8b9e640 1 --2- 172.21.15.3:0/3481379770 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2dd40b76c0 0x7f2dd40b7b50 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:24.236 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:24.253+0000 7f2dd8b9e640 1 --2- 172.21.15.3:0/3481379770 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2dd4161c60 0x7f2dd40d24e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:24.236 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:24.253+0000 7f2dd8b9e640 1 -- 172.21.15.3:0/3481379770 >> 172.21.15.3:0/3481379770 conn(0x7f2dd40d3ff0 msgr2=0x7f2dd40d4400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:24.236 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:24.253+0000 7f2dd8b9e640 1 -- 172.21.15.3:0/3481379770 shutdown_connections 2024-05-09T06:36:24.236 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:24.253+0000 7f2dd8b9e640 1 -- 172.21.15.3:0/3481379770 wait complete. 2024-05-09T06:36:24.236 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:24.255+0000 7f2dd8b9e640 1 Processor -- start 2024-05-09T06:36:24.236 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:24.255+0000 7f2dd8b9e640 1 -- start start 2024-05-09T06:36:24.238 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:24.256+0000 7f2dd8b9e640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2dd40b76c0 0x7f2dd4151bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:24.238 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:24.256+0000 7f2dd8b9e640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2dd4161880 0x7f2dd4159470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:24.238 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:24.256+0000 7f2dd8b9e640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2dd4161c60 0x7f2dd41522f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:24.238 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:24.256+0000 7f2dd8b9e640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f2dd40b5240 con 0x7f2dd40b76c0 2024-05-09T06:36:24.238 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:24.256+0000 7f2dd8b9e640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f2dd40cff00 con 0x7f2dd4161880 2024-05-09T06:36:24.238 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:24.256+0000 7f2dd8b9e640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f2dd40d1630 con 0x7f2dd4161c60 2024-05-09T06:36:24.238 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:24.256+0000 7f2dd37fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2dd40b76c0 0x7f2dd4151bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:24.239 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:24.256+0000 7f2dd37fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2dd40b76c0 0x7f2dd4151bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50140/0 (socket says 172.21.15.3:50140) 2024-05-09T06:36:24.239 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:24.256+0000 7f2dd37fe640 1 -- 172.21.15.3:0/1657211814 learned_addr learned my addr 172.21.15.3:0/1657211814 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:36:24.239 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:24.256+0000 7f2dd3fff640 1 --2- 172.21.15.3:0/1657211814 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2dd4161c60 0x7f2dd41522f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:24.239 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:24.256+0000 7f2dd2ffd640 1 --2- 172.21.15.3:0/1657211814 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2dd4161880 0x7f2dd4159470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:24.239 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:24.256+0000 7f2dd37fe640 1 -- 172.21.15.3:0/1657211814 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2dd4161c60 msgr2=0x7f2dd41522f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:24.239 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:24.256+0000 7f2dd37fe640 1 --2- 172.21.15.3:0/1657211814 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2dd4161c60 0x7f2dd41522f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:24.239 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:24.256+0000 7f2dd37fe640 1 -- 172.21.15.3:0/1657211814 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2dd4161880 msgr2=0x7f2dd4159470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:24.239 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:24.256+0000 7f2dd37fe640 1 --2- 172.21.15.3:0/1657211814 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2dd4161880 0x7f2dd4159470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:24.239 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:24.256+0000 7f2dd37fe640 1 -- 172.21.15.3:0/1657211814 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2dd40c8f50 con 0x7f2dd40b76c0 2024-05-09T06:36:24.239 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:24.256+0000 7f2dd37fe640 1 --2- 172.21.15.3:0/1657211814 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2dd40b76c0 0x7f2dd4151bd0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f2dc400c4d0 tx=0x7f2dc400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:24.239 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:24.256+0000 7f2dbbfff640 1 -- 172.21.15.3:0/1657211814 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dc4018070 con 0x7f2dd40b76c0 2024-05-09T06:36:24.239 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:24.256+0000 7f2dbbfff640 1 -- 172.21.15.3:0/1657211814 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2dc4007500 con 0x7f2dd40b76c0 2024-05-09T06:36:24.239 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:24.256+0000 7f2dd8b9e640 1 -- 172.21.15.3:0/1657211814 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2dd40c91f0 con 0x7f2dd40b76c0 2024-05-09T06:36:24.239 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:24.256+0000 7f2dbbfff640 1 -- 172.21.15.3:0/1657211814 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dc4004a20 con 0x7f2dd40b76c0 2024-05-09T06:36:24.239 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:24.256+0000 7f2dd8b9e640 1 -- 172.21.15.3:0/1657211814 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2dd40c97b0 con 0x7f2dd40b76c0 2024-05-09T06:36:24.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:24.257+0000 7f2dd8b9e640 1 -- 172.21.15.3:0/1657211814 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2dd404f5f0 con 0x7f2dd40b76c0 2024-05-09T06:36:24.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:24.258+0000 7f2dbbfff640 1 -- 172.21.15.3:0/1657211814 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2dc401b070 con 0x7f2dd40b76c0 2024-05-09T06:36:24.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:24.258+0000 7f2dbbfff640 1 --2- 172.21.15.3:0/1657211814 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f2dc006be30 0x7f2dc006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:24.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:24.259+0000 7f2dbbfff640 1 -- 172.21.15.3:0/1657211814 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f2dc408a100 con 0x7f2dd40b76c0 2024-05-09T06:36:24.241 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:24.259+0000 7f2dd2ffd640 1 --2- 172.21.15.3:0/1657211814 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f2dc006be30 0x7f2dc006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:24.241 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:24.259+0000 7f2dd2ffd640 1 --2- 172.21.15.3:0/1657211814 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f2dc006be30 0x7f2dc006e2f0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f2dd40b4ed0 tx=0x7f2dc8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:24.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:24.263+0000 7f2dbbfff640 1 -- 172.21.15.3:0/1657211814 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f2dc4058460 con 0x7f2dd40b76c0 2024-05-09T06:36:24.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:24.486+0000 7f2dd8b9e640 1 -- 172.21.15.3:0/1657211814 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f2dd40d1950 con 0x7f2dd40b76c0 2024-05-09T06:36:24.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:24.486+0000 7f2dbbfff640 1 -- 172.21.15.3:0/1657211814 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) ==== 53+0+73 (secure 0 0 0) 0x7f2dc405c760 con 0x7f2dd40b76c0 2024-05-09T06:36:24.467 INFO:teuthology.orchestra.run.smithi003.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-09T06:36:24.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:24.488+0000 7f2dd8b9e640 1 -- 172.21.15.3:0/1657211814 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f2dc006be30 msgr2=0x7f2dc006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:24.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:24.488+0000 7f2dd8b9e640 1 --2- 172.21.15.3:0/1657211814 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f2dc006be30 0x7f2dc006e2f0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f2dd40b4ed0 tx=0x7f2dc8008040 comp rx=0 tx=0).stop 2024-05-09T06:36:24.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:24.488+0000 7f2dd8b9e640 1 -- 172.21.15.3:0/1657211814 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2dd40b76c0 msgr2=0x7f2dd4151bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:24.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:24.488+0000 7f2dd8b9e640 1 --2- 172.21.15.3:0/1657211814 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2dd40b76c0 0x7f2dd4151bd0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f2dc400c4d0 tx=0x7f2dc400c990 comp rx=0 tx=0).stop 2024-05-09T06:36:24.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:24.488+0000 7f2dd8b9e640 1 -- 172.21.15.3:0/1657211814 shutdown_connections 2024-05-09T06:36:24.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:24.488+0000 7f2dd8b9e640 1 --2- 172.21.15.3:0/1657211814 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f2dc006be30 0x7f2dc006e2f0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:24.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:24.488+0000 7f2dd8b9e640 1 --2- 172.21.15.3:0/1657211814 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2dd4161c60 0x7f2dd41522f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:24.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:24.488+0000 7f2dd8b9e640 1 --2- 172.21.15.3:0/1657211814 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2dd4161880 0x7f2dd4159470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:24.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:24.488+0000 7f2dd8b9e640 1 --2- 172.21.15.3:0/1657211814 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2dd40b76c0 0x7f2dd4151bd0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:24.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:24.488+0000 7f2dd8b9e640 1 -- 172.21.15.3:0/1657211814 >> 172.21.15.3:0/1657211814 conn(0x7f2dd40d3ff0 msgr2=0x7f2dd40d0900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:24.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:24.489+0000 7f2dd8b9e640 1 -- 172.21.15.3:0/1657211814 shutdown_connections 2024-05-09T06:36:24.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:24.489+0000 7f2dd8b9e640 1 -- 172.21.15.3:0/1657211814 wait complete. 2024-05-09T06:36:24.482 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-09T06:36:24.482 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-09T06:36:24.482 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi003.front.sepia.ceph.com 2024-05-09T06:36:24.482 INFO:tasks.cephfs.mount:self.client.name = client.2 2024-05-09T06:36:24.483 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-05-09T06:36:24.483 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-09T06:36:24.483 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-09T06:36:24.483 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-09T06:36:24.483 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2024-05-09T06:36:24.483 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:36:24.483 DEBUG:teuthology.orchestra.run.smithi003:> ip addr 2024-05-09T06:36:24.500 INFO:teuthology.orchestra.run.smithi003.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-09T06:36:24.500 INFO:teuthology.orchestra.run.smithi003.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-09T06:36:24.500 INFO:teuthology.orchestra.run.smithi003.stdout: inet 127.0.0.1/8 scope host lo 2024-05-09T06:36:24.500 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-05-09T06:36:24.500 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 ::1/128 scope host 2024-05-09T06:36:24.500 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-05-09T06:36:24.500 INFO:teuthology.orchestra.run.smithi003.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-09T06:36:24.500 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:6b:fd:ac brd ff:ff:ff:ff:ff:ff 2024-05-09T06:36:24.500 INFO:teuthology.orchestra.run.smithi003.stdout: altname enp7s0f0 2024-05-09T06:36:24.500 INFO:teuthology.orchestra.run.smithi003.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-09T06:36:24.500 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:6b:fd:ad brd ff:ff:ff:ff:ff:ff 2024-05-09T06:36:24.501 INFO:teuthology.orchestra.run.smithi003.stdout: altname enp7s0f1 2024-05-09T06:36:24.501 INFO:teuthology.orchestra.run.smithi003.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-09T06:36:24.501 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:bd:15:dc brd ff:ff:ff:ff:ff:ff 2024-05-09T06:36:24.501 INFO:teuthology.orchestra.run.smithi003.stdout: altname enp1s0f0 2024-05-09T06:36:24.501 INFO:teuthology.orchestra.run.smithi003.stdout: inet 172.21.15.3/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-05-09T06:36:24.501 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft 42692sec preferred_lft 42692sec 2024-05-09T06:36:24.501 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 fe80::ec4:7aff:febd:15dc/64 scope link noprefixroute 2024-05-09T06:36:24.501 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-05-09T06:36:24.501 INFO:teuthology.orchestra.run.smithi003.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-09T06:36:24.501 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:bd:15:dd brd ff:ff:ff:ff:ff:ff 2024-05-09T06:36:24.501 INFO:teuthology.orchestra.run.smithi003.stdout: altname enp1s0f1 2024-05-09T06:36:24.501 INFO:teuthology.orchestra.run.smithi003.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-09T06:36:24.501 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether e2:25:5f:8c:a9:de brd ff:ff:ff:ff:ff:ff 2024-05-09T06:36:24.502 INFO:teuthology.orchestra.run.smithi003.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-09T06:36:24.502 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-05-09T06:36:24.502 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 fe80::7079:baff:fe57:1840/64 scope link 2024-05-09T06:36:24.502 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-05-09T06:36:24.502 INFO:teuthology.orchestra.run.smithi003.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-09T06:36:24.502 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether e2:25:5f:8c:a9:de brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-09T06:36:24.502 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 fe80::e025:5fff:fe8c:a9de/64 scope link tentative 2024-05-09T06:36:24.502 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-05-09T06:36:24.502 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:36:24.502 DEBUG:teuthology.orchestra.run.smithi003:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-09T06:36:24.565 INFO:teuthology.orchestra.run.smithi003.stdout:1 2024-05-09T06:36:24.567 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:36:24.567 DEBUG:teuthology.orchestra.run.smithi003:> route 2024-05-09T06:36:24.625 INFO:teuthology.orchestra.run.smithi003.stdout:Kernel IP routing table 2024-05-09T06:36:24.625 INFO:teuthology.orchestra.run.smithi003.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-09T06:36:24.625 INFO:teuthology.orchestra.run.smithi003.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-05-09T06:36:24.625 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-05-09T06:36:24.625 INFO:teuthology.orchestra.run.smithi003.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-09T06:36:24.626 DEBUG:teuthology.orchestra.run.smithi003:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-09T06:36:24.626 DEBUG:teuthology.orchestra.run.smithi003:> set -e 2024-05-09T06:36:24.626 DEBUG:teuthology.orchestra.run.smithi003:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-05-09T06:36:24.626 DEBUG:teuthology.orchestra.run.smithi003:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-05-09T06:36:24.626 DEBUG:teuthology.orchestra.run.smithi003:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-05-09T06:36:24.626 DEBUG:teuthology.orchestra.run.smithi003:> ') 2024-05-09T06:36:24.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:24 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-09T06:36:24.750 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T06:36:24.756 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:36:24.756 DEBUG:teuthology.orchestra.run.smithi003:> ip netns list 2024-05-09T06:36:24.773 INFO:teuthology.orchestra.run.smithi003.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-09T06:36:24.773 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:36:24.774 DEBUG:teuthology.orchestra.run.smithi003:> ip netns list-id 2024-05-09T06:36:24.830 INFO:teuthology.orchestra.run.smithi003.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2024-05-09T06:36:24.830 DEBUG:teuthology.orchestra.run.smithi003:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-09T06:36:24.830 DEBUG:teuthology.orchestra.run.smithi003:> set -e 2024-05-09T06:36:24.830 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-09T06:36:24.830 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2024-05-09T06:36:24.830 DEBUG:teuthology.orchestra.run.smithi003:> ') 2024-05-09T06:36:24.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:24 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-09T06:36:24.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T06:36:24.951 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:36:24.951 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-05-09T06:36:24.977 INFO:teuthology.orchestra.run.smithi003.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-09T06:36:24.977 INFO:teuthology.orchestra.run.smithi003.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-09T06:36:24.977 INFO:teuthology.orchestra.run.smithi003.stdout: inet 127.0.0.1/8 scope host lo 2024-05-09T06:36:24.977 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-05-09T06:36:24.977 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 ::1/128 scope host 2024-05-09T06:36:24.977 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-05-09T06:36:24.977 INFO:teuthology.orchestra.run.smithi003.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-09T06:36:24.977 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether ae:ec:b8:a8:12:9a brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-05-09T06:36:24.978 INFO:teuthology.orchestra.run.smithi003.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-05-09T06:36:24.978 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-05-09T06:36:24.978 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 fe80::acec:b8ff:fea8:129a/64 scope link 2024-05-09T06:36:24.978 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-05-09T06:36:24.979 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:36:24.979 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-05-09T06:36:25.044 INFO:teuthology.orchestra.run.smithi003.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-09T06:36:25.044 INFO:teuthology.orchestra.run.smithi003.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-09T06:36:25.044 INFO:teuthology.orchestra.run.smithi003.stdout: inet 127.0.0.1/8 scope host lo 2024-05-09T06:36:25.044 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-05-09T06:36:25.044 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 ::1/128 scope host 2024-05-09T06:36:25.044 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-05-09T06:36:25.045 INFO:teuthology.orchestra.run.smithi003.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-09T06:36:25.045 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether ae:ec:b8:a8:12:9a brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-05-09T06:36:25.045 INFO:teuthology.orchestra.run.smithi003.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-05-09T06:36:25.045 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-05-09T06:36:25.045 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 fe80::acec:b8ff:fea8:129a/64 scope link 2024-05-09T06:36:25.045 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-05-09T06:36:25.046 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2024-05-09T06:36:25.046 DEBUG:teuthology.orchestra.run.smithi003:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-09T06:36:25.046 DEBUG:teuthology.orchestra.run.smithi003:> set -e 2024-05-09T06:36:25.046 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2024-05-09T06:36:25.046 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-05-09T06:36:25.046 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2024-05-09T06:36:25.047 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2024-05-09T06:36:25.047 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2024-05-09T06:36:25.047 DEBUG:teuthology.orchestra.run.smithi003:> ') 2024-05-09T06:36:25.137 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:25 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-09T06:36:25.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T06:36:25.204 DEBUG:teuthology.orchestra.run.smithi003:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-09T06:36:25.205 DEBUG:teuthology.orchestra.run.smithi003:> set -e 2024-05-09T06:36:25.205 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip link set brx.1 up 2024-05-09T06:36:25.205 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip link set dev brx.1 master ceph-brx 2024-05-09T06:36:25.205 DEBUG:teuthology.orchestra.run.smithi003:> ') 2024-05-09T06:36:25.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:25 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-09T06:36:25.278 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T06:36:25.286 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-09T06:36:25.286 DEBUG:teuthology.orchestra.run.smithi003:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2024-05-09T06:36:25.303 INFO:teuthology.orchestra.run.smithi003.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2024-05-09T06:36:25.303 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-09T06:36:25.303 DEBUG:teuthology.orchestra.run.smithi003:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2024-05-09T06:36:25.358 INFO:tasks.cephfs.kernel_mount:mounting using device: 2@.cephfs=/ 2024-05-09T06:36:25.359 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:36:25.359 DEBUG:teuthology.orchestra.run.smithi003:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 2@.cephfs=/ /home/ubuntu/cephtest/mnt.2 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-05-09T06:36:25.444 INFO:teuthology.orchestra.run.smithi003.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-05-09T06:36:25.444 INFO:teuthology.orchestra.run.smithi003.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-05-09T06:36:25.457 INFO:teuthology.orchestra.run.smithi003.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-05-09T06:36:25.457 INFO:teuthology.orchestra.run.smithi003.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-05-09T06:36:25.457 INFO:teuthology.orchestra.run.smithi003.stdout:mount.ceph: resolved to: "172.21.15.3:3300,172.21.15.190:3300,172.21.15.190:3301" 2024-05-09T06:36:25.457 INFO:teuthology.orchestra.run.smithi003.stdout:mount.ceph: trying mount with new device syntax: 2@16ed737d-2665-431a-8ac6-ef242be3975a.cephfs=/ 2024-05-09T06:36:25.457 INFO:teuthology.orchestra.run.smithi003.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,ms_mode=prefer-crc,key=2,mon_addr=172.21.15.3:3300/172.21.15.190:3300/172.21.15.190:3301" will pass to kernel 2024-05-09T06:36:25.458 INFO:teuthology.orchestra.run.smithi003.stdout:mount: /home/ubuntu/cephtest/mnt.2 does not contain SELinux labels. 2024-05-09T06:36:25.458 INFO:teuthology.orchestra.run.smithi003.stdout: You just mounted a file system that supports labels which does not 2024-05-09T06:36:25.458 INFO:teuthology.orchestra.run.smithi003.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-05-09T06:36:25.458 INFO:teuthology.orchestra.run.smithi003.stdout: applications will generate AVC messages and not be allowed access to 2024-05-09T06:36:25.458 INFO:teuthology.orchestra.run.smithi003.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-05-09T06:36:25.460 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-09T06:36:25.460 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:36:25.460 DEBUG:teuthology.orchestra.run.smithi003:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2024-05-09T06:36:25.485 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-09T06:36:25.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:25 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-09T06:36:25.579 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T06:36:25.585 INFO:teuthology.orchestra.run.smithi003.stdout:client4683 2024-05-09T06:36:25.585 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-05-09T06:36:25.585 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T06:36:25.602 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-09T06:36:25.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:25 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-09T06:36:25.695 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T06:36:25.701 INFO:teuthology.orchestra.run.smithi003.stdout:client4683 2024-05-09T06:36:25.702 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-09T06:36:25.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:25 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-09T06:36:25.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T06:36:25.760 INFO:teuthology.orchestra.run.smithi003.stdout:client4683 2024-05-09T06:36:25.760 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/16ed737d-2665-431a-8ac6-ef242be3975a.client4683/status') 2024-05-09T06:36:25.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:25 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-09T06:36:25.821 INFO:teuthology.orchestra.run.smithi003.stdout:instance: client.4683 (3)192.168.0.2:0/3338047068 2024-05-09T06:36:25.821 INFO:teuthology.orchestra.run.smithi003.stdout:blocklisted: false 2024-05-09T06:36:25.821 INFO:teuthology.orchestra.run.smithi003.stderr:0+1 records in 2024-05-09T06:36:25.821 INFO:teuthology.orchestra.run.smithi003.stderr:0+1 records out 2024-05-09T06:36:25.821 INFO:teuthology.orchestra.run.smithi003.stderr:69 bytes copied, 2.3977e-05 s, 2.9 MB/s 2024-05-09T06:36:25.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T06:36:25.828 INFO:tasks.kclient:client.1 config is {'syntax': 'v2'} 2024-05-09T06:36:25.828 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-05-09T06:36:25.828 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-05-09T06:36:25.828 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-09T06:36:25.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:25.952+0000 7fa32ca10640 1 Processor -- start 2024-05-09T06:36:25.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:25.952+0000 7fa32ca10640 1 -- start start 2024-05-09T06:36:25.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:25.953+0000 7fa32ca10640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa32806ec60 0x7fa3280af720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:25.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:25.953+0000 7fa32ca10640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fa3280d0120 0x7fa3280d0530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:25.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:25.953+0000 7fa32ca10640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa3280d0a70 con 0x7fa32815e4e0 2024-05-09T06:36:25.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:25.953+0000 7fa32ca10640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fa32806f570 con 0x7fa32806ec60 2024-05-09T06:36:25.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:25.953+0000 7fa32ca10640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7fa3280cea20 con 0x7fa3280d0120 2024-05-09T06:36:25.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:25.953+0000 7fa325d9b640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa32806ec60 0x7fa3280af720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:25.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:25.953+0000 7fa325d9b640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa32806ec60 0x7fa3280af720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50150/0 (socket says 172.21.15.3:50150) 2024-05-09T06:36:25.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:25.953+0000 7fa325d9b640 1 -- 172.21.15.3:0/2408217733 learned_addr learned my addr 172.21.15.3:0/2408217733 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:36:25.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:25.953+0000 7fa325d9b640 1 -- 172.21.15.3:0/2408217733 >> v2:172.21.15.190:3300/0 conn(0x7fa3280d0120 msgr2=0x7fa3280d0530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:25.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:25.953+0000 7fa325d9b640 1 --2- 172.21.15.3:0/2408217733 >> v2:172.21.15.190:3300/0 conn(0x7fa3280d0120 0x7fa3280d0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:25.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:25.953+0000 7fa325d9b640 1 -- 172.21.15.3:0/2408217733 >> v1:172.21.15.190:6789/0 conn(0x7fa32815e4e0 legacy=0x7fa3280cc7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:25.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:25.953+0000 7fa325d9b640 1 -- 172.21.15.3:0/2408217733 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa32806f290 con 0x7fa32806ec60 2024-05-09T06:36:25.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:25.954+0000 7fa325d9b640 1 --2- 172.21.15.3:0/2408217733 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa32806ec60 0x7fa3280af720 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fa32806f6f0 tx=0x7fa31002a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c5e4e61d5602343 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:25.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:25.954+0000 7fa32559a640 1 -- 172.21.15.3:0/2408217733 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa310037070 con 0x7fa32806ec60 2024-05-09T06:36:25.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:25.954+0000 7fa32559a640 1 -- 172.21.15.3:0/2408217733 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa31002ae10 con 0x7fa32806ec60 2024-05-09T06:36:25.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:25.954+0000 7fa32559a640 1 -- 172.21.15.3:0/2408217733 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa310032660 con 0x7fa32806ec60 2024-05-09T06:36:25.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:25.955+0000 7fa32ca10640 1 -- 172.21.15.3:0/2408217733 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa32806ec60 msgr2=0x7fa3280af720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:25.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:25.955+0000 7fa32ca10640 1 --2- 172.21.15.3:0/2408217733 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa32806ec60 0x7fa3280af720 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fa32806f6f0 tx=0x7fa31002a100 comp rx=0 tx=0).stop 2024-05-09T06:36:25.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:25.955+0000 7fa32ca10640 1 -- 172.21.15.3:0/2408217733 shutdown_connections 2024-05-09T06:36:25.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:25.955+0000 7fa32ca10640 1 --2- 172.21.15.3:0/2408217733 >> v2:172.21.15.190:3300/0 conn(0x7fa3280d0120 0x7fa3280d0530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:25.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:25.955+0000 7fa32ca10640 1 --2- 172.21.15.3:0/2408217733 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa32806ec60 0x7fa3280af720 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:25.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:25.955+0000 7fa32ca10640 1 -- 172.21.15.3:0/2408217733 >> 172.21.15.3:0/2408217733 conn(0x7fa328072870 msgr2=0x7fa328072c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:25.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:25.955+0000 7fa32ca10640 1 -- 172.21.15.3:0/2408217733 shutdown_connections 2024-05-09T06:36:25.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:25.955+0000 7fa32ca10640 1 -- 172.21.15.3:0/2408217733 wait complete. 2024-05-09T06:36:25.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:25.956+0000 7fa32ca10640 1 Processor -- start 2024-05-09T06:36:25.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:25.956+0000 7fa32ca10640 1 -- start start 2024-05-09T06:36:25.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:25.957+0000 7fa32ca10640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa32806ec60 0x7fa328147e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:25.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:25.957+0000 7fa32ca10640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa3280d0120 0x7fa3281f4490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:25.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:25.957+0000 7fa32ca10640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa32815e4e0 0x7fa3281f49d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:25.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:25.957+0000 7fa32ca10640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fa3280ce6b0 con 0x7fa3280d0120 2024-05-09T06:36:25.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:25.957+0000 7fa32ca10640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fa3280736b0 con 0x7fa32806ec60 2024-05-09T06:36:25.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:25.957+0000 7fa32ca10640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fa3280ccec0 con 0x7fa32815e4e0 2024-05-09T06:36:25.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:25.957+0000 7fa325d9b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa3280d0120 0x7fa3281f4490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:25.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:25.957+0000 7fa325d9b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa3280d0120 0x7fa3281f4490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50154/0 (socket says 172.21.15.3:50154) 2024-05-09T06:36:25.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:25.957+0000 7fa325d9b640 1 -- 172.21.15.3:0/1701755242 learned_addr learned my addr 172.21.15.3:0/1701755242 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:36:25.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:25.957+0000 7fa32659c640 1 --2- 172.21.15.3:0/1701755242 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa32806ec60 0x7fa328147e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:25.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:25.957+0000 7fa326d9d640 1 --2- 172.21.15.3:0/1701755242 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa32815e4e0 0x7fa3281f49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:25.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:25.958+0000 7fa325d9b640 1 -- 172.21.15.3:0/1701755242 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa32815e4e0 msgr2=0x7fa3281f49d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:25.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:25.958+0000 7fa325d9b640 1 --2- 172.21.15.3:0/1701755242 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa32815e4e0 0x7fa3281f49d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:25.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:25.958+0000 7fa325d9b640 1 -- 172.21.15.3:0/1701755242 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa32806ec60 msgr2=0x7fa328147e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:25.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:25.958+0000 7fa325d9b640 1 --2- 172.21.15.3:0/1701755242 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa32806ec60 0x7fa328147e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:25.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:25.958+0000 7fa325d9b640 1 -- 172.21.15.3:0/1701755242 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa328148350 con 0x7fa3280d0120 2024-05-09T06:36:25.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:25.958+0000 7fa326d9d640 1 --2- 172.21.15.3:0/1701755242 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa32815e4e0 0x7fa3281f49d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:36:25.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:25.958+0000 7fa32659c640 1 --2- 172.21.15.3:0/1701755242 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa32806ec60 0x7fa328147e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:36:25.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:25.958+0000 7fa325d9b640 1 --2- 172.21.15.3:0/1701755242 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa3280d0120 0x7fa3281f4490 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fa310000c00 tx=0x7fa3100046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:25.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:25.958+0000 7fa31effd640 1 -- 172.21.15.3:0/1701755242 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa310042070 con 0x7fa3280d0120 2024-05-09T06:36:25.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:25.958+0000 7fa31effd640 1 -- 172.21.15.3:0/1701755242 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa31003e670 con 0x7fa3280d0120 2024-05-09T06:36:25.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:25.958+0000 7fa32ca10640 1 -- 172.21.15.3:0/1701755242 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3281f5010 con 0x7fa3280d0120 2024-05-09T06:36:25.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:25.958+0000 7fa31effd640 1 -- 172.21.15.3:0/1701755242 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa310037040 con 0x7fa3280d0120 2024-05-09T06:36:25.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:25.958+0000 7fa32ca10640 1 -- 172.21.15.3:0/1701755242 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa328148830 con 0x7fa3280d0120 2024-05-09T06:36:25.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:25.959+0000 7fa32ca10640 1 -- 172.21.15.3:0/1701755242 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa328148620 con 0x7fa3280d0120 2024-05-09T06:36:25.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:25.959+0000 7fa31effd640 1 -- 172.21.15.3:0/1701755242 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa310045070 con 0x7fa3280d0120 2024-05-09T06:36:25.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:25.960+0000 7fa31effd640 1 --2- 172.21.15.3:0/1701755242 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fa30806be30 0x7fa30806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:25.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:25.960+0000 7fa31effd640 1 -- 172.21.15.3:0/1701755242 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fa3100ab730 con 0x7fa3280d0120 2024-05-09T06:36:25.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:25.960+0000 7fa32659c640 1 --2- 172.21.15.3:0/1701755242 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fa30806be30 0x7fa30806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:25.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:25.961+0000 7fa32659c640 1 --2- 172.21.15.3:0/1701755242 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fa30806be30 0x7fa30806e2f0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fa3281f20d0 tx=0x7fa318009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:25.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:25.964+0000 7fa31effd640 1 -- 172.21.15.3:0/1701755242 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fa3280d0a70 con 0x7fa3280d0120 2024-05-09T06:36:26.167 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:26.186+0000 7fa32ca10640 1 -- 172.21.15.3:0/1701755242 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fa3280af2d0 con 0x7fa3280d0120 2024-05-09T06:36:26.167 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:26.186+0000 7fa31effd640 1 -- 172.21.15.3:0/1701755242 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) ==== 53+0+73 (secure 0 0 0) 0x7fa310079fc0 con 0x7fa3280d0120 2024-05-09T06:36:26.167 INFO:teuthology.orchestra.run.smithi003.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-09T06:36:26.169 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:26.188+0000 7fa32ca10640 1 -- 172.21.15.3:0/1701755242 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fa30806be30 msgr2=0x7fa30806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:26.169 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:26.188+0000 7fa32ca10640 1 --2- 172.21.15.3:0/1701755242 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fa30806be30 0x7fa30806e2f0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fa3281f20d0 tx=0x7fa318009040 comp rx=0 tx=0).stop 2024-05-09T06:36:26.169 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:26.188+0000 7fa32ca10640 1 -- 172.21.15.3:0/1701755242 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa3280d0120 msgr2=0x7fa3281f4490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:26.170 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:26.188+0000 7fa32ca10640 1 --2- 172.21.15.3:0/1701755242 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa3280d0120 0x7fa3281f4490 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fa310000c00 tx=0x7fa3100046c0 comp rx=0 tx=0).stop 2024-05-09T06:36:26.170 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:26.188+0000 7fa32ca10640 1 -- 172.21.15.3:0/1701755242 shutdown_connections 2024-05-09T06:36:26.170 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:26.188+0000 7fa32ca10640 1 --2- 172.21.15.3:0/1701755242 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fa30806be30 0x7fa30806e2f0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:26.170 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:26.188+0000 7fa32ca10640 1 --2- 172.21.15.3:0/1701755242 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa32815e4e0 0x7fa3281f49d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:26.170 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:26.188+0000 7fa32ca10640 1 --2- 172.21.15.3:0/1701755242 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa3280d0120 0x7fa3281f4490 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:26.170 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:26.188+0000 7fa32ca10640 1 --2- 172.21.15.3:0/1701755242 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa32806ec60 0x7fa328147e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:26.170 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:26.188+0000 7fa32ca10640 1 -- 172.21.15.3:0/1701755242 >> 172.21.15.3:0/1701755242 conn(0x7fa328072870 msgr2=0x7fa32815ce60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:26.170 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:26.188+0000 7fa32ca10640 1 -- 172.21.15.3:0/1701755242 shutdown_connections 2024-05-09T06:36:26.170 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:26.188+0000 7fa32ca10640 1 -- 172.21.15.3:0/1701755242 wait complete. 2024-05-09T06:36:26.180 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-09T06:36:26.180 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-09T06:36:26.180 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi190.front.sepia.ceph.com 2024-05-09T06:36:26.180 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-05-09T06:36:26.180 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-05-09T06:36:26.180 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-09T06:36:26.181 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-09T06:36:26.181 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-09T06:36:26.181 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-05-09T06:36:26.181 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:36:26.181 DEBUG:teuthology.orchestra.run.smithi190:> ip addr 2024-05-09T06:36:26.200 INFO:teuthology.orchestra.run.smithi190.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-09T06:36:26.201 INFO:teuthology.orchestra.run.smithi190.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-09T06:36:26.201 INFO:teuthology.orchestra.run.smithi190.stdout: inet 127.0.0.1/8 scope host lo 2024-05-09T06:36:26.201 INFO:teuthology.orchestra.run.smithi190.stdout: valid_lft forever preferred_lft forever 2024-05-09T06:36:26.201 INFO:teuthology.orchestra.run.smithi190.stdout: inet6 ::1/128 scope host 2024-05-09T06:36:26.201 INFO:teuthology.orchestra.run.smithi190.stdout: valid_lft forever preferred_lft forever 2024-05-09T06:36:26.201 INFO:teuthology.orchestra.run.smithi190.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-09T06:36:26.201 INFO:teuthology.orchestra.run.smithi190.stdout: link/ether 0c:c4:7a:d9:36:d0 brd ff:ff:ff:ff:ff:ff 2024-05-09T06:36:26.201 INFO:teuthology.orchestra.run.smithi190.stdout: altname enp7s0f0 2024-05-09T06:36:26.201 INFO:teuthology.orchestra.run.smithi190.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-09T06:36:26.201 INFO:teuthology.orchestra.run.smithi190.stdout: link/ether 0c:c4:7a:d9:36:d1 brd ff:ff:ff:ff:ff:ff 2024-05-09T06:36:26.201 INFO:teuthology.orchestra.run.smithi190.stdout: altname enp7s0f1 2024-05-09T06:36:26.201 INFO:teuthology.orchestra.run.smithi190.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-09T06:36:26.201 INFO:teuthology.orchestra.run.smithi190.stdout: link/ether 0c:c4:7a:88:77:62 brd ff:ff:ff:ff:ff:ff 2024-05-09T06:36:26.202 INFO:teuthology.orchestra.run.smithi190.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-09T06:36:26.202 INFO:teuthology.orchestra.run.smithi190.stdout: link/ether 0c:c4:7a:88:77:63 brd ff:ff:ff:ff:ff:ff 2024-05-09T06:36:26.202 INFO:teuthology.orchestra.run.smithi190.stdout: inet 172.21.15.190/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-05-09T06:36:26.202 INFO:teuthology.orchestra.run.smithi190.stdout: valid_lft 42728sec preferred_lft 42728sec 2024-05-09T06:36:26.202 INFO:teuthology.orchestra.run.smithi190.stdout: inet6 fe80::ec4:7aff:fe88:7763/64 scope link 2024-05-09T06:36:26.202 INFO:teuthology.orchestra.run.smithi190.stdout: valid_lft forever preferred_lft forever 2024-05-09T06:36:26.202 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-05-09T06:36:26.202 DEBUG:teuthology.orchestra.run.smithi190:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-09T06:36:26.202 DEBUG:teuthology.orchestra.run.smithi190:> set -e 2024-05-09T06:36:26.202 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip link add name ceph-brx type bridge 2024-05-09T06:36:26.202 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip addr flush dev ceph-brx 2024-05-09T06:36:26.203 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip link set ceph-brx up 2024-05-09T06:36:26.203 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-05-09T06:36:26.203 DEBUG:teuthology.orchestra.run.smithi190:> ') 2024-05-09T06:36:26.297 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:36:26 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-09T06:36:26.393 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:36:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T06:36:26.399 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:36:26.400 DEBUG:teuthology.orchestra.run.smithi190:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-09T06:36:26.424 INFO:teuthology.orchestra.run.smithi190.stdout:1 2024-05-09T06:36:26.426 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:36:26.426 DEBUG:teuthology.orchestra.run.smithi190:> route 2024-05-09T06:36:26.493 INFO:teuthology.orchestra.run.smithi190.stdout:Kernel IP routing table 2024-05-09T06:36:26.493 INFO:teuthology.orchestra.run.smithi190.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-09T06:36:26.493 INFO:teuthology.orchestra.run.smithi190.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-09T06:36:26.493 INFO:teuthology.orchestra.run.smithi190.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-09T06:36:26.494 INFO:teuthology.orchestra.run.smithi190.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-09T06:36:26.494 DEBUG:teuthology.orchestra.run.smithi190:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-09T06:36:26.494 DEBUG:teuthology.orchestra.run.smithi190:> set -e 2024-05-09T06:36:26.494 DEBUG:teuthology.orchestra.run.smithi190:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-09T06:36:26.495 DEBUG:teuthology.orchestra.run.smithi190:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-09T06:36:26.495 DEBUG:teuthology.orchestra.run.smithi190:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-09T06:36:26.495 DEBUG:teuthology.orchestra.run.smithi190:> ') 2024-05-09T06:36:26.582 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:36:26 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-09T06:36:26.761 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:36:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T06:36:26.767 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:36:26.767 DEBUG:teuthology.orchestra.run.smithi190:> ip netns list 2024-05-09T06:36:26.783 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:36:26.783 DEBUG:teuthology.orchestra.run.smithi190:> ip netns list-id 2024-05-09T06:36:26.838 DEBUG:teuthology.orchestra.run.smithi190:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-09T06:36:26.838 DEBUG:teuthology.orchestra.run.smithi190:> set -e 2024-05-09T06:36:26.838 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-09T06:36:26.838 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-05-09T06:36:26.838 DEBUG:teuthology.orchestra.run.smithi190:> ') 2024-05-09T06:36:26.928 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:36:26 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-09T06:36:26.953 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:36:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T06:36:26.959 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-05-09T06:36:26.960 DEBUG:teuthology.orchestra.run.smithi190:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-09T06:36:26.960 DEBUG:teuthology.orchestra.run.smithi190:> set -e 2024-05-09T06:36:26.960 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-05-09T06:36:26.960 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-05-09T06:36:26.960 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-05-09T06:36:26.960 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-05-09T06:36:26.960 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-05-09T06:36:26.960 DEBUG:teuthology.orchestra.run.smithi190:> ') 2024-05-09T06:36:27.010 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:36:27 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-09T06:36:27.080 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:36:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T06:36:27.088 DEBUG:teuthology.orchestra.run.smithi190:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-09T06:36:27.088 DEBUG:teuthology.orchestra.run.smithi190:> set -e 2024-05-09T06:36:27.088 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip link set brx.0 up 2024-05-09T06:36:27.088 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip link set dev brx.0 master ceph-brx 2024-05-09T06:36:27.089 DEBUG:teuthology.orchestra.run.smithi190:> ') 2024-05-09T06:36:27.138 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:36:27 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-09T06:36:27.161 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:36:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T06:36:27.169 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-09T06:36:27.170 DEBUG:teuthology.orchestra.run.smithi190:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-05-09T06:36:27.184 INFO:teuthology.orchestra.run.smithi190.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-05-09T06:36:27.185 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-09T06:36:27.185 DEBUG:teuthology.orchestra.run.smithi190:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-05-09T06:36:27.240 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2024-05-09T06:36:27.240 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:36:27.240 DEBUG:teuthology.orchestra.run.smithi190:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-05-09T06:36:27.334 INFO:teuthology.orchestra.run.smithi190.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-05-09T06:36:27.334 INFO:teuthology.orchestra.run.smithi190.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-05-09T06:36:27.456 INFO:teuthology.orchestra.run.smithi190.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-05-09T06:36:27.456 INFO:teuthology.orchestra.run.smithi190.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-05-09T06:36:27.456 INFO:teuthology.orchestra.run.smithi190.stdout:mount.ceph: resolved to: "172.21.15.3:3300,172.21.15.190:3300,172.21.15.190:3301" 2024-05-09T06:36:27.456 INFO:teuthology.orchestra.run.smithi190.stdout:mount.ceph: trying mount with new device syntax: 1@16ed737d-2665-431a-8ac6-ef242be3975a.cephfs=/ 2024-05-09T06:36:27.457 INFO:teuthology.orchestra.run.smithi190.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.3:3300/172.21.15.190:3300/172.21.15.190:3301" will pass to kernel 2024-05-09T06:36:27.457 INFO:teuthology.orchestra.run.smithi190.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-05-09T06:36:27.457 INFO:teuthology.orchestra.run.smithi190.stdout: You just mounted a file system that supports labels which does not 2024-05-09T06:36:27.457 INFO:teuthology.orchestra.run.smithi190.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-05-09T06:36:27.457 INFO:teuthology.orchestra.run.smithi190.stdout: applications will generate AVC messages and not be allowed access to 2024-05-09T06:36:27.457 INFO:teuthology.orchestra.run.smithi190.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-05-09T06:36:27.459 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-09T06:36:27.459 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:36:27.459 DEBUG:teuthology.orchestra.run.smithi190:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-05-09T06:36:27.484 DEBUG:teuthology.orchestra.run.smithi190:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-09T06:36:27.574 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:36:27 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-09T06:36:27.575 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:36:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T06:36:27.581 INFO:teuthology.orchestra.run.smithi190.stdout:client4692 2024-05-09T06:36:27.582 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-05-09T06:36:27.582 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T06:36:27.598 DEBUG:teuthology.orchestra.run.smithi190:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-09T06:36:27.687 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:36:27 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-09T06:36:27.688 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:36:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T06:36:27.694 INFO:teuthology.orchestra.run.smithi190.stdout:client4692 2024-05-09T06:36:27.694 DEBUG:teuthology.orchestra.run.smithi190:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-09T06:36:27.743 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:36:27 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-09T06:36:27.744 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:36:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T06:36:27.750 INFO:teuthology.orchestra.run.smithi190.stdout:client4692 2024-05-09T06:36:27.750 DEBUG:teuthology.orchestra.run.smithi190:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/16ed737d-2665-431a-8ac6-ef242be3975a.client4692/status') 2024-05-09T06:36:27.799 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:36:27 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-09T06:36:27.809 INFO:teuthology.orchestra.run.smithi190.stdout:instance: client.4692 (3)172.21.15.190:0/1538588534 2024-05-09T06:36:27.809 INFO:teuthology.orchestra.run.smithi190.stdout:blocklisted: false 2024-05-09T06:36:27.809 INFO:teuthology.orchestra.run.smithi190.stderr:0+1 records in 2024-05-09T06:36:27.810 INFO:teuthology.orchestra.run.smithi190.stderr:0+1 records out 2024-05-09T06:36:27.810 INFO:teuthology.orchestra.run.smithi190.stderr:71 bytes copied, 2.3799e-05 s, 3.0 MB/s 2024-05-09T06:36:27.810 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:36:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T06:36:27.817 INFO:tasks.kclient:client.3 config is {'syntax': 'v2'} 2024-05-09T06:36:27.817 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-05-09T06:36:27.817 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-05-09T06:36:27.817 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-09T06:36:27.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:27.953+0000 7f8720aba640 1 Processor -- start 2024-05-09T06:36:27.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:27.953+0000 7f8720aba640 1 -- start start 2024-05-09T06:36:27.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:27.955+0000 7f8720aba640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f871c06b960 0x7f871c0b4640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:27.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:27.955+0000 7f8720aba640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f871c0ae600 con 0x7f871c0b4b80 2024-05-09T06:36:27.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:27.955+0000 7f8720aba640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f871c0b4ff0 con 0x7f871c15e420 2024-05-09T06:36:27.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:27.955+0000 7f8720aba640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f871c15e800 con 0x7f871c06b960 2024-05-09T06:36:27.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:27.955+0000 7f8713fff640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f871c06b960 0x7f871c0b4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:27.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:27.955+0000 7f8713fff640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f871c06b960 0x7f871c0b4640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50158/0 (socket says 172.21.15.3:50158) 2024-05-09T06:36:27.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:27.955+0000 7f8713fff640 1 -- 172.21.15.3:0/234543066 learned_addr learned my addr 172.21.15.3:0/234543066 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:36:27.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:27.955+0000 7f871a59c640 1 --1- 172.21.15.3:0/234543066 >> v1:172.21.15.190:6789/0 conn(0x7f871c15e420 0x7f871c06b250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.3:44772/0 (socket says 172.21.15.3:44772) 2024-05-09T06:36:27.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:27.955+0000 7f871ad9d640 1 --1- 172.21.15.3:0/234543066 >> v1:172.21.15.3:6789/0 conn(0x7f871c0b4b80 0x7f871c0aeae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:47858/0 (socket says 172.21.15.3:47858) 2024-05-09T06:36:27.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:27.955+0000 7f8713fff640 1 -- 172.21.15.3:0/234543066 >> v1:172.21.15.190:6789/0 conn(0x7f871c15e420 legacy=0x7f871c06b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:27.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:27.955+0000 7f8713fff640 1 -- 172.21.15.3:0/234543066 >> v1:172.21.15.3:6789/0 conn(0x7f871c0b4b80 legacy=0x7f871c0aeae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:27.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:27.955+0000 7f8713fff640 1 -- 172.21.15.3:0/234543066 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f871c0b5290 con 0x7f871c06b960 2024-05-09T06:36:27.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:27.956+0000 7f8713fff640 1 --2- 172.21.15.3:0/234543066 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f871c06b960 0x7f871c0b4640 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f8704009870 tx=0x7f870402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15b175c914923f01 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:27.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:27.956+0000 7f8719d9b640 1 -- 172.21.15.3:0/234543066 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8704037070 con 0x7f871c06b960 2024-05-09T06:36:27.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:27.957+0000 7f8719d9b640 1 -- 172.21.15.3:0/234543066 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8704032440 con 0x7f871c06b960 2024-05-09T06:36:27.938 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:27.957+0000 7f8719d9b640 1 -- 172.21.15.3:0/234543066 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8704032810 con 0x7f871c06b960 2024-05-09T06:36:27.938 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:27.957+0000 7f8720aba640 1 -- 172.21.15.3:0/234543066 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f871c06b960 msgr2=0x7f871c0b4640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:27.938 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:27.957+0000 7f8720aba640 1 --2- 172.21.15.3:0/234543066 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f871c06b960 0x7f871c0b4640 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f8704009870 tx=0x7f870402a270 comp rx=0 tx=0).stop 2024-05-09T06:36:27.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:27.957+0000 7f8720aba640 1 -- 172.21.15.3:0/234543066 shutdown_connections 2024-05-09T06:36:27.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:27.957+0000 7f8720aba640 1 --2- 172.21.15.3:0/234543066 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f871c06b960 0x7f871c0b4640 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:27.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:27.957+0000 7f8720aba640 1 -- 172.21.15.3:0/234543066 >> 172.21.15.3:0/234543066 conn(0x7f871c06e3d0 msgr2=0x7f871c06e7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:27.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:27.957+0000 7f8720aba640 1 -- 172.21.15.3:0/234543066 shutdown_connections 2024-05-09T06:36:27.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:27.957+0000 7f8720aba640 1 -- 172.21.15.3:0/234543066 wait complete. 2024-05-09T06:36:27.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:27.959+0000 7f8720aba640 1 Processor -- start 2024-05-09T06:36:27.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:27.959+0000 7f8720aba640 1 -- start start 2024-05-09T06:36:27.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:27.959+0000 7f8720aba640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f871c06b960 0x7f871c1582a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:27.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:27.960+0000 7f8720aba640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f871c0b4b80 0x7f871c1f7950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:27.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:27.960+0000 7f8720aba640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f871c15e420 0x7f871c14d020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:27.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:27.960+0000 7f8720aba640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f871c0b5b30 con 0x7f871c0b4b80 2024-05-09T06:36:27.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:27.960+0000 7f8720aba640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f871c0b5850 con 0x7f871c15e420 2024-05-09T06:36:27.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:27.960+0000 7f8720aba640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f871c0af1f0 con 0x7f871c06b960 2024-05-09T06:36:27.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:27.960+0000 7f8713fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f871c0b4b80 0x7f871c1f7950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:27.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:27.960+0000 7f8713fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f871c0b4b80 0x7f871c1f7950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50170/0 (socket says 172.21.15.3:50170) 2024-05-09T06:36:27.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:27.960+0000 7f8713fff640 1 -- 172.21.15.3:0/3049231382 learned_addr learned my addr 172.21.15.3:0/3049231382 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:36:27.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:27.960+0000 7f871ad9d640 1 --2- 172.21.15.3:0/3049231382 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f871c15e420 0x7f871c14d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:27.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:27.960+0000 7f871a59c640 1 --2- 172.21.15.3:0/3049231382 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f871c06b960 0x7f871c1582a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:27.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:27.960+0000 7f8713fff640 1 -- 172.21.15.3:0/3049231382 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f871c06b960 msgr2=0x7f871c1582a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:27.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:27.960+0000 7f8713fff640 1 --2- 172.21.15.3:0/3049231382 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f871c06b960 0x7f871c1582a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:27.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:27.960+0000 7f8713fff640 1 -- 172.21.15.3:0/3049231382 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f871c15e420 msgr2=0x7f871c14d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:27.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:27.960+0000 7f8713fff640 1 --2- 172.21.15.3:0/3049231382 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f871c15e420 0x7f871c14d020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:27.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:27.960+0000 7f8713fff640 1 -- 172.21.15.3:0/3049231382 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f871c1f8080 con 0x7f871c0b4b80 2024-05-09T06:36:27.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:27.960+0000 7f8713fff640 1 --2- 172.21.15.3:0/3049231382 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f871c0b4b80 0x7f871c1f7950 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f8704000c00 tx=0x7f87040046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:27.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:27.960+0000 7f8712ffd640 1 -- 172.21.15.3:0/3049231382 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8704037040 con 0x7f871c0b4b80 2024-05-09T06:36:27.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:27.960+0000 7f8712ffd640 1 -- 172.21.15.3:0/3049231382 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8704042070 con 0x7f871c0b4b80 2024-05-09T06:36:27.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:27.960+0000 7f8712ffd640 1 -- 172.21.15.3:0/3049231382 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8704030bd0 con 0x7f871c0b4b80 2024-05-09T06:36:27.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:27.960+0000 7f8720aba640 1 -- 172.21.15.3:0/3049231382 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f871c1f8360 con 0x7f871c0b4b80 2024-05-09T06:36:27.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:27.961+0000 7f8720aba640 1 -- 172.21.15.3:0/3049231382 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f871c156730 con 0x7f871c0b4b80 2024-05-09T06:36:27.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:27.961+0000 7f8720aba640 1 -- 172.21.15.3:0/3049231382 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f871c04f660 con 0x7f871c0b4b80 2024-05-09T06:36:27.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:27.962+0000 7f8712ffd640 1 -- 172.21.15.3:0/3049231382 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8704047460 con 0x7f871c0b4b80 2024-05-09T06:36:27.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:27.962+0000 7f8712ffd640 1 --2- 172.21.15.3:0/3049231382 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f86fc06be30 0x7f86fc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:27.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:27.963+0000 7f8712ffd640 1 -- 172.21.15.3:0/3049231382 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f87040ab910 con 0x7f871c0b4b80 2024-05-09T06:36:27.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:27.963+0000 7f871a59c640 1 --2- 172.21.15.3:0/3049231382 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f86fc06be30 0x7f86fc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:27.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:27.963+0000 7f871a59c640 1 --2- 172.21.15.3:0/3049231382 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f86fc06be30 0x7f86fc06e2f0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f870c007680 tx=0x7f870c000fb0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:27.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:27.967+0000 7f8712ffd640 1 -- 172.21.15.3:0/3049231382 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f8704079d00 con 0x7f871c0b4b80 2024-05-09T06:36:28.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:28.191+0000 7f8720aba640 1 -- 172.21.15.3:0/3049231382 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f871c15dc40 con 0x7f871c0b4b80 2024-05-09T06:36:28.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:28.192+0000 7f8712ffd640 1 -- 172.21.15.3:0/3049231382 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) ==== 53+0+73 (secure 0 0 0) 0x7f870407e000 con 0x7f871c0b4b80 2024-05-09T06:36:28.175 INFO:teuthology.orchestra.run.smithi003.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-09T06:36:28.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:28.194+0000 7f8720aba640 1 -- 172.21.15.3:0/3049231382 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f86fc06be30 msgr2=0x7f86fc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:28.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:28.194+0000 7f8720aba640 1 --2- 172.21.15.3:0/3049231382 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f86fc06be30 0x7f86fc06e2f0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f870c007680 tx=0x7f870c000fb0 comp rx=0 tx=0).stop 2024-05-09T06:36:28.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:28.194+0000 7f8720aba640 1 -- 172.21.15.3:0/3049231382 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f871c0b4b80 msgr2=0x7f871c1f7950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:28.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:28.194+0000 7f8720aba640 1 --2- 172.21.15.3:0/3049231382 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f871c0b4b80 0x7f871c1f7950 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f8704000c00 tx=0x7f87040046c0 comp rx=0 tx=0).stop 2024-05-09T06:36:28.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:28.194+0000 7f8720aba640 1 -- 172.21.15.3:0/3049231382 shutdown_connections 2024-05-09T06:36:28.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:28.194+0000 7f8720aba640 1 --2- 172.21.15.3:0/3049231382 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f86fc06be30 0x7f86fc06e2f0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:28.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:28.194+0000 7f8720aba640 1 --2- 172.21.15.3:0/3049231382 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f871c15e420 0x7f871c14d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:28.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:28.194+0000 7f8720aba640 1 --2- 172.21.15.3:0/3049231382 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f871c0b4b80 0x7f871c1f7950 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:28.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:28.194+0000 7f8720aba640 1 --2- 172.21.15.3:0/3049231382 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f871c06b960 0x7f871c1582a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:28.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:28.194+0000 7f8720aba640 1 -- 172.21.15.3:0/3049231382 >> 172.21.15.3:0/3049231382 conn(0x7f871c06e3d0 msgr2=0x7f871c0b2cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:28.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:28.194+0000 7f8720aba640 1 -- 172.21.15.3:0/3049231382 shutdown_connections 2024-05-09T06:36:28.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:28.194+0000 7f8720aba640 1 -- 172.21.15.3:0/3049231382 wait complete. 2024-05-09T06:36:28.186 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-09T06:36:28.186 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-09T06:36:28.186 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi190.front.sepia.ceph.com 2024-05-09T06:36:28.186 INFO:tasks.cephfs.mount:self.client.name = client.3 2024-05-09T06:36:28.186 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-05-09T06:36:28.186 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-09T06:36:28.186 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-09T06:36:28.186 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-09T06:36:28.186 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2024-05-09T06:36:28.186 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:36:28.187 DEBUG:teuthology.orchestra.run.smithi190:> ip addr 2024-05-09T06:36:28.204 INFO:teuthology.orchestra.run.smithi190.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-09T06:36:28.204 INFO:teuthology.orchestra.run.smithi190.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-09T06:36:28.204 INFO:teuthology.orchestra.run.smithi190.stdout: inet 127.0.0.1/8 scope host lo 2024-05-09T06:36:28.205 INFO:teuthology.orchestra.run.smithi190.stdout: valid_lft forever preferred_lft forever 2024-05-09T06:36:28.205 INFO:teuthology.orchestra.run.smithi190.stdout: inet6 ::1/128 scope host 2024-05-09T06:36:28.205 INFO:teuthology.orchestra.run.smithi190.stdout: valid_lft forever preferred_lft forever 2024-05-09T06:36:28.205 INFO:teuthology.orchestra.run.smithi190.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-09T06:36:28.205 INFO:teuthology.orchestra.run.smithi190.stdout: link/ether 0c:c4:7a:d9:36:d0 brd ff:ff:ff:ff:ff:ff 2024-05-09T06:36:28.205 INFO:teuthology.orchestra.run.smithi190.stdout: altname enp7s0f0 2024-05-09T06:36:28.205 INFO:teuthology.orchestra.run.smithi190.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-09T06:36:28.205 INFO:teuthology.orchestra.run.smithi190.stdout: link/ether 0c:c4:7a:d9:36:d1 brd ff:ff:ff:ff:ff:ff 2024-05-09T06:36:28.206 INFO:teuthology.orchestra.run.smithi190.stdout: altname enp7s0f1 2024-05-09T06:36:28.206 INFO:teuthology.orchestra.run.smithi190.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-09T06:36:28.206 INFO:teuthology.orchestra.run.smithi190.stdout: link/ether 0c:c4:7a:88:77:62 brd ff:ff:ff:ff:ff:ff 2024-05-09T06:36:28.206 INFO:teuthology.orchestra.run.smithi190.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-09T06:36:28.206 INFO:teuthology.orchestra.run.smithi190.stdout: link/ether 0c:c4:7a:88:77:63 brd ff:ff:ff:ff:ff:ff 2024-05-09T06:36:28.206 INFO:teuthology.orchestra.run.smithi190.stdout: inet 172.21.15.190/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-05-09T06:36:28.207 INFO:teuthology.orchestra.run.smithi190.stdout: valid_lft 42726sec preferred_lft 42726sec 2024-05-09T06:36:28.207 INFO:teuthology.orchestra.run.smithi190.stdout: inet6 fe80::ec4:7aff:fe88:7763/64 scope link 2024-05-09T06:36:28.207 INFO:teuthology.orchestra.run.smithi190.stdout: valid_lft forever preferred_lft forever 2024-05-09T06:36:28.207 INFO:teuthology.orchestra.run.smithi190.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-09T06:36:28.207 INFO:teuthology.orchestra.run.smithi190.stdout: link/ether c6:5d:ad:92:ed:35 brd ff:ff:ff:ff:ff:ff 2024-05-09T06:36:28.207 INFO:teuthology.orchestra.run.smithi190.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-09T06:36:28.207 INFO:teuthology.orchestra.run.smithi190.stdout: valid_lft forever preferred_lft forever 2024-05-09T06:36:28.208 INFO:teuthology.orchestra.run.smithi190.stdout: inet6 fe80::2c35:40ff:fe34:5f3a/64 scope link 2024-05-09T06:36:28.208 INFO:teuthology.orchestra.run.smithi190.stdout: valid_lft forever preferred_lft forever 2024-05-09T06:36:28.208 INFO:teuthology.orchestra.run.smithi190.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-09T06:36:28.208 INFO:teuthology.orchestra.run.smithi190.stdout: link/ether c6:5d:ad:92:ed:35 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-09T06:36:28.208 INFO:teuthology.orchestra.run.smithi190.stdout: inet6 fe80::c45d:adff:fe92:ed35/64 scope link tentative 2024-05-09T06:36:28.208 INFO:teuthology.orchestra.run.smithi190.stdout: valid_lft forever preferred_lft forever 2024-05-09T06:36:28.209 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:36:28.209 DEBUG:teuthology.orchestra.run.smithi190:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-09T06:36:28.270 INFO:teuthology.orchestra.run.smithi190.stdout:1 2024-05-09T06:36:28.272 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:36:28.272 DEBUG:teuthology.orchestra.run.smithi190:> route 2024-05-09T06:36:28.328 INFO:teuthology.orchestra.run.smithi190.stdout:Kernel IP routing table 2024-05-09T06:36:28.329 INFO:teuthology.orchestra.run.smithi190.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-09T06:36:28.329 INFO:teuthology.orchestra.run.smithi190.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-09T06:36:28.329 INFO:teuthology.orchestra.run.smithi190.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-09T06:36:28.329 INFO:teuthology.orchestra.run.smithi190.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-09T06:36:28.329 DEBUG:teuthology.orchestra.run.smithi190:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-09T06:36:28.329 DEBUG:teuthology.orchestra.run.smithi190:> set -e 2024-05-09T06:36:28.330 DEBUG:teuthology.orchestra.run.smithi190:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-09T06:36:28.330 DEBUG:teuthology.orchestra.run.smithi190:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-09T06:36:28.330 DEBUG:teuthology.orchestra.run.smithi190:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-09T06:36:28.330 DEBUG:teuthology.orchestra.run.smithi190:> ') 2024-05-09T06:36:28.417 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:36:28 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-09T06:36:28.449 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:36:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T06:36:28.455 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:36:28.456 DEBUG:teuthology.orchestra.run.smithi190:> ip netns list 2024-05-09T06:36:28.471 INFO:teuthology.orchestra.run.smithi190.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-05-09T06:36:28.472 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:36:28.472 DEBUG:teuthology.orchestra.run.smithi190:> ip netns list-id 2024-05-09T06:36:28.527 INFO:teuthology.orchestra.run.smithi190.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2024-05-09T06:36:28.527 DEBUG:teuthology.orchestra.run.smithi190:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-09T06:36:28.527 DEBUG:teuthology.orchestra.run.smithi190:> set -e 2024-05-09T06:36:28.527 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2024-05-09T06:36:28.527 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2024-05-09T06:36:28.527 DEBUG:teuthology.orchestra.run.smithi190:> ') 2024-05-09T06:36:28.616 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:36:28 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-09T06:36:28.639 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:36:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T06:36:28.645 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:36:28.645 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-05-09T06:36:28.671 INFO:teuthology.orchestra.run.smithi190.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-09T06:36:28.671 INFO:teuthology.orchestra.run.smithi190.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-09T06:36:28.671 INFO:teuthology.orchestra.run.smithi190.stdout: inet 127.0.0.1/8 scope host lo 2024-05-09T06:36:28.671 INFO:teuthology.orchestra.run.smithi190.stdout: valid_lft forever preferred_lft forever 2024-05-09T06:36:28.671 INFO:teuthology.orchestra.run.smithi190.stdout: inet6 ::1/128 scope host 2024-05-09T06:36:28.672 INFO:teuthology.orchestra.run.smithi190.stdout: valid_lft forever preferred_lft forever 2024-05-09T06:36:28.672 INFO:teuthology.orchestra.run.smithi190.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-09T06:36:28.672 INFO:teuthology.orchestra.run.smithi190.stdout: link/ether 7a:dd:fd:93:7a:c1 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-05-09T06:36:28.672 INFO:teuthology.orchestra.run.smithi190.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-05-09T06:36:28.672 INFO:teuthology.orchestra.run.smithi190.stdout: valid_lft forever preferred_lft forever 2024-05-09T06:36:28.672 INFO:teuthology.orchestra.run.smithi190.stdout: inet6 fe80::78dd:fdff:fe93:7ac1/64 scope link 2024-05-09T06:36:28.672 INFO:teuthology.orchestra.run.smithi190.stdout: valid_lft forever preferred_lft forever 2024-05-09T06:36:28.674 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:36:28.674 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-05-09T06:36:28.739 INFO:teuthology.orchestra.run.smithi190.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-09T06:36:28.739 INFO:teuthology.orchestra.run.smithi190.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-09T06:36:28.740 INFO:teuthology.orchestra.run.smithi190.stdout: inet 127.0.0.1/8 scope host lo 2024-05-09T06:36:28.740 INFO:teuthology.orchestra.run.smithi190.stdout: valid_lft forever preferred_lft forever 2024-05-09T06:36:28.740 INFO:teuthology.orchestra.run.smithi190.stdout: inet6 ::1/128 scope host 2024-05-09T06:36:28.740 INFO:teuthology.orchestra.run.smithi190.stdout: valid_lft forever preferred_lft forever 2024-05-09T06:36:28.740 INFO:teuthology.orchestra.run.smithi190.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-09T06:36:28.740 INFO:teuthology.orchestra.run.smithi190.stdout: link/ether 7a:dd:fd:93:7a:c1 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-05-09T06:36:28.740 INFO:teuthology.orchestra.run.smithi190.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-05-09T06:36:28.740 INFO:teuthology.orchestra.run.smithi190.stdout: valid_lft forever preferred_lft forever 2024-05-09T06:36:28.740 INFO:teuthology.orchestra.run.smithi190.stdout: inet6 fe80::78dd:fdff:fe93:7ac1/64 scope link 2024-05-09T06:36:28.740 INFO:teuthology.orchestra.run.smithi190.stdout: valid_lft forever preferred_lft forever 2024-05-09T06:36:28.741 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2024-05-09T06:36:28.741 DEBUG:teuthology.orchestra.run.smithi190:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-09T06:36:28.741 DEBUG:teuthology.orchestra.run.smithi190:> set -e 2024-05-09T06:36:28.742 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2024-05-09T06:36:28.742 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-05-09T06:36:28.742 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2024-05-09T06:36:28.742 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2024-05-09T06:36:28.742 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2024-05-09T06:36:28.742 DEBUG:teuthology.orchestra.run.smithi190:> ') 2024-05-09T06:36:28.830 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:36:28 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-09T06:36:28.889 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:36:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T06:36:28.895 DEBUG:teuthology.orchestra.run.smithi190:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-09T06:36:28.895 DEBUG:teuthology.orchestra.run.smithi190:> set -e 2024-05-09T06:36:28.895 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip link set brx.1 up 2024-05-09T06:36:28.895 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip link set dev brx.1 master ceph-brx 2024-05-09T06:36:28.895 DEBUG:teuthology.orchestra.run.smithi190:> ') 2024-05-09T06:36:28.944 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:36:28 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-09T06:36:28.966 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:36:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T06:36:28.972 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-09T06:36:28.972 DEBUG:teuthology.orchestra.run.smithi190:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2024-05-09T06:36:28.987 INFO:teuthology.orchestra.run.smithi190.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2024-05-09T06:36:28.988 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-09T06:36:28.988 DEBUG:teuthology.orchestra.run.smithi190:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2024-05-09T06:36:29.043 INFO:tasks.cephfs.kernel_mount:mounting using device: 3@.cephfs=/ 2024-05-09T06:36:29.043 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:36:29.043 DEBUG:teuthology.orchestra.run.smithi190:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 3@.cephfs=/ /home/ubuntu/cephtest/mnt.3 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-05-09T06:36:29.125 INFO:teuthology.orchestra.run.smithi190.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-05-09T06:36:29.126 INFO:teuthology.orchestra.run.smithi190.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-05-09T06:36:29.132 INFO:teuthology.orchestra.run.smithi190.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-05-09T06:36:29.133 INFO:teuthology.orchestra.run.smithi190.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-05-09T06:36:29.133 INFO:teuthology.orchestra.run.smithi190.stdout:mount.ceph: resolved to: "172.21.15.3:3300,172.21.15.190:3300,172.21.15.190:3301" 2024-05-09T06:36:29.133 INFO:teuthology.orchestra.run.smithi190.stdout:mount.ceph: trying mount with new device syntax: 3@16ed737d-2665-431a-8ac6-ef242be3975a.cephfs=/ 2024-05-09T06:36:29.133 INFO:teuthology.orchestra.run.smithi190.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,ms_mode=prefer-crc,key=3,mon_addr=172.21.15.3:3300/172.21.15.190:3300/172.21.15.190:3301" will pass to kernel 2024-05-09T06:36:29.134 INFO:teuthology.orchestra.run.smithi190.stdout:mount: /home/ubuntu/cephtest/mnt.3 does not contain SELinux labels. 2024-05-09T06:36:29.134 INFO:teuthology.orchestra.run.smithi190.stdout: You just mounted a file system that supports labels which does not 2024-05-09T06:36:29.134 INFO:teuthology.orchestra.run.smithi190.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-05-09T06:36:29.134 INFO:teuthology.orchestra.run.smithi190.stdout: applications will generate AVC messages and not be allowed access to 2024-05-09T06:36:29.134 INFO:teuthology.orchestra.run.smithi190.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-05-09T06:36:29.135 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-09T06:36:29.136 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:36:29.136 DEBUG:teuthology.orchestra.run.smithi190:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2024-05-09T06:36:29.203 DEBUG:teuthology.orchestra.run.smithi190:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-09T06:36:29.293 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:36:29 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-09T06:36:29.294 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:36:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T06:36:29.300 INFO:teuthology.orchestra.run.smithi190.stdout:client4627 2024-05-09T06:36:29.300 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-05-09T06:36:29.300 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T06:36:29.317 DEBUG:teuthology.orchestra.run.smithi190:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-09T06:36:29.404 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:36:29 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-09T06:36:29.406 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:36:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T06:36:29.411 INFO:teuthology.orchestra.run.smithi190.stdout:client4627 2024-05-09T06:36:29.412 DEBUG:teuthology.orchestra.run.smithi190:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-09T06:36:29.460 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:36:29 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-09T06:36:29.462 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:36:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T06:36:29.467 INFO:teuthology.orchestra.run.smithi190.stdout:client4627 2024-05-09T06:36:29.468 DEBUG:teuthology.orchestra.run.smithi190:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/16ed737d-2665-431a-8ac6-ef242be3975a.client4627/status') 2024-05-09T06:36:29.516 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:36:29 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-09T06:36:29.527 INFO:teuthology.orchestra.run.smithi190.stdout:instance: client.4627 (3)192.168.0.2:0/4049191505 2024-05-09T06:36:29.527 INFO:teuthology.orchestra.run.smithi190.stdout:blocklisted: false 2024-05-09T06:36:29.528 INFO:teuthology.orchestra.run.smithi190.stderr:0+1 records in 2024-05-09T06:36:29.528 INFO:teuthology.orchestra.run.smithi190.stderr:0+1 records out 2024-05-09T06:36:29.528 INFO:teuthology.orchestra.run.smithi190.stderr:69 bytes copied, 2.5768e-05 s, 2.7 MB/s 2024-05-09T06:36:29.528 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:36:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T06:36:29.534 INFO:teuthology.run_tasks:Running task check-counter... 2024-05-09T06:36:29.551 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2024-05-09T06:36:29.576 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T06:36:29.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:29.711+0000 7f7b8dda9640 1 Processor -- start 2024-05-09T06:36:29.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:29.711+0000 7f7b8dda9640 1 -- start start 2024-05-09T06:36:29.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:29.712+0000 7f7b8dda9640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f7b8815e290 0x7f7b880ae2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:29.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:29.712+0000 7f7b8dda9640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f7b880ae7f0 0x7f7b8806bc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:29.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:29.712+0000 7f7b8dda9640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7b8806cc30 con 0x7f7b8806c1a0 2024-05-09T06:36:29.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:29.712+0000 7f7b8dda9640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f7b880aec50 con 0x7f7b880ae7f0 2024-05-09T06:36:29.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:29.712+0000 7f7b8dda9640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f7b880b4630 con 0x7f7b8815e290 2024-05-09T06:36:29.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:29.712+0000 7f7b86ffd640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f7b880ae7f0 0x7f7b8806bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:29.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:29.712+0000 7f7b87fff640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f7b8806c1a0 0x7f7b88161dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:47864/0 (socket says 172.21.15.3:47864) 2024-05-09T06:36:29.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:29.712+0000 7f7b87fff640 1 -- 172.21.15.3:0/3751871314 learned_addr learned my addr 172.21.15.3:0/3751871314 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-05-09T06:36:29.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:29.713+0000 7f7b877fe640 1 --2- 172.21.15.3:0/3751871314 >> v2:172.21.15.190:3300/0 conn(0x7f7b8815e290 0x7f7b880ae2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:29.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:29.713+0000 7f7b86ffd640 1 -- 172.21.15.3:0/3751871314 >> v2:172.21.15.190:3300/0 conn(0x7f7b8815e290 msgr2=0x7f7b880ae2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:29.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:29.713+0000 7f7b86ffd640 1 --2- 172.21.15.3:0/3751871314 >> v2:172.21.15.190:3300/0 conn(0x7f7b8815e290 0x7f7b880ae2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:29.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:29.713+0000 7f7b86ffd640 1 -- 172.21.15.3:0/3751871314 >> v1:172.21.15.3:6789/0 conn(0x7f7b8806c1a0 legacy=0x7f7b88161dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:29.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:29.713+0000 7f7b86ffd640 1 -- 172.21.15.3:0/3751871314 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b8815e670 con 0x7f7b880ae7f0 2024-05-09T06:36:29.695 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:29.714+0000 7f7b86ffd640 1 --2- 172.21.15.3:0/3751871314 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7b880ae7f0 0x7f7b8806bc60 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f7b880b47b0 tx=0x7f7b7c02a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e51e090499694ad server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:29.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:29.714+0000 7f7b867fc640 1 -- 172.21.15.3:0/3751871314 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b7c037070 con 0x7f7b880ae7f0 2024-05-09T06:36:29.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:29.714+0000 7f7b867fc640 1 -- 172.21.15.3:0/3751871314 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b7c032440 con 0x7f7b880ae7f0 2024-05-09T06:36:29.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:29.714+0000 7f7b867fc640 1 -- 172.21.15.3:0/3751871314 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b7c02ae60 con 0x7f7b880ae7f0 2024-05-09T06:36:29.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:29.714+0000 7f7b8dda9640 1 -- 172.21.15.3:0/3751871314 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7b880ae7f0 msgr2=0x7f7b8806bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:29.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:29.714+0000 7f7b8dda9640 1 --2- 172.21.15.3:0/3751871314 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7b880ae7f0 0x7f7b8806bc60 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f7b880b47b0 tx=0x7f7b7c02a110 comp rx=0 tx=0).stop 2024-05-09T06:36:29.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:29.714+0000 7f7b8dda9640 1 -- 172.21.15.3:0/3751871314 shutdown_connections 2024-05-09T06:36:29.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:29.714+0000 7f7b8dda9640 1 --2- 172.21.15.3:0/3751871314 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7b880ae7f0 0x7f7b8806bc60 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:29.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:29.714+0000 7f7b8dda9640 1 --2- 172.21.15.3:0/3751871314 >> v2:172.21.15.190:3300/0 conn(0x7f7b8815e290 0x7f7b880ae2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:29.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:29.714+0000 7f7b8dda9640 1 -- 172.21.15.3:0/3751871314 >> 172.21.15.3:0/3751871314 conn(0x7f7b880d05d0 msgr2=0x7f7b880d09e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:29.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:29.714+0000 7f7b8dda9640 1 -- 172.21.15.3:0/3751871314 shutdown_connections 2024-05-09T06:36:29.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:29.714+0000 7f7b8dda9640 1 -- 172.21.15.3:0/3751871314 wait complete. 2024-05-09T06:36:29.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:29.716+0000 7f7b8dda9640 1 Processor -- start 2024-05-09T06:36:29.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:29.716+0000 7f7b8dda9640 1 -- start start 2024-05-09T06:36:29.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:29.717+0000 7f7b8dda9640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7b8806c1a0 0x7f7b880cedf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:29.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:29.717+0000 7f7b8dda9640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7b880ae7f0 0x7f7b88150f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:29.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:29.717+0000 7f7b8dda9640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7b8815e290 0x7f7b88146ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:29.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:29.717+0000 7f7b8dda9640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f7b880b3fe0 con 0x7f7b8815e290 2024-05-09T06:36:29.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:29.717+0000 7f7b8dda9640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f7b880b5ad0 con 0x7f7b880ae7f0 2024-05-09T06:36:29.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:29.717+0000 7f7b8dda9640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f7b880b42c0 con 0x7f7b8806c1a0 2024-05-09T06:36:29.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:29.717+0000 7f7b87fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7b8815e290 0x7f7b88146ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:29.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:29.717+0000 7f7b87fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7b8815e290 0x7f7b88146ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50180/0 (socket says 172.21.15.3:50180) 2024-05-09T06:36:29.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:29.717+0000 7f7b877fe640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7b8806c1a0 0x7f7b880cedf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:29.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:29.717+0000 7f7b87fff640 1 -- 172.21.15.3:0/3731497954 learned_addr learned my addr 172.21.15.3:0/3731497954 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:36:29.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:29.717+0000 7f7b86ffd640 1 --2- 172.21.15.3:0/3731497954 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7b880ae7f0 0x7f7b88150f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:29.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:29.717+0000 7f7b87fff640 1 -- 172.21.15.3:0/3731497954 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7b8806c1a0 msgr2=0x7f7b880cedf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:29.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:29.717+0000 7f7b87fff640 1 --2- 172.21.15.3:0/3731497954 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7b8806c1a0 0x7f7b880cedf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:29.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:29.717+0000 7f7b87fff640 1 -- 172.21.15.3:0/3731497954 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7b880ae7f0 msgr2=0x7f7b88150f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:29.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:29.717+0000 7f7b87fff640 1 --2- 172.21.15.3:0/3731497954 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7b880ae7f0 0x7f7b88150f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:29.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:29.717+0000 7f7b87fff640 1 -- 172.21.15.3:0/3731497954 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b88151640 con 0x7f7b8815e290 2024-05-09T06:36:29.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:29.717+0000 7f7b877fe640 1 --2- 172.21.15.3:0/3731497954 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7b8806c1a0 0x7f7b880cedf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T06:36:29.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:29.717+0000 7f7b87fff640 1 --2- 172.21.15.3:0/3731497954 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7b8815e290 0x7f7b88146ea0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f7b8000ca80 tx=0x7f7b8000cf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:29.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:29.717+0000 7f7b6ffff640 1 -- 172.21.15.3:0/3731497954 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b80017070 con 0x7f7b8815e290 2024-05-09T06:36:29.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:29.717+0000 7f7b6ffff640 1 -- 172.21.15.3:0/3731497954 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b8000ac30 con 0x7f7b8815e290 2024-05-09T06:36:29.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:29.718+0000 7f7b6ffff640 1 -- 172.21.15.3:0/3731497954 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b80012600 con 0x7f7b8815e290 2024-05-09T06:36:29.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:29.718+0000 7f7b8dda9640 1 -- 172.21.15.3:0/3731497954 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b88151920 con 0x7f7b8815e290 2024-05-09T06:36:29.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:29.718+0000 7f7b8dda9640 1 -- 172.21.15.3:0/3731497954 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b88151ee0 con 0x7f7b8815e290 2024-05-09T06:36:29.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:29.719+0000 7f7b8dda9640 1 -- 172.21.15.3:0/3731497954 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b8804f660 con 0x7f7b8815e290 2024-05-09T06:36:29.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:29.719+0000 7f7b6ffff640 1 -- 172.21.15.3:0/3731497954 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7b8001a070 con 0x7f7b8815e290 2024-05-09T06:36:29.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:29.720+0000 7f7b6ffff640 1 --2- 172.21.15.3:0/3731497954 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f7b7406be30 0x7f7b7406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:29.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:29.720+0000 7f7b6ffff640 1 -- 172.21.15.3:0/3731497954 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f7b8008aab0 con 0x7f7b8815e290 2024-05-09T06:36:29.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:29.720+0000 7f7b877fe640 1 --2- 172.21.15.3:0/3731497954 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f7b7406be30 0x7f7b7406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:29.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:29.720+0000 7f7b877fe640 1 --2- 172.21.15.3:0/3731497954 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f7b7406be30 0x7f7b7406e2f0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f7b78009f30 tx=0x7f7b78009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:29.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:29.724+0000 7f7b6ffff640 1 -- 172.21.15.3:0/3731497954 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f7b8000ae20 con 0x7f7b8815e290 2024-05-09T06:36:29.889 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:29.909+0000 7f7b8dda9640 1 -- 172.21.15.3:0/3731497954 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7b8806de20 con 0x7f7b8815e290 2024-05-09T06:36:29.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:29.910+0000 7f7b6ffff640 1 -- 172.21.15.3:0/3731497954 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16521 (secure 0 0 0) 0x7f7b80058e10 con 0x7f7b8815e290 2024-05-09T06:36:29.890 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-09T06:36:29.890 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":20,"fsid":"16ed737d-2665-431a-8ac6-ef242be3975a","created":"2024-05-09T06:35:42.541368+0000","modified":"2024-05-09T06:36:08.856391+0000","last_up_change":"2024-05-09T06:35:53.778274+0000","last_in_change":"2024-05-09T06:35:44.780586+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T06:35:54.819999+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T06:36:04.931314+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T06:36:06.200433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fae3a8db-4826-4f3c-b33a-12c09bb5df6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":1897849827},{"type":"v1","addr":"172.21.15.3:6817","nonce":1897849827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":1897849827},{"type":"v1","addr":"172.21.15.3:6819","nonce":1897849827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":1897849827},{"type":"v1","addr":"172.21.15.3:6823","nonce":1897849827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":1897849827},{"type":"v1","addr":"172.21.15.3:6821","nonce":1897849827}]},"public_addr":"172.21.15.3:6817/1897849827","cluster_addr":"172.21.15.3:6819/1897849827","heartbeat_back_addr":"172.21.15.3:6823/1897849827","heartbeat_front_addr":"172.21.15.3:6821/1897849827","state":["exists","up"]},{"osd":1,"uuid":"646e882d-4ce4-4720-8c84-f71df563a35e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":926805498},{"type":"v1","addr":"172.21.15.3:6801","nonce":926805498}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":926805498},{"type":"v1","addr":"172.21.15.3:6803","nonce":926805498}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":926805498},{"type":"v1","addr":"172.21.15.3:6807","nonce":926805498}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":926805498},{"type":"v1","addr":"172.21.15.3:6805","nonce":926805498}]},"public_addr":"172.21.15.3:6801/926805498","cluster_addr":"172.21.15.3:6803/926805498","heartbeat_back_addr":"172.21.15.3:6807/926805498","heartbeat_front_addr":"172.21.15.3:6805/926805498","state":["exists","up"]},{"osd":2,"uuid":"1abbadb5-a570-4359-9643-acff3ede35a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":801295451},{"type":"v1","addr":"172.21.15.3:6809","nonce":801295451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":801295451},{"type":"v1","addr":"172.21.15.3:6811","nonce":801295451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":801295451},{"type":"v1","addr":"172.21.15.3:6815","nonce":801295451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":801295451},{"type":"v1","addr":"172.21.15.3:6813","nonce":801295451}]},"public_addr":"172.21.15.3:6809/801295451","cluster_addr":"172.21.15.3:6811/801295451","heartbeat_back_addr":"172.21.15.3:6815/801295451","heartbeat_front_addr":"172.21.15.3:6813/801295451","state":["exists","up"]},{"osd":3,"uuid":"62ac4dc1-2371-4dd2-acd3-e2dbb11aedf5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":1903389857},{"type":"v1","addr":"172.21.15.3:6825","nonce":1903389857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":1903389857},{"type":"v1","addr":"172.21.15.3:6827","nonce":1903389857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":1903389857},{"type":"v1","addr":"172.21.15.3:6831","nonce":1903389857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":1903389857},{"type":"v1","addr":"172.21.15.3:6829","nonce":1903389857}]},"public_addr":"172.21.15.3:6825/1903389857","cluster_addr":"172.21.15.3:6827/1903389857","heartbeat_back_addr":"172.21.15.3:6831/1903389857","heartbeat_front_addr":"172.21.15.3:6829/1903389857","state":["exists","up"]},{"osd":4,"uuid":"ae808398-8326-4d7c-9319-a2a0eafffb59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":2143624576},{"type":"v1","addr":"172.21.15.190:6801","nonce":2143624576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":2143624576},{"type":"v1","addr":"172.21.15.190:6803","nonce":2143624576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":2143624576},{"type":"v1","addr":"172.21.15.190:6807","nonce":2143624576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":2143624576},{"type":"v1","addr":"172.21.15.190:6805","nonce":2143624576}]},"public_addr":"172.21.15.190:6801/2143624576","cluster_addr":"172.21.15.190:6803/2143624576","heartbeat_back_addr":"172.21.15.190:6807/2143624576","heartbeat_front_addr":"172.21.15.190:6805/2143624576","state":["exists","up"]},{"osd":5,"uuid":"918adf50-c440-48ac-9a8d-4837562d7275","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":1766923126},{"type":"v1","addr":"172.21.15.190:6814","nonce":1766923126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":1766923126},{"type":"v1","addr":"172.21.15.190:6818","nonce":1766923126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":1766923126},{"type":"v1","addr":"172.21.15.190:6823","nonce":1766923126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":1766923126},{"type":"v1","addr":"172.21.15.190:6821","nonce":1766923126}]},"public_addr":"172.21.15.190:6814/1766923126","cluster_addr":"172.21.15.190:6818/1766923126","heartbeat_back_addr":"172.21.15.190:6823/1766923126","heartbeat_front_addr":"172.21.15.190:6821/1766923126","state":["exists","up"]},{"osd":6,"uuid":"ac2882bc-2a7c-463c-8f6c-76ff49828adc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":342414248},{"type":"v1","addr":"172.21.15.190:6825","nonce":342414248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":342414248},{"type":"v1","addr":"172.21.15.190:6827","nonce":342414248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":342414248},{"type":"v1","addr":"172.21.15.190:6831","nonce":342414248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":342414248},{"type":"v1","addr":"172.21.15.190:6829","nonce":342414248}]},"public_addr":"172.21.15.190:6825/342414248","cluster_addr":"172.21.15.190:6827/342414248","heartbeat_back_addr":"172.21.15.190:6831/342414248","heartbeat_front_addr":"172.21.15.190:6829/342414248","state":["exists","up"]},{"osd":7,"uuid":"dd4e767a-4730-4c70-8bd5-3f088140a5a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":537849891},{"type":"v1","addr":"172.21.15.190:6809","nonce":537849891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":537849891},{"type":"v1","addr":"172.21.15.190:6811","nonce":537849891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6817","nonce":537849891},{"type":"v1","addr":"172.21.15.190:6819","nonce":537849891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6813","nonce":537849891},{"type":"v1","addr":"172.21.15.190:6815","nonce":537849891}]},"public_addr":"172.21.15.190:6809/537849891","cluster_addr":"172.21.15.190:6811/537849891","heartbeat_back_addr":"172.21.15.190:6819/537849891","heartbeat_front_addr":"172.21.15.190:6815/537849891","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.889784+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.974002+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.973549+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.955396+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.928471+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.902418+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.958696+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.926474+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T06:36:29.892 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:29.912+0000 7f7b8dda9640 1 -- 172.21.15.3:0/3731497954 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f7b7406be30 msgr2=0x7f7b7406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:29.892 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:29.912+0000 7f7b8dda9640 1 --2- 172.21.15.3:0/3731497954 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f7b7406be30 0x7f7b7406e2f0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f7b78009f30 tx=0x7f7b78009190 comp rx=0 tx=0).stop 2024-05-09T06:36:29.892 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:29.912+0000 7f7b8dda9640 1 -- 172.21.15.3:0/3731497954 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7b8815e290 msgr2=0x7f7b88146ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:29.892 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:29.912+0000 7f7b8dda9640 1 --2- 172.21.15.3:0/3731497954 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7b8815e290 0x7f7b88146ea0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f7b8000ca80 tx=0x7f7b8000cf40 comp rx=0 tx=0).stop 2024-05-09T06:36:29.892 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:29.912+0000 7f7b8dda9640 1 -- 172.21.15.3:0/3731497954 shutdown_connections 2024-05-09T06:36:29.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:29.912+0000 7f7b8dda9640 1 --2- 172.21.15.3:0/3731497954 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f7b7406be30 0x7f7b7406e2f0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:29.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:29.912+0000 7f7b8dda9640 1 --2- 172.21.15.3:0/3731497954 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7b8815e290 0x7f7b88146ea0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:29.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:29.912+0000 7f7b8dda9640 1 --2- 172.21.15.3:0/3731497954 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7b880ae7f0 0x7f7b88150f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:29.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:29.912+0000 7f7b8dda9640 1 --2- 172.21.15.3:0/3731497954 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7b8806c1a0 0x7f7b880cedf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:29.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:29.912+0000 7f7b8dda9640 1 -- 172.21.15.3:0/3731497954 >> 172.21.15.3:0/3731497954 conn(0x7f7b880d05d0 msgr2=0x7f7b8806ea00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:29.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:29.912+0000 7f7b8dda9640 1 -- 172.21.15.3:0/3731497954 shutdown_connections 2024-05-09T06:36:29.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:29.912+0000 7f7b8dda9640 1 -- 172.21.15.3:0/3731497954 wait complete. 2024-05-09T06:36:29.904 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-09T06:35:54.819999+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-09T06:36:04.931314+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-09T06:36:06.200433+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-09T06:36:29.904 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-09T06:36:30.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.032+0000 7f4097efc640 1 Processor -- start 2024-05-09T06:36:30.012 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.032+0000 7f4097efc640 1 -- start start 2024-05-09T06:36:30.013 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.033+0000 7f4097efc640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f4090144a10 0x7f40900cf8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:30.013 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.033+0000 7f4097efc640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f4090072900 0x7f4090071080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:30.013 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.033+0000 7f4097efc640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f40900715c0 con 0x7f40900b4130 2024-05-09T06:36:30.013 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.033+0000 7f4097efc640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f4090072470 con 0x7f4090072900 2024-05-09T06:36:30.013 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.033+0000 7f4097efc640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f4090071860 con 0x7f4090144a10 2024-05-09T06:36:30.013 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.033+0000 7f4095497640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f4090072900 0x7f4090071080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:30.014 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.033+0000 7f4096499640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f40900b4130 0x7f409006e250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:47870/0 (socket says 172.21.15.3:47870) 2024-05-09T06:36:30.014 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.033+0000 7f4096499640 1 -- 172.21.15.3:0/1789284668 learned_addr learned my addr 172.21.15.3:0/1789284668 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-05-09T06:36:30.014 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.033+0000 7f4095c98640 1 --2- 172.21.15.3:0/1789284668 >> v2:172.21.15.190:3300/0 conn(0x7f4090144a10 0x7f40900cf8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:30.014 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.033+0000 7f4095497640 1 -- 172.21.15.3:0/1789284668 >> v2:172.21.15.190:3300/0 conn(0x7f4090144a10 msgr2=0x7f40900cf8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:30.014 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.033+0000 7f4095497640 1 --2- 172.21.15.3:0/1789284668 >> v2:172.21.15.190:3300/0 conn(0x7f4090144a10 0x7f40900cf8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:30.014 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.033+0000 7f4095497640 1 -- 172.21.15.3:0/1789284668 >> v1:172.21.15.3:6789/0 conn(0x7f40900b4130 legacy=0x7f409006e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:30.014 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.033+0000 7f4095497640 1 -- 172.21.15.3:0/1789284668 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4090071b40 con 0x7f4090072900 2024-05-09T06:36:30.014 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.033+0000 7f4094c96640 1 -- 172.21.15.3:0/1789284668 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1290277663 0 0) 0x7f40900715c0 con 0x7f40900b4130 2024-05-09T06:36:30.014 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.033+0000 7f4095c98640 1 --2- 172.21.15.3:0/1789284668 >> v2:172.21.15.190:3300/0 conn(0x7f4090144a10 0x7f40900cf8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:36:30.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.034+0000 7f4095497640 1 --2- 172.21.15.3:0/1789284668 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4090072900 0x7f4090071080 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f40900b2860 tx=0x7f408002a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e23a7b5dd20c3bb7 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:30.016 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.034+0000 7f4094c96640 1 -- 172.21.15.3:0/1789284668 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4080037070 con 0x7f4090072900 2024-05-09T06:36:30.016 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.034+0000 7f4094c96640 1 -- 172.21.15.3:0/1789284668 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4080032440 con 0x7f4090072900 2024-05-09T06:36:30.016 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.034+0000 7f4094c96640 1 -- 172.21.15.3:0/1789284668 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f408002ae60 con 0x7f4090072900 2024-05-09T06:36:30.016 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.035+0000 7f4097efc640 1 -- 172.21.15.3:0/1789284668 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4090072900 msgr2=0x7f4090071080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:30.016 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.035+0000 7f4097efc640 1 --2- 172.21.15.3:0/1789284668 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4090072900 0x7f4090071080 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f40900b2860 tx=0x7f408002a110 comp rx=0 tx=0).stop 2024-05-09T06:36:30.016 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.035+0000 7f4097efc640 1 -- 172.21.15.3:0/1789284668 shutdown_connections 2024-05-09T06:36:30.016 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.035+0000 7f4097efc640 1 --2- 172.21.15.3:0/1789284668 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4090072900 0x7f4090071080 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:30.016 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.035+0000 7f4097efc640 1 --2- 172.21.15.3:0/1789284668 >> v2:172.21.15.190:3300/0 conn(0x7f4090144a10 0x7f40900cf8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:30.016 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.035+0000 7f4097efc640 1 -- 172.21.15.3:0/1789284668 >> 172.21.15.3:0/1789284668 conn(0x7f40900b1c20 msgr2=0x7f40900b2030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:30.016 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.035+0000 7f4097efc640 1 -- 172.21.15.3:0/1789284668 shutdown_connections 2024-05-09T06:36:30.016 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.035+0000 7f4097efc640 1 -- 172.21.15.3:0/1789284668 wait complete. 2024-05-09T06:36:30.016 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.036+0000 7f4097efc640 1 Processor -- start 2024-05-09T06:36:30.016 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.036+0000 7f4097efc640 1 -- start start 2024-05-09T06:36:30.018 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.037+0000 7f4097efc640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4090072900 0x7f40901377d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:30.019 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.037+0000 7f4097efc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f40900b4130 0x7f409015d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:30.019 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.037+0000 7f4097efc640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4090144a10 0x7f409013e270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:30.019 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.037+0000 7f4097efc640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f40900b45e0 con 0x7f40900b4130 2024-05-09T06:36:30.019 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.037+0000 7f4097efc640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f4090072100 con 0x7f4090144a10 2024-05-09T06:36:30.019 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.037+0000 7f4097efc640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f40900b4760 con 0x7f4090072900 2024-05-09T06:36:30.019 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.037+0000 7f4095497640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f40900b4130 0x7f409015d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:30.019 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.037+0000 7f4095497640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f40900b4130 0x7f409015d810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50198/0 (socket says 172.21.15.3:50198) 2024-05-09T06:36:30.019 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.037+0000 7f4095497640 1 -- 172.21.15.3:0/1040023219 learned_addr learned my addr 172.21.15.3:0/1040023219 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:36:30.019 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.037+0000 7f4095c98640 1 --2- 172.21.15.3:0/1040023219 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4090072900 0x7f40901377d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:30.019 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.037+0000 7f4096499640 1 --2- 172.21.15.3:0/1040023219 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4090144a10 0x7f409013e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:30.019 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.037+0000 7f4095497640 1 -- 172.21.15.3:0/1040023219 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4090072900 msgr2=0x7f40901377d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:30.019 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.037+0000 7f4095497640 1 --2- 172.21.15.3:0/1040023219 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4090072900 0x7f40901377d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:30.019 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.037+0000 7f4095497640 1 -- 172.21.15.3:0/1040023219 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4090144a10 msgr2=0x7f409013e270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:30.019 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.037+0000 7f4095497640 1 --2- 172.21.15.3:0/1040023219 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4090144a10 0x7f409013e270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:30.020 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.037+0000 7f4095497640 1 -- 172.21.15.3:0/1040023219 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f409015c770 con 0x7f40900b4130 2024-05-09T06:36:30.020 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.037+0000 7f4095497640 1 --2- 172.21.15.3:0/1040023219 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f40900b4130 0x7f409015d810 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f4080000c00 tx=0x7f40800046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:30.020 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.038+0000 7f40867fc640 1 -- 172.21.15.3:0/1040023219 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4080039800 con 0x7f40900b4130 2024-05-09T06:36:30.020 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.038+0000 7f40867fc640 1 -- 172.21.15.3:0/1040023219 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40800399c0 con 0x7f40900b4130 2024-05-09T06:36:30.020 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.038+0000 7f4097efc640 1 -- 172.21.15.3:0/1040023219 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f409015ca50 con 0x7f40900b4130 2024-05-09T06:36:30.020 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.038+0000 7f40867fc640 1 -- 172.21.15.3:0/1040023219 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4080039db0 con 0x7f40900b4130 2024-05-09T06:36:30.020 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.038+0000 7f4097efc640 1 -- 172.21.15.3:0/1040023219 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f409015d010 con 0x7f40900b4130 2024-05-09T06:36:30.020 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.039+0000 7f4097efc640 1 -- 172.21.15.3:0/1040023219 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4090069550 con 0x7f40900b4130 2024-05-09T06:36:30.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.039+0000 7f40867fc640 1 -- 172.21.15.3:0/1040023219 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f408004a060 con 0x7f40900b4130 2024-05-09T06:36:30.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.040+0000 7f40867fc640 1 --2- 172.21.15.3:0/1040023219 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f407406be30 0x7f407406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:30.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.040+0000 7f40867fc640 1 -- 172.21.15.3:0/1040023219 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f40800ad080 con 0x7f40900b4130 2024-05-09T06:36:30.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.040+0000 7f4095c98640 1 --2- 172.21.15.3:0/1040023219 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f407406be30 0x7f407406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:30.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.041+0000 7f4095c98640 1 --2- 172.21.15.3:0/1040023219 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f407406be30 0x7f407406e2f0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f408c00a950 tx=0x7f408c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:30.024 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.044+0000 7f40867fc640 1 -- 172.21.15.3:0/1040023219 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f408003e070 con 0x7f40900b4130 2024-05-09T06:36:30.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.230+0000 7f4097efc640 1 -- 172.21.15.3:0/1040023219 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f40900b2c30 con 0x7f40900b4130 2024-05-09T06:36:30.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.231+0000 7f40867fc640 1 -- 172.21.15.3:0/1040023219 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7f408007ba30 con 0x7f40900b4130 2024-05-09T06:36:30.211 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 1 2024-05-09T06:36:30.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.233+0000 7f4097efc640 1 -- 172.21.15.3:0/1040023219 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f407406be30 msgr2=0x7f407406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:30.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.233+0000 7f4097efc640 1 --2- 172.21.15.3:0/1040023219 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f407406be30 0x7f407406e2f0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f408c00a950 tx=0x7f408c008040 comp rx=0 tx=0).stop 2024-05-09T06:36:30.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.233+0000 7f4097efc640 1 -- 172.21.15.3:0/1040023219 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f40900b4130 msgr2=0x7f409015d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:30.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.233+0000 7f4097efc640 1 --2- 172.21.15.3:0/1040023219 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f40900b4130 0x7f409015d810 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f4080000c00 tx=0x7f40800046c0 comp rx=0 tx=0).stop 2024-05-09T06:36:30.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.233+0000 7f4097efc640 1 -- 172.21.15.3:0/1040023219 shutdown_connections 2024-05-09T06:36:30.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.233+0000 7f4097efc640 1 --2- 172.21.15.3:0/1040023219 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f407406be30 0x7f407406e2f0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:30.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.233+0000 7f4097efc640 1 --2- 172.21.15.3:0/1040023219 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4090144a10 0x7f409013e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:30.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.233+0000 7f4097efc640 1 --2- 172.21.15.3:0/1040023219 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f40900b4130 0x7f409015d810 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:30.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.233+0000 7f4097efc640 1 --2- 172.21.15.3:0/1040023219 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4090072900 0x7f40901377d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:30.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.233+0000 7f4097efc640 1 -- 172.21.15.3:0/1040023219 >> 172.21.15.3:0/1040023219 conn(0x7f40900b1c20 msgr2=0x7f4090074240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:30.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.233+0000 7f4097efc640 1 -- 172.21.15.3:0/1040023219 shutdown_connections 2024-05-09T06:36:30.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.233+0000 7f4097efc640 1 -- 172.21.15.3:0/1040023219 wait complete. 2024-05-09T06:36:30.225 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-09T06:36:30.332 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.352+0000 7f47862a5640 1 Processor -- start 2024-05-09T06:36:30.332 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.352+0000 7f47862a5640 1 -- start start 2024-05-09T06:36:30.333 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.353+0000 7f47862a5640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f478015e420 0x7f47800af390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:30.333 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.353+0000 7f47862a5640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f47800b53b0 0x7f47800b31c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:30.333 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.353+0000 7f47862a5640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f47800add50 con 0x7f47800b3700 2024-05-09T06:36:30.334 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.353+0000 7f47862a5640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f47800af8d0 con 0x7f478015e420 2024-05-09T06:36:30.334 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.353+0000 7f47862a5640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f47800b0a80 con 0x7f47800b53b0 2024-05-09T06:36:30.334 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.353+0000 7f4784842640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f47800b3700 0x7f4780161eb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:47874/0 (socket says 172.21.15.3:47874) 2024-05-09T06:36:30.334 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.353+0000 7f4784842640 1 -- 172.21.15.3:0/2737189810 learned_addr learned my addr 172.21.15.3:0/2737189810 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-05-09T06:36:30.334 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.353+0000 7f477f7fe640 1 --2- 172.21.15.3:0/2737189810 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f47800b53b0 0x7f47800b31c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:30.334 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.353+0000 7f477ffff640 1 --2- 172.21.15.3:0/2737189810 >> v2:172.21.15.190:3300/0 conn(0x7f478015e420 0x7f47800af390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:30.334 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.353+0000 7f477effd640 1 -- 172.21.15.3:0/2737189810 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3943624055 0 0) 0x7f47800add50 con 0x7f47800b3700 2024-05-09T06:36:30.334 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.353+0000 7f477effd640 1 -- 172.21.15.3:0/2737189810 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4768003520 con 0x7f47800b3700 2024-05-09T06:36:30.334 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.353+0000 7f477f7fe640 1 -- 172.21.15.3:0/2737189810 >> v2:172.21.15.190:3300/0 conn(0x7f478015e420 msgr2=0x7f47800af390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:30.334 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.353+0000 7f477f7fe640 1 --2- 172.21.15.3:0/2737189810 >> v2:172.21.15.190:3300/0 conn(0x7f478015e420 0x7f47800af390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:30.334 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.353+0000 7f477f7fe640 1 -- 172.21.15.3:0/2737189810 >> v1:172.21.15.3:6789/0 conn(0x7f47800b3700 legacy=0x7f4780161eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:30.334 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.353+0000 7f477f7fe640 1 -- 172.21.15.3:0/2737189810 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47800ae030 con 0x7f47800b53b0 2024-05-09T06:36:30.334 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.353+0000 7f477ffff640 1 --2- 172.21.15.3:0/2737189810 >> v2:172.21.15.190:3300/0 conn(0x7f478015e420 0x7f47800af390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T06:36:30.336 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.354+0000 7f477f7fe640 1 --2- 172.21.15.3:0/2737189810 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f47800b53b0 0x7f47800b31c0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f47740097a0 tx=0x7f477402a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6cba33b8b64c149a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:30.336 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.354+0000 7f477effd640 1 -- 172.21.15.3:0/2737189810 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4774037070 con 0x7f47800b53b0 2024-05-09T06:36:30.336 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.354+0000 7f477effd640 1 -- 172.21.15.3:0/2737189810 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4774032440 con 0x7f47800b53b0 2024-05-09T06:36:30.336 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.354+0000 7f477effd640 1 -- 172.21.15.3:0/2737189810 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4774032810 con 0x7f47800b53b0 2024-05-09T06:36:30.336 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.355+0000 7f47862a5640 1 -- 172.21.15.3:0/2737189810 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f47800b53b0 msgr2=0x7f47800b31c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:30.336 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.355+0000 7f47862a5640 1 --2- 172.21.15.3:0/2737189810 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f47800b53b0 0x7f47800b31c0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f47740097a0 tx=0x7f477402a180 comp rx=0 tx=0).stop 2024-05-09T06:36:30.336 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.355+0000 7f47862a5640 1 -- 172.21.15.3:0/2737189810 shutdown_connections 2024-05-09T06:36:30.336 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.355+0000 7f47862a5640 1 --2- 172.21.15.3:0/2737189810 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f47800b53b0 0x7f47800b31c0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:30.336 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.355+0000 7f47862a5640 1 --2- 172.21.15.3:0/2737189810 >> v2:172.21.15.190:3300/0 conn(0x7f478015e420 0x7f47800af390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:30.337 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.355+0000 7f47862a5640 1 -- 172.21.15.3:0/2737189810 >> 172.21.15.3:0/2737189810 conn(0x7f47800b5900 msgr2=0x7f4780072890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:30.337 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.355+0000 7f47862a5640 1 -- 172.21.15.3:0/2737189810 shutdown_connections 2024-05-09T06:36:30.337 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.355+0000 7f47862a5640 1 -- 172.21.15.3:0/2737189810 wait complete. 2024-05-09T06:36:30.337 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.356+0000 7f47862a5640 1 Processor -- start 2024-05-09T06:36:30.337 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.356+0000 7f47862a5640 1 -- start start 2024-05-09T06:36:30.340 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.357+0000 7f47862a5640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f47800b3700 0x7f4780153cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:30.340 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.357+0000 7f47862a5640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f47800b53b0 0x7f4780160c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:30.340 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.357+0000 7f47862a5640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f478015e420 0x7f47801611a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:30.340 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.357+0000 7f47862a5640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f47800ae8d0 con 0x7f47800b53b0 2024-05-09T06:36:30.340 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.357+0000 7f47862a5640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f47800ae5f0 con 0x7f47800b3700 2024-05-09T06:36:30.340 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.357+0000 7f47862a5640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f47800aedf0 con 0x7f478015e420 2024-05-09T06:36:30.340 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.357+0000 7f477f7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f47800b53b0 0x7f4780160c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:30.340 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.357+0000 7f477f7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f47800b53b0 0x7f4780160c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50206/0 (socket says 172.21.15.3:50206) 2024-05-09T06:36:30.340 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.357+0000 7f477f7fe640 1 -- 172.21.15.3:0/58402575 learned_addr learned my addr 172.21.15.3:0/58402575 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:36:30.340 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.357+0000 7f4784842640 1 --2- 172.21.15.3:0/58402575 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f478015e420 0x7f47801611a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:30.340 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.357+0000 7f477ffff640 1 --2- 172.21.15.3:0/58402575 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f47800b3700 0x7f4780153cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:30.340 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.357+0000 7f477f7fe640 1 -- 172.21.15.3:0/58402575 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f478015e420 msgr2=0x7f47801611a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:30.340 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.357+0000 7f477f7fe640 1 --2- 172.21.15.3:0/58402575 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f478015e420 0x7f47801611a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:30.341 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.357+0000 7f477f7fe640 1 -- 172.21.15.3:0/58402575 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f47800b3700 msgr2=0x7f4780153cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:30.341 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.358+0000 7f477f7fe640 1 --2- 172.21.15.3:0/58402575 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f47800b3700 0x7f4780153cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:30.341 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.358+0000 7f477f7fe640 1 -- 172.21.15.3:0/58402575 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f478015f3f0 con 0x7f47800b53b0 2024-05-09T06:36:30.341 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.358+0000 7f477f7fe640 1 --2- 172.21.15.3:0/58402575 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f47800b53b0 0x7f4780160c60 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f4774035040 tx=0x7f47740046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:30.341 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.358+0000 7f477cff9640 1 -- 172.21.15.3:0/58402575 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4774041030 con 0x7f47800b53b0 2024-05-09T06:36:30.341 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.358+0000 7f477cff9640 1 -- 172.21.15.3:0/58402575 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f477403d4e0 con 0x7f47800b53b0 2024-05-09T06:36:30.341 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.358+0000 7f47862a5640 1 -- 172.21.15.3:0/58402575 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47800cb100 con 0x7f47800b53b0 2024-05-09T06:36:30.341 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.358+0000 7f477cff9640 1 -- 172.21.15.3:0/58402575 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4774037040 con 0x7f47800b53b0 2024-05-09T06:36:30.341 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.358+0000 7f47862a5640 1 -- 172.21.15.3:0/58402575 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47800cb6c0 con 0x7f47800b53b0 2024-05-09T06:36:30.341 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.359+0000 7f47862a5640 1 -- 172.21.15.3:0/58402575 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47800b1c40 con 0x7f47800b53b0 2024-05-09T06:36:30.341 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.359+0000 7f477cff9640 1 -- 172.21.15.3:0/58402575 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4774044070 con 0x7f47800b53b0 2024-05-09T06:36:30.341 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.360+0000 7f477cff9640 1 --2- 172.21.15.3:0/58402575 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f476c06be30 0x7f476c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:30.341 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.360+0000 7f477cff9640 1 -- 172.21.15.3:0/58402575 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f47740ab4d0 con 0x7f47800b53b0 2024-05-09T06:36:30.342 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.360+0000 7f477ffff640 1 --2- 172.21.15.3:0/58402575 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f476c06be30 0x7f476c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:30.342 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.361+0000 7f477ffff640 1 --2- 172.21.15.3:0/58402575 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f476c06be30 0x7f476c06e2f0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f4770009560 tx=0x7f4770008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:30.344 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.364+0000 7f477cff9640 1 -- 172.21.15.3:0/58402575 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f47800b1c40 con 0x7f47800b53b0 2024-05-09T06:36:30.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.548+0000 7f47862a5640 1 -- 172.21.15.3:0/58402575 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f4780069550 con 0x7f47800b53b0 2024-05-09T06:36:30.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.548+0000 7f477cff9640 1 -- 172.21.15.3:0/58402575 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7f478015f3f0 con 0x7f47800b53b0 2024-05-09T06:36:30.528 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 32 2024-05-09T06:36:30.530 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.550+0000 7f47862a5640 1 -- 172.21.15.3:0/58402575 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f476c06be30 msgr2=0x7f476c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:30.530 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.550+0000 7f47862a5640 1 --2- 172.21.15.3:0/58402575 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f476c06be30 0x7f476c06e2f0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f4770009560 tx=0x7f4770008040 comp rx=0 tx=0).stop 2024-05-09T06:36:30.530 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.550+0000 7f47862a5640 1 -- 172.21.15.3:0/58402575 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f47800b53b0 msgr2=0x7f4780160c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:30.530 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.550+0000 7f47862a5640 1 --2- 172.21.15.3:0/58402575 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f47800b53b0 0x7f4780160c60 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f4774035040 tx=0x7f47740046c0 comp rx=0 tx=0).stop 2024-05-09T06:36:30.530 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.550+0000 7f47862a5640 1 -- 172.21.15.3:0/58402575 shutdown_connections 2024-05-09T06:36:30.530 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.550+0000 7f47862a5640 1 --2- 172.21.15.3:0/58402575 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f476c06be30 0x7f476c06e2f0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:30.530 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.550+0000 7f47862a5640 1 --2- 172.21.15.3:0/58402575 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f478015e420 0x7f47801611a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:30.530 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.550+0000 7f47862a5640 1 --2- 172.21.15.3:0/58402575 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f47800b53b0 0x7f4780160c60 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:30.531 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.550+0000 7f47862a5640 1 --2- 172.21.15.3:0/58402575 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f47800b3700 0x7f4780153cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:30.531 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.550+0000 7f47862a5640 1 -- 172.21.15.3:0/58402575 >> 172.21.15.3:0/58402575 conn(0x7f47800b5900 msgr2=0x7f4780157c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:30.531 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.550+0000 7f47862a5640 1 -- 172.21.15.3:0/58402575 shutdown_connections 2024-05-09T06:36:30.531 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.550+0000 7f47862a5640 1 -- 172.21.15.3:0/58402575 wait complete. 2024-05-09T06:36:30.541 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-09T06:36:30.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.669+0000 7fb31679e640 1 Processor -- start 2024-05-09T06:36:30.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.669+0000 7fb31679e640 1 -- start start 2024-05-09T06:36:30.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.670+0000 7fb31679e640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fb31015e350 0x7fb3100d2700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:30.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.670+0000 7fb31679e640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fb3100d2c40 0x7fb3100cec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:30.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.670+0000 7fb31679e640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb3100cf930 con 0x7fb3100cf190 2024-05-09T06:36:30.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.670+0000 7fb31679e640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fb3100d2060 con 0x7fb3100d2c40 2024-05-09T06:36:30.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.670+0000 7fb31679e640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7fb3100d0540 con 0x7fb31015e350 2024-05-09T06:36:30.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.670+0000 7fb314f9b640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fb3100d2c40 0x7fb3100cec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:30.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.670+0000 7fb314f9b640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fb3100d2c40 0x7fb3100cec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:35362/0 (socket says 172.21.15.3:35362) 2024-05-09T06:36:30.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.670+0000 7fb314f9b640 1 -- 172.21.15.3:0/1976568243 learned_addr learned my addr 172.21.15.3:0/1976568243 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:36:30.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.670+0000 7fb31579c640 1 --2- 172.21.15.3:0/1976568243 >> v2:172.21.15.190:3300/0 conn(0x7fb31015e350 0x7fb3100d2700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:30.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.670+0000 7fb314f9b640 1 -- 172.21.15.3:0/1976568243 >> v2:172.21.15.190:3300/0 conn(0x7fb31015e350 msgr2=0x7fb3100d2700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:30.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.670+0000 7fb314f9b640 1 --2- 172.21.15.3:0/1976568243 >> v2:172.21.15.190:3300/0 conn(0x7fb31015e350 0x7fb3100d2700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:30.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.670+0000 7fb314f9b640 1 -- 172.21.15.3:0/1976568243 >> v1:172.21.15.190:6789/0 conn(0x7fb3100cf190 legacy=0x7fb3100d0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:30.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.670+0000 7fb314f9b640 1 -- 172.21.15.3:0/1976568243 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3100cfc10 con 0x7fb3100d2c40 2024-05-09T06:36:30.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.671+0000 7fb314f9b640 1 --2- 172.21.15.3:0/1976568243 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb3100d2c40 0x7fb3100cec50 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fb31008e2a0 tx=0x7fb30002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5253594d72cbd3f2 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:30.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.671+0000 7fb307fff640 1 -- 172.21.15.3:0/1976568243 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb300037070 con 0x7fb3100d2c40 2024-05-09T06:36:30.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.671+0000 7fb307fff640 1 -- 172.21.15.3:0/1976568243 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb300032440 con 0x7fb3100d2c40 2024-05-09T06:36:30.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.671+0000 7fb307fff640 1 -- 172.21.15.3:0/1976568243 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb300032810 con 0x7fb3100d2c40 2024-05-09T06:36:30.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.672+0000 7fb31679e640 1 -- 172.21.15.3:0/1976568243 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb3100d2c40 msgr2=0x7fb3100cec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:30.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.672+0000 7fb31679e640 1 --2- 172.21.15.3:0/1976568243 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb3100d2c40 0x7fb3100cec50 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fb31008e2a0 tx=0x7fb30002a230 comp rx=0 tx=0).stop 2024-05-09T06:36:30.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.672+0000 7fb31679e640 1 -- 172.21.15.3:0/1976568243 shutdown_connections 2024-05-09T06:36:30.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.672+0000 7fb31679e640 1 --2- 172.21.15.3:0/1976568243 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb3100d2c40 0x7fb3100cec50 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:30.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.672+0000 7fb31679e640 1 --2- 172.21.15.3:0/1976568243 >> v2:172.21.15.190:3300/0 conn(0x7fb31015e350 0x7fb3100d2700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:30.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.672+0000 7fb31679e640 1 -- 172.21.15.3:0/1976568243 >> 172.21.15.3:0/1976568243 conn(0x7fb31008bfc0 msgr2=0x7fb31009d410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:30.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.672+0000 7fb31679e640 1 -- 172.21.15.3:0/1976568243 shutdown_connections 2024-05-09T06:36:30.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.672+0000 7fb31679e640 1 -- 172.21.15.3:0/1976568243 wait complete. 2024-05-09T06:36:30.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.673+0000 7fb31679e640 1 Processor -- start 2024-05-09T06:36:30.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.673+0000 7fb31679e640 1 -- start start 2024-05-09T06:36:30.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.674+0000 7fb31679e640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb3100cf190 0x7fb3100cca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:30.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.674+0000 7fb31679e640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb3100d2c40 0x7fb3101515f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:30.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.674+0000 7fb31679e640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb31015e350 0x7fb310151b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:30.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.674+0000 7fb31679e640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fb3100cf620 con 0x7fb3100cf190 2024-05-09T06:36:30.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.674+0000 7fb31679e640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fb3100d01d0 con 0x7fb31015e350 2024-05-09T06:36:30.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.674+0000 7fb31679e640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fb31015e730 con 0x7fb3100d2c40 2024-05-09T06:36:30.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.674+0000 7fb31579c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb3100cf190 0x7fb3100cca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:30.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.674+0000 7fb31579c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb3100cf190 0x7fb3100cca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:35368/0 (socket says 172.21.15.3:35368) 2024-05-09T06:36:30.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.674+0000 7fb31579c640 1 -- 172.21.15.3:0/3920595971 learned_addr learned my addr 172.21.15.3:0/3920595971 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:36:30.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.674+0000 7fb314f9b640 1 --2- 172.21.15.3:0/3920595971 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb3100d2c40 0x7fb3101515f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:30.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.674+0000 7fb315f9d640 1 --2- 172.21.15.3:0/3920595971 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb31015e350 0x7fb310151b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:30.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.674+0000 7fb31579c640 1 -- 172.21.15.3:0/3920595971 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb3100d2c40 msgr2=0x7fb3101515f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:30.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.674+0000 7fb31579c640 1 --2- 172.21.15.3:0/3920595971 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb3100d2c40 0x7fb3101515f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:30.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.674+0000 7fb31579c640 1 -- 172.21.15.3:0/3920595971 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb31015e350 msgr2=0x7fb310151b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:30.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.674+0000 7fb31579c640 1 --2- 172.21.15.3:0/3920595971 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb31015e350 0x7fb310151b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:30.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.674+0000 7fb31579c640 1 -- 172.21.15.3:0/3920595971 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb31015a6c0 con 0x7fb3100cf190 2024-05-09T06:36:30.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.675+0000 7fb31579c640 1 --2- 172.21.15.3:0/3920595971 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb3100cf190 0x7fb3100cca70 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fb30c00d830 tx=0x7fb30c00dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:30.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.675+0000 7fb305ffb640 1 -- 172.21.15.3:0/3920595971 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb30c00b6c0 con 0x7fb3100cf190 2024-05-09T06:36:30.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.675+0000 7fb305ffb640 1 -- 172.21.15.3:0/3920595971 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb30c019070 con 0x7fb3100cf190 2024-05-09T06:36:30.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.675+0000 7fb305ffb640 1 -- 172.21.15.3:0/3920595971 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb30c00ba80 con 0x7fb3100cf190 2024-05-09T06:36:30.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.675+0000 7fb31679e640 1 -- 172.21.15.3:0/3920595971 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb31015a9a0 con 0x7fb3100cf190 2024-05-09T06:36:30.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.675+0000 7fb31679e640 1 -- 172.21.15.3:0/3920595971 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb31015f7f0 con 0x7fb3100cf190 2024-05-09T06:36:30.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.676+0000 7fb31679e640 1 -- 172.21.15.3:0/3920595971 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb310069550 con 0x7fb3100cf190 2024-05-09T06:36:30.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.676+0000 7fb305ffb640 1 -- 172.21.15.3:0/3920595971 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb30c01e460 con 0x7fb3100cf190 2024-05-09T06:36:30.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.677+0000 7fb305ffb640 1 --2- 172.21.15.3:0/3920595971 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fb2f406be30 0x7fb2f406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:30.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.677+0000 7fb305ffb640 1 -- 172.21.15.3:0/3920595971 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fb30c08b540 con 0x7fb3100cf190 2024-05-09T06:36:30.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.677+0000 7fb314f9b640 1 --2- 172.21.15.3:0/3920595971 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fb2f406be30 0x7fb2f406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:30.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.677+0000 7fb314f9b640 1 --2- 172.21.15.3:0/3920595971 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fb2f406be30 0x7fb2f406e2f0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fb300034000 tx=0x7fb30002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:30.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.681+0000 7fb305ffb640 1 -- 172.21.15.3:0/3920595971 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fb30c059ef0 con 0x7fb3100cf190 2024-05-09T06:36:30.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.867+0000 7fb31679e640 1 -- 172.21.15.3:0/3920595971 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fb3100c9d60 con 0x7fb3100cf190 2024-05-09T06:36:30.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.867+0000 7fb305ffb640 1 -- 172.21.15.3:0/3920595971 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7fb30c0600c0 con 0x7fb3100cf190 2024-05-09T06:36:30.847 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 64 2024-05-09T06:36:30.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.869+0000 7fb31679e640 1 -- 172.21.15.3:0/3920595971 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fb2f406be30 msgr2=0x7fb2f406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:30.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.869+0000 7fb31679e640 1 --2- 172.21.15.3:0/3920595971 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fb2f406be30 0x7fb2f406e2f0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fb300034000 tx=0x7fb30002e000 comp rx=0 tx=0).stop 2024-05-09T06:36:30.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.869+0000 7fb31679e640 1 -- 172.21.15.3:0/3920595971 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb3100cf190 msgr2=0x7fb3100cca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:30.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.869+0000 7fb31679e640 1 --2- 172.21.15.3:0/3920595971 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb3100cf190 0x7fb3100cca70 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fb30c00d830 tx=0x7fb30c00dcf0 comp rx=0 tx=0).stop 2024-05-09T06:36:30.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.869+0000 7fb31679e640 1 -- 172.21.15.3:0/3920595971 shutdown_connections 2024-05-09T06:36:30.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.869+0000 7fb31679e640 1 --2- 172.21.15.3:0/3920595971 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fb2f406be30 0x7fb2f406e2f0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:30.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.869+0000 7fb31679e640 1 --2- 172.21.15.3:0/3920595971 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb31015e350 0x7fb310151b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:30.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.869+0000 7fb31679e640 1 --2- 172.21.15.3:0/3920595971 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb3100d2c40 0x7fb3101515f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:30.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.869+0000 7fb31679e640 1 --2- 172.21.15.3:0/3920595971 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb3100cf190 0x7fb3100cca70 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:30.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.869+0000 7fb31679e640 1 -- 172.21.15.3:0/3920595971 >> 172.21.15.3:0/3920595971 conn(0x7fb31008bfc0 msgr2=0x7fb31006bf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:30.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.869+0000 7fb31679e640 1 -- 172.21.15.3:0/3920595971 shutdown_connections 2024-05-09T06:36:30.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.869+0000 7fb31679e640 1 -- 172.21.15.3:0/3920595971 wait complete. 2024-05-09T06:36:30.860 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T06:36:30.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.987+0000 7f6b1fbee640 1 Processor -- start 2024-05-09T06:36:30.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.987+0000 7f6b1fbee640 1 -- start start 2024-05-09T06:36:30.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.988+0000 7f6b1fbee640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6b180b4d50 0x7f6b18168f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:30.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.988+0000 7f6b1fbee640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6b180ae600 con 0x7f6b1815e420 2024-05-09T06:36:30.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.988+0000 7f6b1fbee640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6b180b5290 con 0x7f6b1806b960 2024-05-09T06:36:30.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.988+0000 7f6b1fbee640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f6b1815e800 con 0x7f6b180b4d50 2024-05-09T06:36:30.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.988+0000 7f6b1d98a640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f6b1815e420 0x7f6b1806b250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:58896/0 (socket says 172.21.15.3:58896) 2024-05-09T06:36:30.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.988+0000 7f6b1d98a640 1 -- 172.21.15.3:0/4141438981 learned_addr learned my addr 172.21.15.3:0/4141438981 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-05-09T06:36:30.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.988+0000 7f6b1e18b640 1 --2- 172.21.15.3:0/4141438981 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6b180b4d50 0x7f6b18168f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:30.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.988+0000 7f6b1c988640 1 -- 172.21.15.3:0/4141438981 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2775413316 0 0) 0x7f6b180ae600 con 0x7f6b1815e420 2024-05-09T06:36:30.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.988+0000 7f6b1c988640 1 -- 172.21.15.3:0/4141438981 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6b00003520 con 0x7f6b1815e420 2024-05-09T06:36:30.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.989+0000 7f6b1c988640 1 -- 172.21.15.3:0/4141438981 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2543221190 0 0) 0x7f6b00003520 con 0x7f6b1815e420 2024-05-09T06:36:30.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.989+0000 7f6b1c988640 1 -- 172.21.15.3:0/4141438981 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6b180b4d50 msgr2=0x7f6b18168f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:30.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.989+0000 7f6b1c988640 1 --2- 172.21.15.3:0/4141438981 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6b180b4d50 0x7f6b18168f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:30.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.989+0000 7f6b1c988640 1 -- 172.21.15.3:0/4141438981 >> v1:172.21.15.190:6789/0 conn(0x7f6b1806b960 legacy=0x7f6b180b4640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:30.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.989+0000 7f6b1c988640 1 -- 172.21.15.3:0/4141438981 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b180b5570 con 0x7f6b1815e420 2024-05-09T06:36:30.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.989+0000 7f6b1c988640 1 -- 172.21.15.3:0/4141438981 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 563312031 0 0) 0x7f6b10001670 con 0x7f6b1815e420 2024-05-09T06:36:30.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.989+0000 7f6b1c988640 1 -- 172.21.15.3:0/4141438981 >> v1:172.21.15.3:6789/0 conn(0x7f6b1815e420 legacy=0x7f6b1806b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:30.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.989+0000 7f6b1c988640 1 --2- 172.21.15.3:0/4141438981 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6b000039e0 0x7f6b00003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:30.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.989+0000 7f6b1c988640 1 --2- 172.21.15.3:0/4141438981 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6b180b4d50 0x7f6b00004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:30.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.989+0000 7f6b1c988640 1 --2- 172.21.15.3:0/4141438981 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6b00004d40 0x7f6b000051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:30.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.989+0000 7f6b1c988640 1 -- 172.21.15.3:0/4141438981 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f6b1815e800 con 0x7f6b000039e0 2024-05-09T06:36:30.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.989+0000 7f6b1c988640 1 -- 172.21.15.3:0/4141438981 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f6b0000ba70 con 0x7f6b00004d40 2024-05-09T06:36:30.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.989+0000 7f6b1c988640 1 -- 172.21.15.3:0/4141438981 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f6b0000bc70 con 0x7f6b180b4d50 2024-05-09T06:36:30.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.989+0000 7f6b1d98a640 1 --2- 172.21.15.3:0/4141438981 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6b000039e0 0x7f6b00003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:30.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.989+0000 7f6b1e18b640 1 --2- 172.21.15.3:0/4141438981 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6b00004d40 0x7f6b000051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:30.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.989+0000 7f6b1d189640 1 --2- 172.21.15.3:0/4141438981 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6b180b4d50 0x7f6b00004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:30.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.989+0000 7f6b1d98a640 1 -- 172.21.15.3:0/4141438981 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6b180b4d50 msgr2=0x7f6b00004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:30.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.989+0000 7f6b1d98a640 1 --2- 172.21.15.3:0/4141438981 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6b180b4d50 0x7f6b00004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:30.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.989+0000 7f6b1d98a640 1 -- 172.21.15.3:0/4141438981 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6b00004d40 msgr2=0x7f6b000051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:30.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.989+0000 7f6b1d98a640 1 --2- 172.21.15.3:0/4141438981 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6b00004d40 0x7f6b000051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:30.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.989+0000 7f6b1d98a640 1 -- 172.21.15.3:0/4141438981 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b00003520 con 0x7f6b000039e0 2024-05-09T06:36:30.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.990+0000 7f6b1d98a640 1 --2- 172.21.15.3:0/4141438981 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6b000039e0 0x7f6b00003dc0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f6b1000c240 tx=0x7f6b00005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42cead15925153d7 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:30.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.990+0000 7f6b1c988640 1 -- 172.21.15.3:0/4141438981 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b1002e070 con 0x7f6b000039e0 2024-05-09T06:36:30.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.990+0000 7f6b1c988640 1 -- 172.21.15.3:0/4141438981 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b10005af0 con 0x7f6b000039e0 2024-05-09T06:36:30.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.990+0000 7f6b1c988640 1 -- 172.21.15.3:0/4141438981 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b10007d30 con 0x7f6b000039e0 2024-05-09T06:36:30.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.991+0000 7f6b1fbee640 1 -- 172.21.15.3:0/4141438981 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6b000039e0 msgr2=0x7f6b00003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:30.978 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.991+0000 7f6b1fbee640 1 --2- 172.21.15.3:0/4141438981 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6b000039e0 0x7f6b00003dc0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f6b1000c240 tx=0x7f6b00005830 comp rx=0 tx=0).stop 2024-05-09T06:36:30.978 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.991+0000 7f6b1e18b640 1 -- 172.21.15.3:0/4141438981 reap_dead start 2024-05-09T06:36:30.978 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.991+0000 7f6b1fbee640 1 -- 172.21.15.3:0/4141438981 shutdown_connections 2024-05-09T06:36:30.978 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.991+0000 7f6b1fbee640 1 -- 172.21.15.3:0/4141438981 >> 172.21.15.3:0/4141438981 conn(0x7f6b1806e3d0 msgr2=0x7f6b1806e7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:30.978 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.991+0000 7f6b1fbee640 1 -- 172.21.15.3:0/4141438981 shutdown_connections 2024-05-09T06:36:30.978 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.991+0000 7f6b1fbee640 1 -- 172.21.15.3:0/4141438981 wait complete. 2024-05-09T06:36:30.978 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.992+0000 7f6b1fbee640 1 Processor -- start 2024-05-09T06:36:30.979 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.992+0000 7f6b1fbee640 1 -- start start 2024-05-09T06:36:30.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.993+0000 7f6b1fbee640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6b000039e0 0x7f6b181597c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:30.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.993+0000 7f6b1fbee640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6b181531f0 0x7f6b18153aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:30.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.993+0000 7f6b1fbee640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6b18160f10 0x7f6b18161370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:30.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.993+0000 7f6b1fbee640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f6b180afaf0 con 0x7f6b181531f0 2024-05-09T06:36:30.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.993+0000 7f6b1fbee640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f6b180b5b30 con 0x7f6b000039e0 2024-05-09T06:36:30.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.993+0000 7f6b1fbee640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f6b180afc70 con 0x7f6b18160f10 2024-05-09T06:36:30.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.993+0000 7f6b1d189640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6b181531f0 0x7f6b18153aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:30.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.993+0000 7f6b1d189640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6b181531f0 0x7f6b18153aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:35384/0 (socket says 172.21.15.3:35384) 2024-05-09T06:36:30.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.993+0000 7f6b1d189640 1 -- 172.21.15.3:0/2696155540 learned_addr learned my addr 172.21.15.3:0/2696155540 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:36:30.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.993+0000 7f6b1d98a640 1 --2- 172.21.15.3:0/2696155540 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6b000039e0 0x7f6b181597c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:30.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.993+0000 7f6b1e18b640 1 --2- 172.21.15.3:0/2696155540 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6b18160f10 0x7f6b18161370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:30.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.993+0000 7f6b1d189640 1 -- 172.21.15.3:0/2696155540 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6b18160f10 msgr2=0x7f6b18161370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:30.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.993+0000 7f6b1d189640 1 --2- 172.21.15.3:0/2696155540 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6b18160f10 0x7f6b18161370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:30.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.993+0000 7f6b1d189640 1 -- 172.21.15.3:0/2696155540 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6b000039e0 msgr2=0x7f6b181597c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:30.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.993+0000 7f6b1d189640 1 --2- 172.21.15.3:0/2696155540 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6b000039e0 0x7f6b181597c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:30.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.993+0000 7f6b1d189640 1 -- 172.21.15.3:0/2696155540 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b181f49f0 con 0x7f6b181531f0 2024-05-09T06:36:30.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.993+0000 7f6b1e18b640 1 --2- 172.21.15.3:0/2696155540 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6b18160f10 0x7f6b18161370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:36:30.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.993+0000 7f6b1d189640 1 --2- 172.21.15.3:0/2696155540 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6b181531f0 0x7f6b18153aa0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f6b0c00f830 tx=0x7f6b0c00fcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:30.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.993+0000 7f6b167fc640 1 -- 172.21.15.3:0/2696155540 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b0c0052e0 con 0x7f6b181531f0 2024-05-09T06:36:30.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.994+0000 7f6b167fc640 1 -- 172.21.15.3:0/2696155540 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b0c018070 con 0x7f6b181531f0 2024-05-09T06:36:30.984 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.994+0000 7f6b1fbee640 1 -- 172.21.15.3:0/2696155540 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b181f3c30 con 0x7f6b181531f0 2024-05-09T06:36:30.984 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.994+0000 7f6b167fc640 1 -- 172.21.15.3:0/2696155540 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b0c00d8f0 con 0x7f6b181531f0 2024-05-09T06:36:30.984 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.994+0000 7f6b1fbee640 1 -- 172.21.15.3:0/2696155540 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b181f41f0 con 0x7f6b181531f0 2024-05-09T06:36:30.984 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.994+0000 7f6b1fbee640 1 -- 172.21.15.3:0/2696155540 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b1804f660 con 0x7f6b181531f0 2024-05-09T06:36:30.984 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.995+0000 7f6b167fc640 1 -- 172.21.15.3:0/2696155540 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6b0c00da90 con 0x7f6b181531f0 2024-05-09T06:36:30.984 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.995+0000 7f6b167fc640 1 --2- 172.21.15.3:0/2696155540 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f6b0406be30 0x7f6b0406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:30.984 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.996+0000 7f6b167fc640 1 -- 172.21.15.3:0/2696155540 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f6b0c08ad80 con 0x7f6b181531f0 2024-05-09T06:36:30.984 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.996+0000 7f6b1d98a640 1 --2- 172.21.15.3:0/2696155540 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f6b0406be30 0x7f6b0406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:30.984 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:30.996+0000 7f6b1d98a640 1 --2- 172.21.15.3:0/2696155540 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f6b0406be30 0x7f6b0406e2f0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f6b10033000 tx=0x7f6b1002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:30.984 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:31.000+0000 7f6b167fc640 1 -- 172.21.15.3:0/2696155540 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f6b0c059730 con 0x7f6b181531f0 2024-05-09T06:36:31.164 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:31.184+0000 7f6b1fbee640 1 -- 172.21.15.3:0/2696155540 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6b1815d7f0 con 0x7f6b181531f0 2024-05-09T06:36:31.165 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:31.185+0000 7f6b167fc640 1 -- 172.21.15.3:0/2696155540 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16521 (secure 0 0 0) 0x7f6b0c059030 con 0x7f6b181531f0 2024-05-09T06:36:31.165 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-09T06:36:31.165 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":20,"fsid":"16ed737d-2665-431a-8ac6-ef242be3975a","created":"2024-05-09T06:35:42.541368+0000","modified":"2024-05-09T06:36:08.856391+0000","last_up_change":"2024-05-09T06:35:53.778274+0000","last_in_change":"2024-05-09T06:35:44.780586+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T06:35:54.819999+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T06:36:04.931314+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T06:36:06.200433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fae3a8db-4826-4f3c-b33a-12c09bb5df6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":1897849827},{"type":"v1","addr":"172.21.15.3:6817","nonce":1897849827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":1897849827},{"type":"v1","addr":"172.21.15.3:6819","nonce":1897849827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":1897849827},{"type":"v1","addr":"172.21.15.3:6823","nonce":1897849827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":1897849827},{"type":"v1","addr":"172.21.15.3:6821","nonce":1897849827}]},"public_addr":"172.21.15.3:6817/1897849827","cluster_addr":"172.21.15.3:6819/1897849827","heartbeat_back_addr":"172.21.15.3:6823/1897849827","heartbeat_front_addr":"172.21.15.3:6821/1897849827","state":["exists","up"]},{"osd":1,"uuid":"646e882d-4ce4-4720-8c84-f71df563a35e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":926805498},{"type":"v1","addr":"172.21.15.3:6801","nonce":926805498}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":926805498},{"type":"v1","addr":"172.21.15.3:6803","nonce":926805498}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":926805498},{"type":"v1","addr":"172.21.15.3:6807","nonce":926805498}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":926805498},{"type":"v1","addr":"172.21.15.3:6805","nonce":926805498}]},"public_addr":"172.21.15.3:6801/926805498","cluster_addr":"172.21.15.3:6803/926805498","heartbeat_back_addr":"172.21.15.3:6807/926805498","heartbeat_front_addr":"172.21.15.3:6805/926805498","state":["exists","up"]},{"osd":2,"uuid":"1abbadb5-a570-4359-9643-acff3ede35a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":801295451},{"type":"v1","addr":"172.21.15.3:6809","nonce":801295451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":801295451},{"type":"v1","addr":"172.21.15.3:6811","nonce":801295451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":801295451},{"type":"v1","addr":"172.21.15.3:6815","nonce":801295451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":801295451},{"type":"v1","addr":"172.21.15.3:6813","nonce":801295451}]},"public_addr":"172.21.15.3:6809/801295451","cluster_addr":"172.21.15.3:6811/801295451","heartbeat_back_addr":"172.21.15.3:6815/801295451","heartbeat_front_addr":"172.21.15.3:6813/801295451","state":["exists","up"]},{"osd":3,"uuid":"62ac4dc1-2371-4dd2-acd3-e2dbb11aedf5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":1903389857},{"type":"v1","addr":"172.21.15.3:6825","nonce":1903389857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":1903389857},{"type":"v1","addr":"172.21.15.3:6827","nonce":1903389857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":1903389857},{"type":"v1","addr":"172.21.15.3:6831","nonce":1903389857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":1903389857},{"type":"v1","addr":"172.21.15.3:6829","nonce":1903389857}]},"public_addr":"172.21.15.3:6825/1903389857","cluster_addr":"172.21.15.3:6827/1903389857","heartbeat_back_addr":"172.21.15.3:6831/1903389857","heartbeat_front_addr":"172.21.15.3:6829/1903389857","state":["exists","up"]},{"osd":4,"uuid":"ae808398-8326-4d7c-9319-a2a0eafffb59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":2143624576},{"type":"v1","addr":"172.21.15.190:6801","nonce":2143624576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":2143624576},{"type":"v1","addr":"172.21.15.190:6803","nonce":2143624576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":2143624576},{"type":"v1","addr":"172.21.15.190:6807","nonce":2143624576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":2143624576},{"type":"v1","addr":"172.21.15.190:6805","nonce":2143624576}]},"public_addr":"172.21.15.190:6801/2143624576","cluster_addr":"172.21.15.190:6803/2143624576","heartbeat_back_addr":"172.21.15.190:6807/2143624576","heartbeat_front_addr":"172.21.15.190:6805/2143624576","state":["exists","up"]},{"osd":5,"uuid":"918adf50-c440-48ac-9a8d-4837562d7275","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":1766923126},{"type":"v1","addr":"172.21.15.190:6814","nonce":1766923126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":1766923126},{"type":"v1","addr":"172.21.15.190:6818","nonce":1766923126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":1766923126},{"type":"v1","addr":"172.21.15.190:6823","nonce":1766923126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":1766923126},{"type":"v1","addr":"172.21.15.190:6821","nonce":1766923126}]},"public_addr":"172.21.15.190:6814/1766923126","cluster_addr":"172.21.15.190:6818/1766923126","heartbeat_back_addr":"172.21.15.190:6823/1766923126","heartbeat_front_addr":"172.21.15.190:6821/1766923126","state":["exists","up"]},{"osd":6,"uuid":"ac2882bc-2a7c-463c-8f6c-76ff49828adc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":342414248},{"type":"v1","addr":"172.21.15.190:6825","nonce":342414248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":342414248},{"type":"v1","addr":"172.21.15.190:6827","nonce":342414248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":342414248},{"type":"v1","addr":"172.21.15.190:6831","nonce":342414248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":342414248},{"type":"v1","addr":"172.21.15.190:6829","nonce":342414248}]},"public_addr":"172.21.15.190:6825/342414248","cluster_addr":"172.21.15.190:6827/342414248","heartbeat_back_addr":"172.21.15.190:6831/342414248","heartbeat_front_addr":"172.21.15.190:6829/342414248","state":["exists","up"]},{"osd":7,"uuid":"dd4e767a-4730-4c70-8bd5-3f088140a5a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":537849891},{"type":"v1","addr":"172.21.15.190:6809","nonce":537849891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":537849891},{"type":"v1","addr":"172.21.15.190:6811","nonce":537849891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6817","nonce":537849891},{"type":"v1","addr":"172.21.15.190:6819","nonce":537849891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6813","nonce":537849891},{"type":"v1","addr":"172.21.15.190:6815","nonce":537849891}]},"public_addr":"172.21.15.190:6809/537849891","cluster_addr":"172.21.15.190:6811/537849891","heartbeat_back_addr":"172.21.15.190:6819/537849891","heartbeat_front_addr":"172.21.15.190:6815/537849891","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.889784+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.974002+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.973549+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.955396+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.928471+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.902418+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.958696+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.926474+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T06:36:31.167 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:31.187+0000 7f6b1fbee640 1 -- 172.21.15.3:0/2696155540 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f6b0406be30 msgr2=0x7f6b0406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:31.167 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:31.187+0000 7f6b1fbee640 1 --2- 172.21.15.3:0/2696155540 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f6b0406be30 0x7f6b0406e2f0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f6b10033000 tx=0x7f6b1002e000 comp rx=0 tx=0).stop 2024-05-09T06:36:31.167 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:31.187+0000 7f6b1fbee640 1 -- 172.21.15.3:0/2696155540 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6b181531f0 msgr2=0x7f6b18153aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:31.167 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:31.187+0000 7f6b1fbee640 1 --2- 172.21.15.3:0/2696155540 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6b181531f0 0x7f6b18153aa0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f6b0c00f830 tx=0x7f6b0c00fcf0 comp rx=0 tx=0).stop 2024-05-09T06:36:31.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:31.187+0000 7f6b1fbee640 1 -- 172.21.15.3:0/2696155540 shutdown_connections 2024-05-09T06:36:31.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:31.187+0000 7f6b1fbee640 1 --2- 172.21.15.3:0/2696155540 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f6b0406be30 0x7f6b0406e2f0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:31.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:31.187+0000 7f6b1fbee640 1 --2- 172.21.15.3:0/2696155540 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6b18160f10 0x7f6b18161370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:31.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:31.187+0000 7f6b1fbee640 1 --2- 172.21.15.3:0/2696155540 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6b181531f0 0x7f6b18153aa0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:31.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:31.187+0000 7f6b1fbee640 1 --2- 172.21.15.3:0/2696155540 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6b000039e0 0x7f6b181597c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:31.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:31.187+0000 7f6b1fbee640 1 -- 172.21.15.3:0/2696155540 >> 172.21.15.3:0/2696155540 conn(0x7f6b1806e3d0 msgr2=0x7f6b1815cb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:31.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:31.187+0000 7f6b1fbee640 1 -- 172.21.15.3:0/2696155540 shutdown_connections 2024-05-09T06:36:31.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:31.187+0000 7f6b1fbee640 1 -- 172.21.15.3:0/2696155540 wait complete. 2024-05-09T06:36:31.179 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-09T06:35:54.819999+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-09T06:36:04.931314+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-09T06:36:06.200433+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-09T06:36:31.180 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-09T06:36:31.287 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:31.307+0000 7ffa1419e640 1 Processor -- start 2024-05-09T06:36:31.287 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:31.308+0000 7ffa1419e640 1 -- start start 2024-05-09T06:36:31.289 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:31.308+0000 7ffa1419e640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffa0c15e5c0 0x7ffa0c0d2800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:31.289 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:31.308+0000 7ffa1419e640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ffa0c0d4c40 0x7ffa0c0d0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:31.289 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:31.308+0000 7ffa1419e640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7ffa0c0d1130 0x7ffa0c0ced30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:31.289 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:31.308+0000 7ffa1419e640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7ffa0c15e9a0 con 0x7ffa0c0d4c40 2024-05-09T06:36:31.289 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:31.308+0000 7ffa1419e640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7ffa0c0d31c0 con 0x7ffa0c0d1130 2024-05-09T06:36:31.289 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:31.308+0000 7ffa1419e640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7ffa0c0d50a0 con 0x7ffa0c15e5c0 2024-05-09T06:36:31.289 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:31.309+0000 7ffa11739640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ffa0c0d4c40 0x7ffa0c0d0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:31.289 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:31.309+0000 7ffa11739640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ffa0c0d4c40 0x7ffa0c0d0bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:35394/0 (socket says 172.21.15.3:35394) 2024-05-09T06:36:31.289 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:31.309+0000 7ffa11739640 1 -- 172.21.15.3:0/72357792 learned_addr learned my addr 172.21.15.3:0/72357792 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:36:31.289 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:31.309+0000 7ffa11f3a640 1 --2- 172.21.15.3:0/72357792 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffa0c15e5c0 0x7ffa0c0d2800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:31.290 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:31.309+0000 7ffa1273b640 1 --2- 172.21.15.3:0/72357792 >> v2:172.21.15.190:3300/0 conn(0x7ffa0c0d1130 0x7ffa0c0ced30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:31.290 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:31.309+0000 7ffa11739640 1 -- 172.21.15.3:0/72357792 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffa0c15e5c0 msgr2=0x7ffa0c0d2800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:31.290 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:31.309+0000 7ffa11739640 1 --2- 172.21.15.3:0/72357792 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffa0c15e5c0 0x7ffa0c0d2800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:31.290 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:31.309+0000 7ffa11739640 1 -- 172.21.15.3:0/72357792 >> v2:172.21.15.190:3300/0 conn(0x7ffa0c0d1130 msgr2=0x7ffa0c0ced30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:31.290 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:31.309+0000 7ffa11739640 1 --2- 172.21.15.3:0/72357792 >> v2:172.21.15.190:3300/0 conn(0x7ffa0c0d1130 0x7ffa0c0ced30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:31.290 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:31.309+0000 7ffa11739640 1 -- 172.21.15.3:0/72357792 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa0c0d2ee0 con 0x7ffa0c0d4c40 2024-05-09T06:36:31.291 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:31.309+0000 7ffa11739640 1 --2- 172.21.15.3:0/72357792 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffa0c0d4c40 0x7ffa0c0d0bf0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7ffa0c0cb250 tx=0x7ffa0002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b7528d9e3c6c16d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:31.291 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:31.310+0000 7ffa10f38640 1 -- 172.21.15.3:0/72357792 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa00037070 con 0x7ffa0c0d4c40 2024-05-09T06:36:31.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:31.310+0000 7ffa10f38640 1 -- 172.21.15.3:0/72357792 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa00032440 con 0x7ffa0c0d4c40 2024-05-09T06:36:31.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:31.310+0000 7ffa10f38640 1 -- 172.21.15.3:0/72357792 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa00032810 con 0x7ffa0c0d4c40 2024-05-09T06:36:31.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:31.310+0000 7ffa1419e640 1 -- 172.21.15.3:0/72357792 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffa0c0d4c40 msgr2=0x7ffa0c0d0bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:31.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:31.310+0000 7ffa1419e640 1 --2- 172.21.15.3:0/72357792 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffa0c0d4c40 0x7ffa0c0d0bf0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7ffa0c0cb250 tx=0x7ffa0002a1e0 comp rx=0 tx=0).stop 2024-05-09T06:36:31.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:31.310+0000 7ffa1419e640 1 -- 172.21.15.3:0/72357792 shutdown_connections 2024-05-09T06:36:31.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:31.310+0000 7ffa1419e640 1 --2- 172.21.15.3:0/72357792 >> v2:172.21.15.190:3300/0 conn(0x7ffa0c0d1130 0x7ffa0c0ced30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:31.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:31.310+0000 7ffa1419e640 1 --2- 172.21.15.3:0/72357792 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffa0c0d4c40 0x7ffa0c0d0bf0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:31.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:31.310+0000 7ffa1419e640 1 --2- 172.21.15.3:0/72357792 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffa0c15e5c0 0x7ffa0c0d2800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:31.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:31.310+0000 7ffa1419e640 1 -- 172.21.15.3:0/72357792 >> 172.21.15.3:0/72357792 conn(0x7ffa0c09d500 msgr2=0x7ffa0c0d4220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:31.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:31.310+0000 7ffa1419e640 1 -- 172.21.15.3:0/72357792 shutdown_connections 2024-05-09T06:36:31.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:31.310+0000 7ffa1419e640 1 -- 172.21.15.3:0/72357792 wait complete. 2024-05-09T06:36:31.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:31.312+0000 7ffa1419e640 1 Processor -- start 2024-05-09T06:36:31.293 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:31.312+0000 7ffa1419e640 1 -- start start 2024-05-09T06:36:31.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:31.312+0000 7ffa1419e640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffa0c0d1130 0x7ffa0c0cdfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:31.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:31.312+0000 7ffa1419e640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffa0c0d4c40 0x7ffa0c1f7a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:31.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:31.312+0000 7ffa1419e640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffa0c15e5c0 0x7ffa0c15c080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:31.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:31.312+0000 7ffa1419e640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7ffa0c0b42c0 con 0x7ffa0c0d1130 2024-05-09T06:36:31.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:31.312+0000 7ffa1419e640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7ffa0c0b3fe0 con 0x7ffa0c15e5c0 2024-05-09T06:36:31.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:31.312+0000 7ffa1419e640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7ffa0c0cf270 con 0x7ffa0c0d4c40 2024-05-09T06:36:31.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:31.312+0000 7ffa11f3a640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffa0c0d1130 0x7ffa0c0cdfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:31.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:31.313+0000 7ffa11f3a640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffa0c0d1130 0x7ffa0c0cdfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:35396/0 (socket says 172.21.15.3:35396) 2024-05-09T06:36:31.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:31.313+0000 7ffa11f3a640 1 -- 172.21.15.3:0/4159747343 learned_addr learned my addr 172.21.15.3:0/4159747343 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:36:31.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:31.313+0000 7ffa1273b640 1 --2- 172.21.15.3:0/4159747343 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffa0c15e5c0 0x7ffa0c15c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:31.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:31.313+0000 7ffa11739640 1 --2- 172.21.15.3:0/4159747343 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffa0c0d4c40 0x7ffa0c1f7a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:31.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:31.313+0000 7ffa11f3a640 1 -- 172.21.15.3:0/4159747343 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffa0c0d4c40 msgr2=0x7ffa0c1f7a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:31.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:31.313+0000 7ffa11f3a640 1 --2- 172.21.15.3:0/4159747343 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffa0c0d4c40 0x7ffa0c1f7a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:31.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:31.313+0000 7ffa11f3a640 1 -- 172.21.15.3:0/4159747343 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffa0c15e5c0 msgr2=0x7ffa0c15c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:31.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:31.313+0000 7ffa11f3a640 1 --2- 172.21.15.3:0/4159747343 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffa0c15e5c0 0x7ffa0c15c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:31.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:31.313+0000 7ffa11f3a640 1 -- 172.21.15.3:0/4159747343 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa0c1f8220 con 0x7ffa0c0d1130 2024-05-09T06:36:31.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:31.313+0000 7ffa1273b640 1 --2- 172.21.15.3:0/4159747343 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffa0c15e5c0 0x7ffa0c15c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:36:31.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:31.313+0000 7ffa11f3a640 1 --2- 172.21.15.3:0/4159747343 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffa0c0d1130 0x7ffa0c0cdfd0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7ffa0400d5f0 tx=0x7ffa0400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:31.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:31.313+0000 7ffa0a7fc640 1 -- 172.21.15.3:0/4159747343 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa040125f0 con 0x7ffa0c0d1130 2024-05-09T06:36:31.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:31.313+0000 7ffa0a7fc640 1 -- 172.21.15.3:0/4159747343 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa04019070 con 0x7ffa0c0d1130 2024-05-09T06:36:31.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:31.313+0000 7ffa1419e640 1 -- 172.21.15.3:0/4159747343 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa0c1f8500 con 0x7ffa0c0d1130 2024-05-09T06:36:31.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:31.313+0000 7ffa0a7fc640 1 -- 172.21.15.3:0/4159747343 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa040129d0 con 0x7ffa0c0d1130 2024-05-09T06:36:31.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:31.313+0000 7ffa1419e640 1 -- 172.21.15.3:0/4159747343 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa0c15c5c0 con 0x7ffa0c0d1130 2024-05-09T06:36:31.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:31.314+0000 7ffa1419e640 1 -- 172.21.15.3:0/4159747343 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa0c069550 con 0x7ffa0c0d1130 2024-05-09T06:36:31.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:31.315+0000 7ffa0a7fc640 1 -- 172.21.15.3:0/4159747343 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ffa04012b70 con 0x7ffa0c0d1130 2024-05-09T06:36:31.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:31.315+0000 7ffa0a7fc640 1 --2- 172.21.15.3:0/4159747343 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7ff9f806be30 0x7ff9f806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:31.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:31.315+0000 7ffa0a7fc640 1 -- 172.21.15.3:0/4159747343 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7ffa0408b120 con 0x7ffa0c0d1130 2024-05-09T06:36:31.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:31.316+0000 7ffa11739640 1 --2- 172.21.15.3:0/4159747343 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7ff9f806be30 0x7ff9f806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:31.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:31.316+0000 7ffa11739640 1 --2- 172.21.15.3:0/4159747343 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7ff9f806be30 0x7ff9f806e2f0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7ffa00034000 tx=0x7ffa0002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:31.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:31.319+0000 7ffa0a7fc640 1 -- 172.21.15.3:0/4159747343 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7ffa04059b60 con 0x7ffa0c0d1130 2024-05-09T06:36:31.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:31.501+0000 7ffa1419e640 1 -- 172.21.15.3:0/4159747343 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7ffa0c08def0 con 0x7ffa0c0d1130 2024-05-09T06:36:31.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:31.501+0000 7ffa0a7fc640 1 -- 172.21.15.3:0/4159747343 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7ffa04059460 con 0x7ffa0c0d1130 2024-05-09T06:36:31.481 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 1 2024-05-09T06:36:31.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:31.503+0000 7ffa1419e640 1 -- 172.21.15.3:0/4159747343 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7ff9f806be30 msgr2=0x7ff9f806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:31.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:31.503+0000 7ffa1419e640 1 --2- 172.21.15.3:0/4159747343 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7ff9f806be30 0x7ff9f806e2f0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7ffa00034000 tx=0x7ffa0002e000 comp rx=0 tx=0).stop 2024-05-09T06:36:31.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:31.503+0000 7ffa1419e640 1 -- 172.21.15.3:0/4159747343 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffa0c0d1130 msgr2=0x7ffa0c0cdfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:31.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:31.503+0000 7ffa1419e640 1 --2- 172.21.15.3:0/4159747343 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffa0c0d1130 0x7ffa0c0cdfd0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7ffa0400d5f0 tx=0x7ffa0400dab0 comp rx=0 tx=0).stop 2024-05-09T06:36:31.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:31.503+0000 7ffa1419e640 1 -- 172.21.15.3:0/4159747343 shutdown_connections 2024-05-09T06:36:31.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:31.503+0000 7ffa1419e640 1 --2- 172.21.15.3:0/4159747343 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7ff9f806be30 0x7ff9f806e2f0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:31.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:31.503+0000 7ffa1419e640 1 --2- 172.21.15.3:0/4159747343 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffa0c15e5c0 0x7ffa0c15c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:31.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:31.503+0000 7ffa1419e640 1 --2- 172.21.15.3:0/4159747343 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffa0c0d4c40 0x7ffa0c1f7a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:31.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:31.503+0000 7ffa1419e640 1 --2- 172.21.15.3:0/4159747343 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffa0c0d1130 0x7ffa0c0cdfd0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:31.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:31.503+0000 7ffa1419e640 1 -- 172.21.15.3:0/4159747343 >> 172.21.15.3:0/4159747343 conn(0x7ffa0c09d500 msgr2=0x7ffa0c0b2540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:31.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:31.503+0000 7ffa1419e640 1 -- 172.21.15.3:0/4159747343 shutdown_connections 2024-05-09T06:36:31.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:31.503+0000 7ffa1419e640 1 -- 172.21.15.3:0/4159747343 wait complete. 2024-05-09T06:36:31.494 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-09T06:36:31.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:31.621+0000 7f672ae4d640 1 Processor -- start 2024-05-09T06:36:31.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:31.621+0000 7f672ae4d640 1 -- start start 2024-05-09T06:36:31.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:31.622+0000 7f672ae4d640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f67240d2800 0x7f67240d4c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:31.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:31.622+0000 7f672ae4d640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f672415e9a0 con 0x7f67240d1300 2024-05-09T06:36:31.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:31.622+0000 7f672ae4d640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f67240d31c0 con 0x7f672415e5c0 2024-05-09T06:36:31.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:31.622+0000 7f672ae4d640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f67240d2c90 con 0x7f67240d2800 2024-05-09T06:36:31.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:31.622+0000 7f6723fff640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f67240d1300 0x7f67240b47e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:58902/0 (socket says 172.21.15.3:58902) 2024-05-09T06:36:31.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:31.622+0000 7f6723fff640 1 -- 172.21.15.3:0/3553236889 learned_addr learned my addr 172.21.15.3:0/3553236889 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-05-09T06:36:31.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:31.623+0000 7f67293ea640 1 --2- 172.21.15.3:0/3553236889 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f67240d2800 0x7f67240d4c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:31.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:31.623+0000 7f67237fe640 1 -- 172.21.15.3:0/3553236889 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2209998341 0 0) 0x7f672415e9a0 con 0x7f67240d1300 2024-05-09T06:36:31.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:31.623+0000 7f67237fe640 1 -- 172.21.15.3:0/3553236889 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f670c003520 con 0x7f67240d1300 2024-05-09T06:36:31.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:31.623+0000 7f67237fe640 1 -- 172.21.15.3:0/3553236889 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1640062833 0 0) 0x7f67240d31c0 con 0x7f672415e5c0 2024-05-09T06:36:31.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:31.623+0000 7f67237fe640 1 -- 172.21.15.3:0/3553236889 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f672415e9a0 con 0x7f672415e5c0 2024-05-09T06:36:31.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:31.623+0000 7f67293ea640 1 -- 172.21.15.3:0/3553236889 >> v1:172.21.15.190:6789/0 conn(0x7f672415e5c0 legacy=0x7f67240d0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:31.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:31.623+0000 7f67293ea640 1 -- 172.21.15.3:0/3553236889 >> v1:172.21.15.3:6789/0 conn(0x7f67240d1300 legacy=0x7f67240b47e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:31.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:31.623+0000 7f67237fe640 1 -- 172.21.15.3:0/3553236889 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2469202855 0 0) 0x7f670c003520 con 0x7f67240d1300 2024-05-09T06:36:31.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:31.623+0000 7f67293ea640 1 -- 172.21.15.3:0/3553236889 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67240d2ee0 con 0x7f67240d2800 2024-05-09T06:36:31.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:31.624+0000 7f67293ea640 1 --2- 172.21.15.3:0/3553236889 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f67240d2800 0x7f67240d4c40 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f67140097d0 tx=0x7f671402a1a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=52a1c4b6aa6d9b9c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:31.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:31.624+0000 7f67237fe640 1 -- 172.21.15.3:0/3553236889 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6714037070 con 0x7f67240d2800 2024-05-09T06:36:31.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:31.624+0000 7f67237fe640 1 -- 172.21.15.3:0/3553236889 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f671402ac30 con 0x7f67240d2800 2024-05-09T06:36:31.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:31.624+0000 7f67237fe640 1 -- 172.21.15.3:0/3553236889 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6718001670 con 0x7f67240d2800 2024-05-09T06:36:31.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:31.625+0000 7f672ae4d640 1 -- 172.21.15.3:0/3553236889 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f67240d2800 msgr2=0x7f67240d4c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:31.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:31.625+0000 7f672ae4d640 1 --2- 172.21.15.3:0/3553236889 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f67240d2800 0x7f67240d4c40 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f67140097d0 tx=0x7f671402a1a0 comp rx=0 tx=0).stop 2024-05-09T06:36:31.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:31.625+0000 7f672ae4d640 1 -- 172.21.15.3:0/3553236889 shutdown_connections 2024-05-09T06:36:31.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:31.625+0000 7f672ae4d640 1 --2- 172.21.15.3:0/3553236889 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f67240d2800 0x7f67240d4c40 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:31.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:31.625+0000 7f672ae4d640 1 -- 172.21.15.3:0/3553236889 >> 172.21.15.3:0/3553236889 conn(0x7f672409d500 msgr2=0x7f67240d4220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:31.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:31.625+0000 7f672ae4d640 1 -- 172.21.15.3:0/3553236889 shutdown_connections 2024-05-09T06:36:31.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:31.625+0000 7f672ae4d640 1 -- 172.21.15.3:0/3553236889 wait complete. 2024-05-09T06:36:31.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:31.626+0000 7f672ae4d640 1 Processor -- start 2024-05-09T06:36:31.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:31.626+0000 7f672ae4d640 1 -- start start 2024-05-09T06:36:31.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:31.627+0000 7f672ae4d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f67240d1300 0x7f6724155740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:31.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:31.627+0000 7f672ae4d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f67240d2800 0x7f67240cdeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:31.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:31.627+0000 7f672ae4d640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f672415e5c0 0x7f67240b83d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:31.609 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:31.627+0000 7f672ae4d640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f67240b42c0 con 0x7f67240d2800 2024-05-09T06:36:31.609 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:31.627+0000 7f672ae4d640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f67240b3fe0 con 0x7f67240d1300 2024-05-09T06:36:31.609 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:31.627+0000 7f672ae4d640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f67240b4ef0 con 0x7f672415e5c0 2024-05-09T06:36:31.609 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:31.627+0000 7f6723fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f67240d2800 0x7f67240cdeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:31.609 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:31.627+0000 7f6723fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f67240d2800 0x7f67240cdeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:35398/0 (socket says 172.21.15.3:35398) 2024-05-09T06:36:31.609 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:31.627+0000 7f6723fff640 1 -- 172.21.15.3:0/554990011 learned_addr learned my addr 172.21.15.3:0/554990011 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:36:31.609 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:31.627+0000 7f6723fff640 1 -- 172.21.15.3:0/554990011 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f672415e5c0 msgr2=0x7f67240b83d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:31.609 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:31.627+0000 7f6723fff640 1 --2- 172.21.15.3:0/554990011 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f672415e5c0 0x7f67240b83d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:31.609 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:31.627+0000 7f6723fff640 1 -- 172.21.15.3:0/554990011 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f67240d1300 msgr2=0x7f6724155740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:31.609 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:31.627+0000 7f6723fff640 1 --2- 172.21.15.3:0/554990011 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f67240d1300 0x7f6724155740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:31.609 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:31.627+0000 7f6723fff640 1 -- 172.21.15.3:0/554990011 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67240b7330 con 0x7f67240d2800 2024-05-09T06:36:31.609 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:31.627+0000 7f6723fff640 1 --2- 172.21.15.3:0/554990011 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f67240d2800 0x7f67240cdeb0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f671800c5f0 tx=0x7f671800cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:31.609 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:31.628+0000 7f67217fa640 1 -- 172.21.15.3:0/554990011 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6718017070 con 0x7f67240d2800 2024-05-09T06:36:31.609 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:31.628+0000 7f672ae4d640 1 -- 172.21.15.3:0/554990011 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67240b7610 con 0x7f67240d2800 2024-05-09T06:36:31.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:31.628+0000 7f67217fa640 1 -- 172.21.15.3:0/554990011 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6718005d00 con 0x7f67240d2800 2024-05-09T06:36:31.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:31.628+0000 7f67217fa640 1 -- 172.21.15.3:0/554990011 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67180072b0 con 0x7f67240d2800 2024-05-09T06:36:31.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:31.628+0000 7f672ae4d640 1 -- 172.21.15.3:0/554990011 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67240b7bd0 con 0x7f67240d2800 2024-05-09T06:36:31.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:31.629+0000 7f672ae4d640 1 -- 172.21.15.3:0/554990011 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6724069550 con 0x7f67240d2800 2024-05-09T06:36:31.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:31.629+0000 7f67217fa640 1 -- 172.21.15.3:0/554990011 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f671801a070 con 0x7f67240d2800 2024-05-09T06:36:31.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:31.630+0000 7f67217fa640 1 --2- 172.21.15.3:0/554990011 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f671006be30 0x7f671006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:31.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:31.630+0000 7f67217fa640 1 -- 172.21.15.3:0/554990011 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f671808a4e0 con 0x7f67240d2800 2024-05-09T06:36:31.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:31.630+0000 7f6728be9640 1 --2- 172.21.15.3:0/554990011 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f671006be30 0x7f671006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:31.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:31.630+0000 7f6728be9640 1 --2- 172.21.15.3:0/554990011 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f671006be30 0x7f671006e2f0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f671c001460 tx=0x7f671c007cd0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:31.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:31.634+0000 7f67217fa640 1 -- 172.21.15.3:0/554990011 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f6718058d80 con 0x7f67240d2800 2024-05-09T06:36:31.798 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:31.818+0000 7f672ae4d640 1 -- 172.21.15.3:0/554990011 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f67240b2b50 con 0x7f67240d2800 2024-05-09T06:36:31.799 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:31.819+0000 7f67217fa640 1 -- 172.21.15.3:0/554990011 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7f6718058680 con 0x7f67240d2800 2024-05-09T06:36:31.799 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 32 2024-05-09T06:36:31.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:31.821+0000 7f672ae4d640 1 -- 172.21.15.3:0/554990011 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f671006be30 msgr2=0x7f671006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:31.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:31.821+0000 7f672ae4d640 1 --2- 172.21.15.3:0/554990011 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f671006be30 0x7f671006e2f0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f671c001460 tx=0x7f671c007cd0 comp rx=0 tx=0).stop 2024-05-09T06:36:31.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:31.821+0000 7f672ae4d640 1 -- 172.21.15.3:0/554990011 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f67240d2800 msgr2=0x7f67240cdeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:31.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:31.821+0000 7f672ae4d640 1 --2- 172.21.15.3:0/554990011 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f67240d2800 0x7f67240cdeb0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f671800c5f0 tx=0x7f671800cab0 comp rx=0 tx=0).stop 2024-05-09T06:36:31.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:31.821+0000 7f672ae4d640 1 -- 172.21.15.3:0/554990011 shutdown_connections 2024-05-09T06:36:31.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:31.821+0000 7f672ae4d640 1 --2- 172.21.15.3:0/554990011 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f671006be30 0x7f671006e2f0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:31.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:31.821+0000 7f672ae4d640 1 --2- 172.21.15.3:0/554990011 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f672415e5c0 0x7f67240b83d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:31.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:31.821+0000 7f672ae4d640 1 --2- 172.21.15.3:0/554990011 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f67240d2800 0x7f67240cdeb0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:31.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:31.821+0000 7f672ae4d640 1 --2- 172.21.15.3:0/554990011 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f67240d1300 0x7f6724155740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:31.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:31.821+0000 7f672ae4d640 1 -- 172.21.15.3:0/554990011 >> 172.21.15.3:0/554990011 conn(0x7f672409d500 msgr2=0x7f67240b47e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:31.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:31.821+0000 7f672ae4d640 1 -- 172.21.15.3:0/554990011 shutdown_connections 2024-05-09T06:36:31.802 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:31.821+0000 7f672ae4d640 1 -- 172.21.15.3:0/554990011 wait complete. 2024-05-09T06:36:31.812 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-09T06:36:31.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:31.942+0000 7ffb37015640 1 Processor -- start 2024-05-09T06:36:31.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:31.942+0000 7ffb37015640 1 -- start start 2024-05-09T06:36:31.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:31.942+0000 7ffb37015640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7ffb3015e5c0 0x7ffb300d2800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:31.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:31.942+0000 7ffb37015640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ffb300d1300 0x7ffb300b47e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:31.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:31.942+0000 7ffb37015640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffb3015e9a0 con 0x7ffb300d4c40 2024-05-09T06:36:31.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:31.942+0000 7ffb37015640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7ffb300d50d0 con 0x7ffb300d1300 2024-05-09T06:36:31.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:31.942+0000 7ffb37015640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7ffb300d31c0 con 0x7ffb3015e5c0 2024-05-09T06:36:31.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:31.943+0000 7ffb355b2640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ffb300d1300 0x7ffb300b47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:31.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:31.943+0000 7ffb2ffff640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7ffb300d4c40 0x7ffb300d0bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:58914/0 (socket says 172.21.15.3:58914) 2024-05-09T06:36:31.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:31.943+0000 7ffb2ffff640 1 -- 172.21.15.3:0/2177372934 learned_addr learned my addr 172.21.15.3:0/2177372934 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-05-09T06:36:31.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:31.943+0000 7ffb34db1640 1 --2- 172.21.15.3:0/2177372934 >> v2:172.21.15.190:3300/0 conn(0x7ffb3015e5c0 0x7ffb300d2800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:31.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:31.943+0000 7ffb355b2640 1 -- 172.21.15.3:0/2177372934 >> v2:172.21.15.190:3300/0 conn(0x7ffb3015e5c0 msgr2=0x7ffb300d2800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:31.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:31.943+0000 7ffb355b2640 1 --2- 172.21.15.3:0/2177372934 >> v2:172.21.15.190:3300/0 conn(0x7ffb3015e5c0 0x7ffb300d2800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:31.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:31.943+0000 7ffb355b2640 1 -- 172.21.15.3:0/2177372934 >> v1:172.21.15.3:6789/0 conn(0x7ffb300d4c40 legacy=0x7ffb300d0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:31.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:31.943+0000 7ffb355b2640 1 -- 172.21.15.3:0/2177372934 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb300d2ee0 con 0x7ffb300d1300 2024-05-09T06:36:31.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:31.943+0000 7ffb2f7fe640 1 -- 172.21.15.3:0/2177372934 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1115180054 0 0) 0x7ffb3015e9a0 con 0x7ffb300d4c40 2024-05-09T06:36:31.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:31.944+0000 7ffb355b2640 1 --2- 172.21.15.3:0/2177372934 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffb300d1300 0x7ffb300b47e0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7ffb28009d90 tx=0x7ffb2802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8c63ac3b961c152 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:31.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:31.944+0000 7ffb2f7fe640 1 -- 172.21.15.3:0/2177372934 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb28007740 con 0x7ffb300d1300 2024-05-09T06:36:31.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:31.944+0000 7ffb2f7fe640 1 -- 172.21.15.3:0/2177372934 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb280078e0 con 0x7ffb300d1300 2024-05-09T06:36:31.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:31.944+0000 7ffb2f7fe640 1 -- 172.21.15.3:0/2177372934 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb28007cb0 con 0x7ffb300d1300 2024-05-09T06:36:31.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:31.944+0000 7ffb37015640 1 -- 172.21.15.3:0/2177372934 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffb300d1300 msgr2=0x7ffb300b47e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:31.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:31.944+0000 7ffb37015640 1 --2- 172.21.15.3:0/2177372934 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffb300d1300 0x7ffb300b47e0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7ffb28009d90 tx=0x7ffb2802a6a0 comp rx=0 tx=0).stop 2024-05-09T06:36:31.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:31.944+0000 7ffb37015640 1 -- 172.21.15.3:0/2177372934 shutdown_connections 2024-05-09T06:36:31.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:31.944+0000 7ffb37015640 1 --2- 172.21.15.3:0/2177372934 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffb300d1300 0x7ffb300b47e0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:31.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:31.944+0000 7ffb37015640 1 --2- 172.21.15.3:0/2177372934 >> v2:172.21.15.190:3300/0 conn(0x7ffb3015e5c0 0x7ffb300d2800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:31.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:31.944+0000 7ffb37015640 1 -- 172.21.15.3:0/2177372934 >> 172.21.15.3:0/2177372934 conn(0x7ffb3009d500 msgr2=0x7ffb300d4220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:31.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:31.945+0000 7ffb37015640 1 -- 172.21.15.3:0/2177372934 shutdown_connections 2024-05-09T06:36:31.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:31.945+0000 7ffb37015640 1 -- 172.21.15.3:0/2177372934 wait complete. 2024-05-09T06:36:31.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:31.946+0000 7ffb37015640 1 Processor -- start 2024-05-09T06:36:31.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:31.946+0000 7ffb37015640 1 -- start start 2024-05-09T06:36:31.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:31.947+0000 7ffb37015640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffb300d1300 0x7ffb300caa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:31.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:31.947+0000 7ffb37015640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffb300d4c40 0x7ffb300b7b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:31.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:31.947+0000 7ffb37015640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffb3015e5c0 0x7ffb300cddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:31.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:31.947+0000 7ffb37015640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7ffb300b42c0 con 0x7ffb300d4c40 2024-05-09T06:36:31.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:31.947+0000 7ffb37015640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7ffb300b3fe0 con 0x7ffb300d1300 2024-05-09T06:36:31.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:31.947+0000 7ffb37015640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7ffb300b4d20 con 0x7ffb3015e5c0 2024-05-09T06:36:31.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:31.947+0000 7ffb2ffff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffb300d4c40 0x7ffb300b7b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:31.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:31.947+0000 7ffb2ffff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffb300d4c40 0x7ffb300b7b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:35420/0 (socket says 172.21.15.3:35420) 2024-05-09T06:36:31.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:31.947+0000 7ffb2ffff640 1 -- 172.21.15.3:0/4294398183 learned_addr learned my addr 172.21.15.3:0/4294398183 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:36:31.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:31.947+0000 7ffb34db1640 1 --2- 172.21.15.3:0/4294398183 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffb300d1300 0x7ffb300caa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:31.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:31.947+0000 7ffb355b2640 1 --2- 172.21.15.3:0/4294398183 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffb3015e5c0 0x7ffb300cddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:31.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:31.947+0000 7ffb2ffff640 1 -- 172.21.15.3:0/4294398183 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffb3015e5c0 msgr2=0x7ffb300cddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:31.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:31.947+0000 7ffb2ffff640 1 --2- 172.21.15.3:0/4294398183 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffb3015e5c0 0x7ffb300cddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:31.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:31.947+0000 7ffb2ffff640 1 -- 172.21.15.3:0/4294398183 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffb300d1300 msgr2=0x7ffb300caa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:31.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:31.947+0000 7ffb2ffff640 1 --2- 172.21.15.3:0/4294398183 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffb300d1300 0x7ffb300caa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:31.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:31.947+0000 7ffb2ffff640 1 -- 172.21.15.3:0/4294398183 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb300ced90 con 0x7ffb300d4c40 2024-05-09T06:36:31.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:31.947+0000 7ffb355b2640 1 --2- 172.21.15.3:0/4294398183 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffb3015e5c0 0x7ffb300cddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:36:31.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:31.947+0000 7ffb2ffff640 1 --2- 172.21.15.3:0/4294398183 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffb300d4c40 0x7ffb300b7b70 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7ffb1c00c4d0 tx=0x7ffb1c00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:31.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:31.947+0000 7ffb2d7fa640 1 -- 172.21.15.3:0/4294398183 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb1c018070 con 0x7ffb300d4c40 2024-05-09T06:36:31.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:31.947+0000 7ffb2d7fa640 1 -- 172.21.15.3:0/4294398183 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb1c007500 con 0x7ffb300d4c40 2024-05-09T06:36:31.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:31.947+0000 7ffb2d7fa640 1 -- 172.21.15.3:0/4294398183 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb1c004a20 con 0x7ffb300d4c40 2024-05-09T06:36:31.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:31.947+0000 7ffb37015640 1 -- 172.21.15.3:0/4294398183 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb300cf030 con 0x7ffb300d4c40 2024-05-09T06:36:31.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:31.948+0000 7ffb37015640 1 -- 172.21.15.3:0/4294398183 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb300cf5f0 con 0x7ffb300d4c40 2024-05-09T06:36:31.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:31.949+0000 7ffb37015640 1 -- 172.21.15.3:0/4294398183 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb30069550 con 0x7ffb300d4c40 2024-05-09T06:36:31.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:31.949+0000 7ffb2d7fa640 1 -- 172.21.15.3:0/4294398183 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ffb1c01b070 con 0x7ffb300d4c40 2024-05-09T06:36:31.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:31.950+0000 7ffb2d7fa640 1 --2- 172.21.15.3:0/4294398183 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7ffb1406be30 0x7ffb1406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:31.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:31.950+0000 7ffb2d7fa640 1 -- 172.21.15.3:0/4294398183 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7ffb1c08a100 con 0x7ffb300d4c40 2024-05-09T06:36:31.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:31.950+0000 7ffb34db1640 1 --2- 172.21.15.3:0/4294398183 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7ffb1406be30 0x7ffb1406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:31.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:31.950+0000 7ffb34db1640 1 --2- 172.21.15.3:0/4294398183 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7ffb1406be30 0x7ffb1406e2f0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7ffb240079a0 tx=0x7ffb24008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:31.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:31.954+0000 7ffb2d7fa640 1 -- 172.21.15.3:0/4294398183 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7ffb1c058460 con 0x7ffb300d4c40 2024-05-09T06:36:32.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.136+0000 7ffb37015640 1 -- 172.21.15.3:0/4294398183 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7ffb300cf7c0 con 0x7ffb300d4c40 2024-05-09T06:36:32.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.136+0000 7ffb2d7fa640 1 -- 172.21.15.3:0/4294398183 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7ffb1c05c760 con 0x7ffb300d4c40 2024-05-09T06:36:32.116 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 64 2024-05-09T06:36:32.118 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.138+0000 7ffb37015640 1 -- 172.21.15.3:0/4294398183 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7ffb1406be30 msgr2=0x7ffb1406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:32.118 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.138+0000 7ffb37015640 1 --2- 172.21.15.3:0/4294398183 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7ffb1406be30 0x7ffb1406e2f0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7ffb240079a0 tx=0x7ffb24008040 comp rx=0 tx=0).stop 2024-05-09T06:36:32.119 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.138+0000 7ffb37015640 1 -- 172.21.15.3:0/4294398183 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffb300d4c40 msgr2=0x7ffb300b7b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:32.119 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.138+0000 7ffb37015640 1 --2- 172.21.15.3:0/4294398183 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffb300d4c40 0x7ffb300b7b70 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7ffb1c00c4d0 tx=0x7ffb1c00c990 comp rx=0 tx=0).stop 2024-05-09T06:36:32.119 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.138+0000 7ffb37015640 1 -- 172.21.15.3:0/4294398183 shutdown_connections 2024-05-09T06:36:32.119 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.138+0000 7ffb37015640 1 --2- 172.21.15.3:0/4294398183 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7ffb1406be30 0x7ffb1406e2f0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:32.119 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.138+0000 7ffb37015640 1 --2- 172.21.15.3:0/4294398183 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffb3015e5c0 0x7ffb300cddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:32.119 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.138+0000 7ffb37015640 1 --2- 172.21.15.3:0/4294398183 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffb300d4c40 0x7ffb300b7b70 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:32.119 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.138+0000 7ffb37015640 1 --2- 172.21.15.3:0/4294398183 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffb300d1300 0x7ffb300caa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:32.119 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.138+0000 7ffb37015640 1 -- 172.21.15.3:0/4294398183 >> 172.21.15.3:0/4294398183 conn(0x7ffb3009d500 msgr2=0x7ffb30160ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:32.119 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.138+0000 7ffb37015640 1 -- 172.21.15.3:0/4294398183 shutdown_connections 2024-05-09T06:36:32.119 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.138+0000 7ffb37015640 1 -- 172.21.15.3:0/4294398183 wait complete. 2024-05-09T06:36:32.130 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T06:36:32.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.253+0000 7f7eafb70640 1 Processor -- start 2024-05-09T06:36:32.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.254+0000 7f7eafb70640 1 -- start start 2024-05-09T06:36:32.237 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.254+0000 7f7eafb70640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7ea80d2c20 0x7f7ea8169ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:32.237 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.254+0000 7f7eafb70640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7ea80d4e50 con 0x7f7ea80d27c0 2024-05-09T06:36:32.237 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.254+0000 7f7eafb70640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7ea80d0270 con 0x7f7ea815e480 2024-05-09T06:36:32.237 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.254+0000 7f7eafb70640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f7ea80d30d0 con 0x7f7ea80d2c20 2024-05-09T06:36:32.237 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.255+0000 7f7ead10b640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f7ea80d27c0 0x7f7ea8166390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:58926/0 (socket says 172.21.15.3:58926) 2024-05-09T06:36:32.237 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.255+0000 7f7ead10b640 1 -- 172.21.15.3:0/1150894506 learned_addr learned my addr 172.21.15.3:0/1150894506 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-05-09T06:36:32.237 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.255+0000 7f7eae10d640 1 --2- 172.21.15.3:0/1150894506 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7ea80d2c20 0x7f7ea8169ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:32.237 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.255+0000 7f7eac90a640 1 -- 172.21.15.3:0/1150894506 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3678607946 0 0) 0x7f7e9c001110 con 0x7f7ea80d27c0 2024-05-09T06:36:32.237 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.255+0000 7f7eac90a640 1 -- 172.21.15.3:0/1150894506 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7e90003520 con 0x7f7ea80d27c0 2024-05-09T06:36:32.237 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.255+0000 7f7eac90a640 1 -- 172.21.15.3:0/1150894506 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 924368160 0 0) 0x7f7ea80d0270 con 0x7f7ea815e480 2024-05-09T06:36:32.237 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.255+0000 7f7eac90a640 1 -- 172.21.15.3:0/1150894506 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7e9c001110 con 0x7f7ea815e480 2024-05-09T06:36:32.237 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.255+0000 7f7eac90a640 1 -- 172.21.15.3:0/1150894506 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2525199912 0 0) 0x7f7e90003520 con 0x7f7ea80d27c0 2024-05-09T06:36:32.237 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.255+0000 7f7eac90a640 1 -- 172.21.15.3:0/1150894506 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7ea80d2c20 msgr2=0x7f7ea8169ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:32.238 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.255+0000 7f7eac90a640 1 --2- 172.21.15.3:0/1150894506 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7ea80d2c20 0x7f7ea8169ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:32.238 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.255+0000 7f7eac90a640 1 -- 172.21.15.3:0/1150894506 >> v1:172.21.15.190:6789/0 conn(0x7f7ea815e480 legacy=0x7f7ea80d1dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:32.238 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.255+0000 7f7eac90a640 1 -- 172.21.15.3:0/1150894506 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ea80cf1d0 con 0x7f7ea80d27c0 2024-05-09T06:36:32.238 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.255+0000 7f7eac90a640 1 -- 172.21.15.3:0/1150894506 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 563312031 0 0) 0x7f7e9c001840 con 0x7f7ea80d27c0 2024-05-09T06:36:32.238 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.255+0000 7f7eac90a640 1 -- 172.21.15.3:0/1150894506 >> v1:172.21.15.3:6789/0 conn(0x7f7ea80d27c0 legacy=0x7f7ea8166390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:32.238 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.255+0000 7f7eac90a640 1 --2- 172.21.15.3:0/1150894506 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7e90003b70 0x7f7e90003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:32.238 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.255+0000 7f7eac90a640 1 --2- 172.21.15.3:0/1150894506 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7ea80d2c20 0x7f7e90004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:32.238 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.255+0000 7f7eac90a640 1 --2- 172.21.15.3:0/1150894506 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7e90004d40 0x7f7e900051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:32.238 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.255+0000 7f7eac90a640 1 -- 172.21.15.3:0/1150894506 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f7ea80d30d0 con 0x7f7e90003b70 2024-05-09T06:36:32.238 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.255+0000 7f7eac90a640 1 -- 172.21.15.3:0/1150894506 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f7e9000ba70 con 0x7f7e90004d40 2024-05-09T06:36:32.239 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.255+0000 7f7eac90a640 1 -- 172.21.15.3:0/1150894506 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f7e9000bc70 con 0x7f7ea80d2c20 2024-05-09T06:36:32.239 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.255+0000 7f7ead90c640 1 --2- 172.21.15.3:0/1150894506 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7e90003b70 0x7f7e90003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:32.239 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.255+0000 7f7eae10d640 1 --2- 172.21.15.3:0/1150894506 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7e90004d40 0x7f7e900051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:32.239 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.255+0000 7f7ead10b640 1 --2- 172.21.15.3:0/1150894506 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7ea80d2c20 0x7f7e90004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:32.239 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.256+0000 7f7ead90c640 1 -- 172.21.15.3:0/1150894506 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7ea80d2c20 msgr2=0x7f7e90004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:32.239 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.256+0000 7f7ead90c640 1 --2- 172.21.15.3:0/1150894506 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7ea80d2c20 0x7f7e90004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:32.239 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.256+0000 7f7ead90c640 1 -- 172.21.15.3:0/1150894506 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7e90004d40 msgr2=0x7f7e900051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:32.239 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.256+0000 7f7ead90c640 1 --2- 172.21.15.3:0/1150894506 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7e90004d40 0x7f7e900051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:32.239 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.256+0000 7f7ead90c640 1 -- 172.21.15.3:0/1150894506 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e90003520 con 0x7f7e90003b70 2024-05-09T06:36:32.239 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.256+0000 7f7ead10b640 1 --2- 172.21.15.3:0/1150894506 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7ea80d2c20 0x7f7e90004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:36:32.239 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.256+0000 7f7ead90c640 1 --2- 172.21.15.3:0/1150894506 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7e90003b70 0x7f7e90003f70 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f7ea00018d0 tx=0x7f7e90005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9bafa443e1fdef2 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:32.239 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.256+0000 7f7eac90a640 1 -- 172.21.15.3:0/1150894506 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ea00014f0 con 0x7f7e90003b70 2024-05-09T06:36:32.239 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.256+0000 7f7eac90a640 1 -- 172.21.15.3:0/1150894506 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ea00052d0 con 0x7f7e90003b70 2024-05-09T06:36:32.239 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.256+0000 7f7eac90a640 1 -- 172.21.15.3:0/1150894506 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ea002aca0 con 0x7f7e90003b70 2024-05-09T06:36:32.239 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.257+0000 7f7eafb70640 1 -- 172.21.15.3:0/1150894506 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7e90003b70 msgr2=0x7f7e90003f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:32.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.257+0000 7f7eafb70640 1 --2- 172.21.15.3:0/1150894506 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7e90003b70 0x7f7e90003f70 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f7ea00018d0 tx=0x7f7e90005830 comp rx=0 tx=0).stop 2024-05-09T06:36:32.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.257+0000 7f7eae10d640 1 -- 172.21.15.3:0/1150894506 reap_dead start 2024-05-09T06:36:32.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.257+0000 7f7eafb70640 1 -- 172.21.15.3:0/1150894506 shutdown_connections 2024-05-09T06:36:32.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.257+0000 7f7eafb70640 1 -- 172.21.15.3:0/1150894506 >> 172.21.15.3:0/1150894506 conn(0x7f7ea809d4d0 msgr2=0x7f7ea80d4200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:32.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.257+0000 7f7eafb70640 1 -- 172.21.15.3:0/1150894506 shutdown_connections 2024-05-09T06:36:32.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.257+0000 7f7eafb70640 1 -- 172.21.15.3:0/1150894506 wait complete. 2024-05-09T06:36:32.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.258+0000 7f7eafb70640 1 Processor -- start 2024-05-09T06:36:32.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.258+0000 7f7eafb70640 1 -- start start 2024-05-09T06:36:32.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.259+0000 7f7eafb70640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7e90003b70 0x7f7ea81f82e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:32.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.259+0000 7f7eafb70640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7ea81f3ba0 0x7f7ea80b81e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:32.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.259+0000 7f7eafb70640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7ea81f4020 0x7f7ea80b8720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:32.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.259+0000 7f7eafb70640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f7ea80cfa70 con 0x7f7ea81f4020 2024-05-09T06:36:32.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.259+0000 7f7eafb70640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f7ea80cf790 con 0x7f7ea81f3ba0 2024-05-09T06:36:32.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.259+0000 7f7eafb70640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f7ea80cfd50 con 0x7f7e90003b70 2024-05-09T06:36:32.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.259+0000 7f7eae10d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7ea81f4020 0x7f7ea80b8720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:32.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.259+0000 7f7eae10d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7ea81f4020 0x7f7ea80b8720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:35440/0 (socket says 172.21.15.3:35440) 2024-05-09T06:36:32.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.259+0000 7f7eae10d640 1 -- 172.21.15.3:0/3092696624 learned_addr learned my addr 172.21.15.3:0/3092696624 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:36:32.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.259+0000 7f7ead10b640 1 --2- 172.21.15.3:0/3092696624 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7ea81f3ba0 0x7f7ea80b81e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:32.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.259+0000 7f7ead90c640 1 --2- 172.21.15.3:0/3092696624 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7e90003b70 0x7f7ea81f82e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:32.244 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.259+0000 7f7eae10d640 1 -- 172.21.15.3:0/3092696624 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7e90003b70 msgr2=0x7f7ea81f82e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:32.244 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.259+0000 7f7eae10d640 1 --2- 172.21.15.3:0/3092696624 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7e90003b70 0x7f7ea81f82e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:32.244 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.259+0000 7f7eae10d640 1 -- 172.21.15.3:0/3092696624 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7ea81f3ba0 msgr2=0x7f7ea80b81e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:32.244 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.259+0000 7f7eae10d640 1 --2- 172.21.15.3:0/3092696624 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7ea81f3ba0 0x7f7ea80b81e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:32.244 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.259+0000 7f7eae10d640 1 -- 172.21.15.3:0/3092696624 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ea81566e0 con 0x7f7ea81f4020 2024-05-09T06:36:32.244 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.260+0000 7f7eae10d640 1 --2- 172.21.15.3:0/3092696624 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7ea81f4020 0x7f7ea80b8720 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f7ea80d41d0 tx=0x7f7e9800f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:32.244 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.260+0000 7f7ea67fc640 1 -- 172.21.15.3:0/3092696624 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e9800fde0 con 0x7f7ea81f4020 2024-05-09T06:36:32.244 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.260+0000 7f7ea67fc640 1 -- 172.21.15.3:0/3092696624 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e98015430 con 0x7f7ea81f4020 2024-05-09T06:36:32.244 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.260+0000 7f7ea67fc640 1 -- 172.21.15.3:0/3092696624 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e98015820 con 0x7f7ea81f4020 2024-05-09T06:36:32.244 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.260+0000 7f7eafb70640 1 -- 172.21.15.3:0/3092696624 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ea81f8820 con 0x7f7ea81f4020 2024-05-09T06:36:32.244 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.260+0000 7f7eafb70640 1 -- 172.21.15.3:0/3092696624 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ea8156c60 con 0x7f7ea81f4020 2024-05-09T06:36:32.244 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.261+0000 7f7eafb70640 1 -- 172.21.15.3:0/3092696624 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ea804f660 con 0x7f7ea81f4020 2024-05-09T06:36:32.244 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.261+0000 7f7ea67fc640 1 -- 172.21.15.3:0/3092696624 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7e980159c0 con 0x7f7ea81f4020 2024-05-09T06:36:32.244 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.262+0000 7f7ea67fc640 1 --2- 172.21.15.3:0/3092696624 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f7e9406be30 0x7f7e9406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:32.244 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.262+0000 7f7ea67fc640 1 -- 172.21.15.3:0/3092696624 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f7e9808bad0 con 0x7f7ea81f4020 2024-05-09T06:36:32.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.262+0000 7f7ead90c640 1 --2- 172.21.15.3:0/3092696624 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f7e9406be30 0x7f7e9406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:32.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.263+0000 7f7ead90c640 1 --2- 172.21.15.3:0/3092696624 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f7e9406be30 0x7f7e9406e2f0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f7ea0032000 tx=0x7f7ea0009030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:32.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.266+0000 7f7ea67fc640 1 -- 172.21.15.3:0/3092696624 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f7e98019030 con 0x7f7ea81f4020 2024-05-09T06:36:32.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.451+0000 7f7eafb70640 1 -- 172.21.15.3:0/3092696624 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7ea80cabf0 con 0x7f7ea81f4020 2024-05-09T06:36:32.432 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.452+0000 7f7ea67fc640 1 -- 172.21.15.3:0/3092696624 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16521 (secure 0 0 0) 0x7f7e98010040 con 0x7f7ea81f4020 2024-05-09T06:36:32.432 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-09T06:36:32.432 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":20,"fsid":"16ed737d-2665-431a-8ac6-ef242be3975a","created":"2024-05-09T06:35:42.541368+0000","modified":"2024-05-09T06:36:08.856391+0000","last_up_change":"2024-05-09T06:35:53.778274+0000","last_in_change":"2024-05-09T06:35:44.780586+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T06:35:54.819999+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T06:36:04.931314+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T06:36:06.200433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fae3a8db-4826-4f3c-b33a-12c09bb5df6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":1897849827},{"type":"v1","addr":"172.21.15.3:6817","nonce":1897849827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":1897849827},{"type":"v1","addr":"172.21.15.3:6819","nonce":1897849827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":1897849827},{"type":"v1","addr":"172.21.15.3:6823","nonce":1897849827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":1897849827},{"type":"v1","addr":"172.21.15.3:6821","nonce":1897849827}]},"public_addr":"172.21.15.3:6817/1897849827","cluster_addr":"172.21.15.3:6819/1897849827","heartbeat_back_addr":"172.21.15.3:6823/1897849827","heartbeat_front_addr":"172.21.15.3:6821/1897849827","state":["exists","up"]},{"osd":1,"uuid":"646e882d-4ce4-4720-8c84-f71df563a35e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":926805498},{"type":"v1","addr":"172.21.15.3:6801","nonce":926805498}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":926805498},{"type":"v1","addr":"172.21.15.3:6803","nonce":926805498}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":926805498},{"type":"v1","addr":"172.21.15.3:6807","nonce":926805498}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":926805498},{"type":"v1","addr":"172.21.15.3:6805","nonce":926805498}]},"public_addr":"172.21.15.3:6801/926805498","cluster_addr":"172.21.15.3:6803/926805498","heartbeat_back_addr":"172.21.15.3:6807/926805498","heartbeat_front_addr":"172.21.15.3:6805/926805498","state":["exists","up"]},{"osd":2,"uuid":"1abbadb5-a570-4359-9643-acff3ede35a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":801295451},{"type":"v1","addr":"172.21.15.3:6809","nonce":801295451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":801295451},{"type":"v1","addr":"172.21.15.3:6811","nonce":801295451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":801295451},{"type":"v1","addr":"172.21.15.3:6815","nonce":801295451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":801295451},{"type":"v1","addr":"172.21.15.3:6813","nonce":801295451}]},"public_addr":"172.21.15.3:6809/801295451","cluster_addr":"172.21.15.3:6811/801295451","heartbeat_back_addr":"172.21.15.3:6815/801295451","heartbeat_front_addr":"172.21.15.3:6813/801295451","state":["exists","up"]},{"osd":3,"uuid":"62ac4dc1-2371-4dd2-acd3-e2dbb11aedf5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":1903389857},{"type":"v1","addr":"172.21.15.3:6825","nonce":1903389857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":1903389857},{"type":"v1","addr":"172.21.15.3:6827","nonce":1903389857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":1903389857},{"type":"v1","addr":"172.21.15.3:6831","nonce":1903389857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":1903389857},{"type":"v1","addr":"172.21.15.3:6829","nonce":1903389857}]},"public_addr":"172.21.15.3:6825/1903389857","cluster_addr":"172.21.15.3:6827/1903389857","heartbeat_back_addr":"172.21.15.3:6831/1903389857","heartbeat_front_addr":"172.21.15.3:6829/1903389857","state":["exists","up"]},{"osd":4,"uuid":"ae808398-8326-4d7c-9319-a2a0eafffb59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":2143624576},{"type":"v1","addr":"172.21.15.190:6801","nonce":2143624576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":2143624576},{"type":"v1","addr":"172.21.15.190:6803","nonce":2143624576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":2143624576},{"type":"v1","addr":"172.21.15.190:6807","nonce":2143624576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":2143624576},{"type":"v1","addr":"172.21.15.190:6805","nonce":2143624576}]},"public_addr":"172.21.15.190:6801/2143624576","cluster_addr":"172.21.15.190:6803/2143624576","heartbeat_back_addr":"172.21.15.190:6807/2143624576","heartbeat_front_addr":"172.21.15.190:6805/2143624576","state":["exists","up"]},{"osd":5,"uuid":"918adf50-c440-48ac-9a8d-4837562d7275","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":1766923126},{"type":"v1","addr":"172.21.15.190:6814","nonce":1766923126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":1766923126},{"type":"v1","addr":"172.21.15.190:6818","nonce":1766923126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":1766923126},{"type":"v1","addr":"172.21.15.190:6823","nonce":1766923126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":1766923126},{"type":"v1","addr":"172.21.15.190:6821","nonce":1766923126}]},"public_addr":"172.21.15.190:6814/1766923126","cluster_addr":"172.21.15.190:6818/1766923126","heartbeat_back_addr":"172.21.15.190:6823/1766923126","heartbeat_front_addr":"172.21.15.190:6821/1766923126","state":["exists","up"]},{"osd":6,"uuid":"ac2882bc-2a7c-463c-8f6c-76ff49828adc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":342414248},{"type":"v1","addr":"172.21.15.190:6825","nonce":342414248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":342414248},{"type":"v1","addr":"172.21.15.190:6827","nonce":342414248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":342414248},{"type":"v1","addr":"172.21.15.190:6831","nonce":342414248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":342414248},{"type":"v1","addr":"172.21.15.190:6829","nonce":342414248}]},"public_addr":"172.21.15.190:6825/342414248","cluster_addr":"172.21.15.190:6827/342414248","heartbeat_back_addr":"172.21.15.190:6831/342414248","heartbeat_front_addr":"172.21.15.190:6829/342414248","state":["exists","up"]},{"osd":7,"uuid":"dd4e767a-4730-4c70-8bd5-3f088140a5a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":537849891},{"type":"v1","addr":"172.21.15.190:6809","nonce":537849891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":537849891},{"type":"v1","addr":"172.21.15.190:6811","nonce":537849891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6817","nonce":537849891},{"type":"v1","addr":"172.21.15.190:6819","nonce":537849891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6813","nonce":537849891},{"type":"v1","addr":"172.21.15.190:6815","nonce":537849891}]},"public_addr":"172.21.15.190:6809/537849891","cluster_addr":"172.21.15.190:6811/537849891","heartbeat_back_addr":"172.21.15.190:6819/537849891","heartbeat_front_addr":"172.21.15.190:6815/537849891","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.889784+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.974002+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.973549+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.955396+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.928471+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.902418+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.958696+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.926474+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T06:36:32.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.454+0000 7f7eafb70640 1 -- 172.21.15.3:0/3092696624 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f7e9406be30 msgr2=0x7f7e9406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:32.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.454+0000 7f7eafb70640 1 --2- 172.21.15.3:0/3092696624 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f7e9406be30 0x7f7e9406e2f0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f7ea0032000 tx=0x7f7ea0009030 comp rx=0 tx=0).stop 2024-05-09T06:36:32.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.454+0000 7f7eafb70640 1 -- 172.21.15.3:0/3092696624 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7ea81f4020 msgr2=0x7f7ea80b8720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:32.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.454+0000 7f7eafb70640 1 --2- 172.21.15.3:0/3092696624 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7ea81f4020 0x7f7ea80b8720 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f7ea80d41d0 tx=0x7f7e9800f580 comp rx=0 tx=0).stop 2024-05-09T06:36:32.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.454+0000 7f7eafb70640 1 -- 172.21.15.3:0/3092696624 shutdown_connections 2024-05-09T06:36:32.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.454+0000 7f7eafb70640 1 --2- 172.21.15.3:0/3092696624 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f7e9406be30 0x7f7e9406e2f0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:32.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.454+0000 7f7eafb70640 1 --2- 172.21.15.3:0/3092696624 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7ea81f4020 0x7f7ea80b8720 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:32.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.454+0000 7f7eafb70640 1 --2- 172.21.15.3:0/3092696624 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7ea81f3ba0 0x7f7ea80b81e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:32.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.454+0000 7f7eafb70640 1 --2- 172.21.15.3:0/3092696624 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7e90003b70 0x7f7ea81f82e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:32.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.454+0000 7f7eafb70640 1 -- 172.21.15.3:0/3092696624 >> 172.21.15.3:0/3092696624 conn(0x7f7ea809d4d0 msgr2=0x7f7ea80d00c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:32.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.454+0000 7f7eafb70640 1 -- 172.21.15.3:0/3092696624 shutdown_connections 2024-05-09T06:36:32.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.454+0000 7f7eafb70640 1 -- 172.21.15.3:0/3092696624 wait complete. 2024-05-09T06:36:32.446 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-09T06:35:54.819999+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-09T06:36:04.931314+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-09T06:36:06.200433+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-09T06:36:32.446 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-09T06:36:32.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.575+0000 7fe31de9e640 1 Processor -- start 2024-05-09T06:36:32.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.575+0000 7fe31de9e640 1 -- start start 2024-05-09T06:36:32.557 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.576+0000 7fe31de9e640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fe3180d2800 0x7fe3180d4c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:32.557 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.576+0000 7fe31de9e640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe31815e9a0 con 0x7fe3180d1300 2024-05-09T06:36:32.557 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.576+0000 7fe31de9e640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe3180d2c50 con 0x7fe31815e5c0 2024-05-09T06:36:32.557 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.576+0000 7fe31de9e640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7fe3180d31c0 con 0x7fe3180d2800 2024-05-09T06:36:32.557 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.576+0000 7fe30ffff640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fe3180d1300 0x7fe3180b47e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:58930/0 (socket says 172.21.15.3:58930) 2024-05-09T06:36:32.557 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.576+0000 7fe30ffff640 1 -- 172.21.15.3:0/948203562 learned_addr learned my addr 172.21.15.3:0/948203562 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-05-09T06:36:32.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.576+0000 7fe317fff640 1 --2- 172.21.15.3:0/948203562 >> v2:172.21.15.190:3300/0 conn(0x7fe3180d2800 0x7fe3180d4c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:32.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.576+0000 7fe316ffd640 1 -- 172.21.15.3:0/948203562 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 308764472 0 0) 0x7fe31815e9a0 con 0x7fe3180d1300 2024-05-09T06:36:32.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.576+0000 7fe316ffd640 1 -- 172.21.15.3:0/948203562 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe304003520 con 0x7fe3180d1300 2024-05-09T06:36:32.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.576+0000 7fe317fff640 1 -- 172.21.15.3:0/948203562 >> v1:172.21.15.190:6789/0 conn(0x7fe31815e5c0 legacy=0x7fe3180d0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:32.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.576+0000 7fe317fff640 1 -- 172.21.15.3:0/948203562 >> v1:172.21.15.3:6789/0 conn(0x7fe3180d1300 legacy=0x7fe3180b47e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:32.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.576+0000 7fe316ffd640 1 -- 172.21.15.3:0/948203562 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2806044374 0 0) 0x7fe304003520 con 0x7fe3180d1300 2024-05-09T06:36:32.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.576+0000 7fe317fff640 1 -- 172.21.15.3:0/948203562 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3180d2ee0 con 0x7fe3180d2800 2024-05-09T06:36:32.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.577+0000 7fe317fff640 1 --2- 172.21.15.3:0/948203562 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe3180d2800 0x7fe3180d4c40 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fe310009d90 tx=0x7fe31002a680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=31b5aa20d492c6d4 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:32.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.577+0000 7fe316ffd640 1 -- 172.21.15.3:0/948203562 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe300001670 con 0x7fe3180d2800 2024-05-09T06:36:32.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.577+0000 7fe316ffd640 1 -- 172.21.15.3:0/948203562 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe310007540 con 0x7fe3180d2800 2024-05-09T06:36:32.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.577+0000 7fe316ffd640 1 -- 172.21.15.3:0/948203562 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe310007910 con 0x7fe3180d2800 2024-05-09T06:36:32.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.578+0000 7fe31de9e640 1 -- 172.21.15.3:0/948203562 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe3180d2800 msgr2=0x7fe3180d4c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:32.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.578+0000 7fe31de9e640 1 --2- 172.21.15.3:0/948203562 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe3180d2800 0x7fe3180d4c40 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fe310009d90 tx=0x7fe31002a680 comp rx=0 tx=0).stop 2024-05-09T06:36:32.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.578+0000 7fe31de9e640 1 -- 172.21.15.3:0/948203562 shutdown_connections 2024-05-09T06:36:32.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.578+0000 7fe31de9e640 1 --2- 172.21.15.3:0/948203562 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe3180d2800 0x7fe3180d4c40 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:32.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.578+0000 7fe31de9e640 1 -- 172.21.15.3:0/948203562 >> 172.21.15.3:0/948203562 conn(0x7fe31809d500 msgr2=0x7fe3180d4220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:32.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.578+0000 7fe31de9e640 1 -- 172.21.15.3:0/948203562 shutdown_connections 2024-05-09T06:36:32.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.578+0000 7fe31de9e640 1 -- 172.21.15.3:0/948203562 wait complete. 2024-05-09T06:36:32.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.579+0000 7fe31de9e640 1 Processor -- start 2024-05-09T06:36:32.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.579+0000 7fe31de9e640 1 -- start start 2024-05-09T06:36:32.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.580+0000 7fe31de9e640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe3180d1300 0x7fe318153740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:32.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.580+0000 7fe31de9e640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe3180d2800 0x7fe3180ccee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:32.564 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.580+0000 7fe31de9e640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe31815e5c0 0x7fe3180cd420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:32.564 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.580+0000 7fe31de9e640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fe3180b42c0 con 0x7fe31815e5c0 2024-05-09T06:36:32.564 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.580+0000 7fe31de9e640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fe3180b3fe0 con 0x7fe3180d1300 2024-05-09T06:36:32.564 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.580+0000 7fe31de9e640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fe3180b4ef0 con 0x7fe3180d2800 2024-05-09T06:36:32.564 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.580+0000 7fe317fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe31815e5c0 0x7fe3180cd420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:32.564 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.580+0000 7fe317fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe31815e5c0 0x7fe3180cd420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:35448/0 (socket says 172.21.15.3:35448) 2024-05-09T06:36:32.564 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.580+0000 7fe317fff640 1 -- 172.21.15.3:0/988438462 learned_addr learned my addr 172.21.15.3:0/988438462 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:36:32.564 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.580+0000 7fe30ffff640 1 --2- 172.21.15.3:0/988438462 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe3180d2800 0x7fe3180ccee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:32.564 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.580+0000 7fe317fff640 1 -- 172.21.15.3:0/988438462 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe3180d2800 msgr2=0x7fe3180ccee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:32.564 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.580+0000 7fe3177fe640 1 --2- 172.21.15.3:0/988438462 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe3180d1300 0x7fe318153740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:32.564 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.580+0000 7fe317fff640 1 --2- 172.21.15.3:0/988438462 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe3180d2800 0x7fe3180ccee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:32.564 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.580+0000 7fe317fff640 1 -- 172.21.15.3:0/988438462 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe3180d1300 msgr2=0x7fe318153740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:32.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.580+0000 7fe317fff640 1 --2- 172.21.15.3:0/988438462 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe3180d1300 0x7fe318153740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:32.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.580+0000 7fe317fff640 1 -- 172.21.15.3:0/988438462 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3180cbcb0 con 0x7fe31815e5c0 2024-05-09T06:36:32.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.580+0000 7fe3177fe640 1 --2- 172.21.15.3:0/988438462 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe3180d1300 0x7fe318153740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:36:32.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.581+0000 7fe317fff640 1 --2- 172.21.15.3:0/988438462 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe31815e5c0 0x7fe3180cd420 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fe310009eb0 tx=0x7fe310004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:32.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.581+0000 7fe314ff9640 1 -- 172.21.15.3:0/988438462 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe310040070 con 0x7fe31815e5c0 2024-05-09T06:36:32.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.581+0000 7fe314ff9640 1 -- 172.21.15.3:0/988438462 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe310031c40 con 0x7fe31815e5c0 2024-05-09T06:36:32.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.581+0000 7fe31de9e640 1 -- 172.21.15.3:0/988438462 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3180b9b60 con 0x7fe31815e5c0 2024-05-09T06:36:32.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.581+0000 7fe314ff9640 1 -- 172.21.15.3:0/988438462 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe310009450 con 0x7fe31815e5c0 2024-05-09T06:36:32.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.581+0000 7fe31de9e640 1 -- 172.21.15.3:0/988438462 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3180cc230 con 0x7fe31815e5c0 2024-05-09T06:36:32.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.582+0000 7fe31de9e640 1 -- 172.21.15.3:0/988438462 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe31815f8b0 con 0x7fe31815e5c0 2024-05-09T06:36:32.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.582+0000 7fe314ff9640 1 -- 172.21.15.3:0/988438462 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe310039040 con 0x7fe31815e5c0 2024-05-09T06:36:32.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.583+0000 7fe314ff9640 1 --2- 172.21.15.3:0/988438462 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fe2f806be60 0x7fe2f806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:32.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.583+0000 7fe314ff9640 1 -- 172.21.15.3:0/988438462 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fe3100ac2a0 con 0x7fe31815e5c0 2024-05-09T06:36:32.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.583+0000 7fe3177fe640 1 --2- 172.21.15.3:0/988438462 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fe2f806be60 0x7fe2f806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:32.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.584+0000 7fe3177fe640 1 --2- 172.21.15.3:0/988438462 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fe2f806be60 0x7fe2f806e320 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fe308000f60 tx=0x7fe308009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:32.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.587+0000 7fe314ff9640 1 -- 172.21.15.3:0/988438462 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fe31815f8b0 con 0x7fe31815e5c0 2024-05-09T06:36:32.750 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.770+0000 7fe31de9e640 1 -- 172.21.15.3:0/988438462 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fe3180cc610 con 0x7fe31815e5c0 2024-05-09T06:36:32.750 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.771+0000 7fe314ff9640 1 -- 172.21.15.3:0/988438462 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7fe31004b340 con 0x7fe31815e5c0 2024-05-09T06:36:32.751 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 1 2024-05-09T06:36:32.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.773+0000 7fe31de9e640 1 -- 172.21.15.3:0/988438462 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fe2f806be60 msgr2=0x7fe2f806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:32.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.773+0000 7fe31de9e640 1 --2- 172.21.15.3:0/988438462 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fe2f806be60 0x7fe2f806e320 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fe308000f60 tx=0x7fe308009040 comp rx=0 tx=0).stop 2024-05-09T06:36:32.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.773+0000 7fe31de9e640 1 -- 172.21.15.3:0/988438462 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe31815e5c0 msgr2=0x7fe3180cd420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:32.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.773+0000 7fe31de9e640 1 --2- 172.21.15.3:0/988438462 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe31815e5c0 0x7fe3180cd420 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fe310009eb0 tx=0x7fe310004a40 comp rx=0 tx=0).stop 2024-05-09T06:36:32.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.773+0000 7fe31de9e640 1 -- 172.21.15.3:0/988438462 shutdown_connections 2024-05-09T06:36:32.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.773+0000 7fe31de9e640 1 --2- 172.21.15.3:0/988438462 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fe2f806be60 0x7fe2f806e320 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:32.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.773+0000 7fe31de9e640 1 --2- 172.21.15.3:0/988438462 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe31815e5c0 0x7fe3180cd420 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:32.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.773+0000 7fe31de9e640 1 --2- 172.21.15.3:0/988438462 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe3180d2800 0x7fe3180ccee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:32.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.773+0000 7fe31de9e640 1 --2- 172.21.15.3:0/988438462 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe3180d1300 0x7fe318153740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:32.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.773+0000 7fe31de9e640 1 -- 172.21.15.3:0/988438462 >> 172.21.15.3:0/988438462 conn(0x7fe31809d500 msgr2=0x7fe3180b1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:32.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.773+0000 7fe31de9e640 1 -- 172.21.15.3:0/988438462 shutdown_connections 2024-05-09T06:36:32.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.773+0000 7fe31de9e640 1 -- 172.21.15.3:0/988438462 wait complete. 2024-05-09T06:36:32.764 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-09T06:36:32.871 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.891+0000 7fb594093640 1 Processor -- start 2024-05-09T06:36:32.871 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.891+0000 7fb594093640 1 -- start start 2024-05-09T06:36:32.872 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.892+0000 7fb594093640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fb58c06cd70 0x7fb58c162a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:32.872 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.892+0000 7fb594093640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb58c0d02b0 0x7fb58c06d150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:32.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.892+0000 7fb594093640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb58c162fa0 con 0x7fb58c09aa30 2024-05-09T06:36:32.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.892+0000 7fb594093640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7fb58c161e40 con 0x7fb58c06cd70 2024-05-09T06:36:32.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.892+0000 7fb594093640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fb58c06d690 con 0x7fb58c0d02b0 2024-05-09T06:36:32.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.892+0000 7fb592630640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fb58c09aa30 0x7fb58c09aec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:58942/0 (socket says 172.21.15.3:58942) 2024-05-09T06:36:32.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.892+0000 7fb592630640 1 -- 172.21.15.3:0/1833489667 learned_addr learned my addr 172.21.15.3:0/1833489667 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-05-09T06:36:32.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.892+0000 7fb59162e640 1 --2- 172.21.15.3:0/1833489667 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb58c0d02b0 0x7fb58c06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:32.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.892+0000 7fb591e2f640 1 --2- 172.21.15.3:0/1833489667 >> v2:172.21.15.190:3300/0 conn(0x7fb58c06cd70 0x7fb58c162a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:32.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.892+0000 7fb590e2d640 1 -- 172.21.15.3:0/1833489667 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2489428004 0 0) 0x7fb58c162fa0 con 0x7fb58c09aa30 2024-05-09T06:36:32.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.892+0000 7fb590e2d640 1 -- 172.21.15.3:0/1833489667 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb578003520 con 0x7fb58c09aa30 2024-05-09T06:36:32.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.892+0000 7fb59162e640 1 -- 172.21.15.3:0/1833489667 >> v2:172.21.15.190:3300/0 conn(0x7fb58c06cd70 msgr2=0x7fb58c162a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:32.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.892+0000 7fb59162e640 1 --2- 172.21.15.3:0/1833489667 >> v2:172.21.15.190:3300/0 conn(0x7fb58c06cd70 0x7fb58c162a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:32.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.892+0000 7fb59162e640 1 -- 172.21.15.3:0/1833489667 >> v1:172.21.15.3:6789/0 conn(0x7fb58c09aa30 legacy=0x7fb58c09aec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:32.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.892+0000 7fb59162e640 1 -- 172.21.15.3:0/1833489667 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb58c0d0870 con 0x7fb58c0d02b0 2024-05-09T06:36:32.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.892+0000 7fb591e2f640 1 --2- 172.21.15.3:0/1833489667 >> v2:172.21.15.190:3300/0 conn(0x7fb58c06cd70 0x7fb58c162a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T06:36:32.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.893+0000 7fb59162e640 1 --2- 172.21.15.3:0/1833489667 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb58c0d02b0 0x7fb58c06d150 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fb57c0097a0 tx=0x7fb57c02a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=506f571bb9c3998d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:32.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.893+0000 7fb590e2d640 1 -- 172.21.15.3:0/1833489667 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb57c037070 con 0x7fb58c0d02b0 2024-05-09T06:36:32.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.893+0000 7fb590e2d640 1 -- 172.21.15.3:0/1833489667 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb57c032440 con 0x7fb58c0d02b0 2024-05-09T06:36:32.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.893+0000 7fb590e2d640 1 -- 172.21.15.3:0/1833489667 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb57c0327f0 con 0x7fb58c0d02b0 2024-05-09T06:36:32.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.894+0000 7fb594093640 1 -- 172.21.15.3:0/1833489667 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb58c0d02b0 msgr2=0x7fb58c06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:32.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.894+0000 7fb594093640 1 --2- 172.21.15.3:0/1833489667 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb58c0d02b0 0x7fb58c06d150 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fb57c0097a0 tx=0x7fb57c02a180 comp rx=0 tx=0).stop 2024-05-09T06:36:32.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.894+0000 7fb594093640 1 -- 172.21.15.3:0/1833489667 shutdown_connections 2024-05-09T06:36:32.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.894+0000 7fb594093640 1 --2- 172.21.15.3:0/1833489667 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb58c0d02b0 0x7fb58c06d150 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:32.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.894+0000 7fb594093640 1 --2- 172.21.15.3:0/1833489667 >> v2:172.21.15.190:3300/0 conn(0x7fb58c06cd70 0x7fb58c162a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:32.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.894+0000 7fb594093640 1 -- 172.21.15.3:0/1833489667 >> 172.21.15.3:0/1833489667 conn(0x7fb58c0b3af0 msgr2=0x7fb58c0b3f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:32.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.894+0000 7fb594093640 1 -- 172.21.15.3:0/1833489667 shutdown_connections 2024-05-09T06:36:32.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.894+0000 7fb594093640 1 -- 172.21.15.3:0/1833489667 wait complete. 2024-05-09T06:36:32.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.895+0000 7fb594093640 1 Processor -- start 2024-05-09T06:36:32.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.895+0000 7fb594093640 1 -- start start 2024-05-09T06:36:32.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.896+0000 7fb594093640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb58c06cd70 0x7fb58c0c5b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:32.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.896+0000 7fb594093640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb58c09aa30 0x7fb58c0b2f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:32.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.896+0000 7fb594093640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb58c0d02b0 0x7fb58c1500e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:32.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.896+0000 7fb594093640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fb58c0d1110 con 0x7fb58c09aa30 2024-05-09T06:36:32.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.896+0000 7fb594093640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fb58c0d0e30 con 0x7fb58c0d02b0 2024-05-09T06:36:32.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.896+0000 7fb594093640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fb58c0d13f0 con 0x7fb58c06cd70 2024-05-09T06:36:32.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.896+0000 7fb59162e640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb58c09aa30 0x7fb58c0b2f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:32.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.896+0000 7fb59162e640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb58c09aa30 0x7fb58c0b2f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:35464/0 (socket says 172.21.15.3:35464) 2024-05-09T06:36:32.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.896+0000 7fb59162e640 1 -- 172.21.15.3:0/661388917 learned_addr learned my addr 172.21.15.3:0/661388917 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:36:32.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.896+0000 7fb591e2f640 1 --2- 172.21.15.3:0/661388917 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb58c06cd70 0x7fb58c0c5b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:32.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.896+0000 7fb592630640 1 --2- 172.21.15.3:0/661388917 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb58c0d02b0 0x7fb58c1500e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:32.883 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.897+0000 7fb59162e640 1 -- 172.21.15.3:0/661388917 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb58c06cd70 msgr2=0x7fb58c0c5b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:32.883 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.897+0000 7fb59162e640 1 --2- 172.21.15.3:0/661388917 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb58c06cd70 0x7fb58c0c5b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:32.883 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.897+0000 7fb59162e640 1 -- 172.21.15.3:0/661388917 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb58c0d02b0 msgr2=0x7fb58c1500e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:32.883 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.897+0000 7fb59162e640 1 --2- 172.21.15.3:0/661388917 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb58c0d02b0 0x7fb58c1500e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:32.883 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.897+0000 7fb59162e640 1 -- 172.21.15.3:0/661388917 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb58c0c8900 con 0x7fb58c09aa30 2024-05-09T06:36:32.883 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.897+0000 7fb59162e640 1 --2- 172.21.15.3:0/661388917 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb58c09aa30 0x7fb58c0b2f10 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fb57c02a1c0 tx=0x7fb57c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:32.883 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.897+0000 7fb5827fc640 1 -- 172.21.15.3:0/661388917 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb57c037040 con 0x7fb58c09aa30 2024-05-09T06:36:32.883 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.897+0000 7fb5827fc640 1 -- 172.21.15.3:0/661388917 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb57c042070 con 0x7fb58c09aa30 2024-05-09T06:36:32.883 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.897+0000 7fb594093640 1 -- 172.21.15.3:0/661388917 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb58c0c7860 con 0x7fb58c09aa30 2024-05-09T06:36:32.883 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.897+0000 7fb5827fc640 1 -- 172.21.15.3:0/661388917 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb57c03b930 con 0x7fb58c09aa30 2024-05-09T06:36:32.883 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.897+0000 7fb594093640 1 -- 172.21.15.3:0/661388917 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb58c0c7e20 con 0x7fb58c09aa30 2024-05-09T06:36:32.883 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.898+0000 7fb594093640 1 -- 172.21.15.3:0/661388917 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb58c0d1a90 con 0x7fb58c09aa30 2024-05-09T06:36:32.883 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.898+0000 7fb5827fc640 1 -- 172.21.15.3:0/661388917 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb57c03bad0 con 0x7fb58c09aa30 2024-05-09T06:36:32.883 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.899+0000 7fb5827fc640 1 --2- 172.21.15.3:0/661388917 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fb57006be30 0x7fb57006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:32.883 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.899+0000 7fb5827fc640 1 -- 172.21.15.3:0/661388917 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fb57c0ab670 con 0x7fb58c09aa30 2024-05-09T06:36:32.884 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.899+0000 7fb591e2f640 1 --2- 172.21.15.3:0/661388917 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fb57006be30 0x7fb57006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:32.884 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.900+0000 7fb591e2f640 1 --2- 172.21.15.3:0/661388917 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fb57006be30 0x7fb57006e2f0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fb588007440 tx=0x7fb58800b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:32.884 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:32.903+0000 7fb5827fc640 1 -- 172.21.15.3:0/661388917 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fb58c0d1a90 con 0x7fb58c09aa30 2024-05-09T06:36:33.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.087+0000 7fb594093640 1 -- 172.21.15.3:0/661388917 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fb58c06bd80 con 0x7fb58c09aa30 2024-05-09T06:36:33.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.087+0000 7fb5827fc640 1 -- 172.21.15.3:0/661388917 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7fb58c06bd80 con 0x7fb58c09aa30 2024-05-09T06:36:33.067 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 32 2024-05-09T06:36:33.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.089+0000 7fb594093640 1 -- 172.21.15.3:0/661388917 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fb57006be30 msgr2=0x7fb57006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:33.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.089+0000 7fb594093640 1 --2- 172.21.15.3:0/661388917 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fb57006be30 0x7fb57006e2f0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fb588007440 tx=0x7fb58800b040 comp rx=0 tx=0).stop 2024-05-09T06:36:33.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.089+0000 7fb594093640 1 -- 172.21.15.3:0/661388917 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb58c09aa30 msgr2=0x7fb58c0b2f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:33.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.089+0000 7fb594093640 1 --2- 172.21.15.3:0/661388917 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb58c09aa30 0x7fb58c0b2f10 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fb57c02a1c0 tx=0x7fb57c0046c0 comp rx=0 tx=0).stop 2024-05-09T06:36:33.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.089+0000 7fb594093640 1 -- 172.21.15.3:0/661388917 shutdown_connections 2024-05-09T06:36:33.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.089+0000 7fb594093640 1 --2- 172.21.15.3:0/661388917 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fb57006be30 0x7fb57006e2f0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:33.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.089+0000 7fb594093640 1 --2- 172.21.15.3:0/661388917 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb58c0d02b0 0x7fb58c1500e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:33.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.089+0000 7fb594093640 1 --2- 172.21.15.3:0/661388917 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb58c09aa30 0x7fb58c0b2f10 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:33.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.089+0000 7fb594093640 1 --2- 172.21.15.3:0/661388917 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb58c06cd70 0x7fb58c0c5b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:33.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.089+0000 7fb594093640 1 -- 172.21.15.3:0/661388917 >> 172.21.15.3:0/661388917 conn(0x7fb58c0b3af0 msgr2=0x7fb58c09aec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:33.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.089+0000 7fb594093640 1 -- 172.21.15.3:0/661388917 shutdown_connections 2024-05-09T06:36:33.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.089+0000 7fb594093640 1 -- 172.21.15.3:0/661388917 wait complete. 2024-05-09T06:36:33.080 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-09T06:36:33.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.207+0000 7fc14eb63640 1 Processor -- start 2024-05-09T06:36:33.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.207+0000 7fc14eb63640 1 -- start start 2024-05-09T06:36:33.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.208+0000 7fc14eb63640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fc14806cd70 0x7fc148162a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:33.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.208+0000 7fc14eb63640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc1480cc750 0x7fc1480ccbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:33.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.208+0000 7fc14eb63640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc148162fa0 con 0x7fc1480d02b0 2024-05-09T06:36:33.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.208+0000 7fc14eb63640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7fc148161e40 con 0x7fc14806cd70 2024-05-09T06:36:33.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.208+0000 7fc14eb63640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fc1480d0870 con 0x7fc1480cc750 2024-05-09T06:36:33.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.208+0000 7fc147fff640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fc1480d02b0 0x7fc14806d150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:58948/0 (socket says 172.21.15.3:58948) 2024-05-09T06:36:33.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.208+0000 7fc147fff640 1 -- 172.21.15.3:0/2926570727 learned_addr learned my addr 172.21.15.3:0/2926570727 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-05-09T06:36:33.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.209+0000 7fc14c8ff640 1 --2- 172.21.15.3:0/2926570727 >> v2:172.21.15.190:3300/0 conn(0x7fc14806cd70 0x7fc148162a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:33.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.209+0000 7fc14d100640 1 --2- 172.21.15.3:0/2926570727 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc1480cc750 0x7fc1480ccbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:33.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.209+0000 7fc1477fe640 1 -- 172.21.15.3:0/2926570727 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1055584922 0 0) 0x7fc148162fa0 con 0x7fc1480d02b0 2024-05-09T06:36:33.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.209+0000 7fc1477fe640 1 -- 172.21.15.3:0/2926570727 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc134003520 con 0x7fc1480d02b0 2024-05-09T06:36:33.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.209+0000 7fc1477fe640 1 -- 172.21.15.3:0/2926570727 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4246695621 0 0) 0x7fc134003520 con 0x7fc1480d02b0 2024-05-09T06:36:33.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.209+0000 7fc1477fe640 1 -- 172.21.15.3:0/2926570727 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc1480cc750 msgr2=0x7fc1480ccbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:33.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.209+0000 7fc1477fe640 1 --2- 172.21.15.3:0/2926570727 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc1480cc750 0x7fc1480ccbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:33.192 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.209+0000 7fc1477fe640 1 -- 172.21.15.3:0/2926570727 >> v2:172.21.15.190:3300/0 conn(0x7fc14806cd70 msgr2=0x7fc148162a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:33.192 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.209+0000 7fc1477fe640 1 --2- 172.21.15.3:0/2926570727 >> v2:172.21.15.190:3300/0 conn(0x7fc14806cd70 0x7fc148162a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:33.192 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.209+0000 7fc1477fe640 1 -- 172.21.15.3:0/2926570727 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1480d0b50 con 0x7fc1480d02b0 2024-05-09T06:36:33.192 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.209+0000 7fc1477fe640 1 -- 172.21.15.3:0/2926570727 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 563312031 0 0) 0x7fc1380019f0 con 0x7fc1480d02b0 2024-05-09T06:36:33.192 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.209+0000 7fc1477fe640 1 -- 172.21.15.3:0/2926570727 >> v1:172.21.15.3:6789/0 conn(0x7fc1480d02b0 legacy=0x7fc14806d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:33.192 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.209+0000 7fc1477fe640 1 --2- 172.21.15.3:0/2926570727 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc1340039b0 0x7fc134003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:33.192 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.209+0000 7fc1477fe640 1 --2- 172.21.15.3:0/2926570727 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc134004800 0x7fc134004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:33.192 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.209+0000 7fc1477fe640 1 --2- 172.21.15.3:0/2926570727 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc1480cc750 0x7fc1340051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:33.192 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.209+0000 7fc1477fe640 1 -- 172.21.15.3:0/2926570727 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fc148161e40 con 0x7fc1340039b0 2024-05-09T06:36:33.192 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.209+0000 7fc1477fe640 1 -- 172.21.15.3:0/2926570727 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fc1480d0870 con 0x7fc134004800 2024-05-09T06:36:33.192 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.209+0000 7fc1477fe640 1 -- 172.21.15.3:0/2926570727 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fc13400b9c0 con 0x7fc1480cc750 2024-05-09T06:36:33.192 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.209+0000 7fc14c8ff640 1 --2- 172.21.15.3:0/2926570727 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc1340039b0 0x7fc134003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:33.192 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.209+0000 7fc147fff640 1 --2- 172.21.15.3:0/2926570727 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc134004800 0x7fc134004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:33.192 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.209+0000 7fc14d100640 1 --2- 172.21.15.3:0/2926570727 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc1480cc750 0x7fc1340051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:33.193 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.209+0000 7fc14c8ff640 1 -- 172.21.15.3:0/2926570727 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc1480cc750 msgr2=0x7fc1340051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:33.193 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.209+0000 7fc14c8ff640 1 --2- 172.21.15.3:0/2926570727 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc1480cc750 0x7fc1340051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:33.193 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.209+0000 7fc14c8ff640 1 -- 172.21.15.3:0/2926570727 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc134004800 msgr2=0x7fc134004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:33.193 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.209+0000 7fc14c8ff640 1 --2- 172.21.15.3:0/2926570727 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc134004800 0x7fc134004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:33.193 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.209+0000 7fc14c8ff640 1 -- 172.21.15.3:0/2926570727 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc134003520 con 0x7fc1340039b0 2024-05-09T06:36:33.193 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.210+0000 7fc14c8ff640 1 --2- 172.21.15.3:0/2926570727 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc1340039b0 0x7fc134003dd0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fc140009ac0 tx=0x7fc134005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d839a4cc8001b6e5 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:33.193 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.210+0000 7fc1477fe640 1 -- 172.21.15.3:0/2926570727 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc140036070 con 0x7fc1340039b0 2024-05-09T06:36:33.193 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.210+0000 7fc1477fe640 1 -- 172.21.15.3:0/2926570727 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc140031400 con 0x7fc1340039b0 2024-05-09T06:36:33.193 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.210+0000 7fc1477fe640 1 -- 172.21.15.3:0/2926570727 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc140009690 con 0x7fc1340039b0 2024-05-09T06:36:33.193 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.211+0000 7fc14eb63640 1 -- 172.21.15.3:0/2926570727 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc1340039b0 msgr2=0x7fc134003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:33.193 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.211+0000 7fc14eb63640 1 --2- 172.21.15.3:0/2926570727 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc1340039b0 0x7fc134003dd0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fc140009ac0 tx=0x7fc134005830 comp rx=0 tx=0).stop 2024-05-09T06:36:33.193 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.211+0000 7fc14d100640 1 -- 172.21.15.3:0/2926570727 reap_dead start 2024-05-09T06:36:33.193 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.211+0000 7fc14eb63640 1 -- 172.21.15.3:0/2926570727 shutdown_connections 2024-05-09T06:36:33.193 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.211+0000 7fc14eb63640 1 -- 172.21.15.3:0/2926570727 >> 172.21.15.3:0/2926570727 conn(0x7fc1480b3af0 msgr2=0x7fc1480b3f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:33.194 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.211+0000 7fc14eb63640 1 -- 172.21.15.3:0/2926570727 shutdown_connections 2024-05-09T06:36:33.194 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.211+0000 7fc14eb63640 1 -- 172.21.15.3:0/2926570727 wait complete. 2024-05-09T06:36:33.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.212+0000 7fc14eb63640 1 Processor -- start 2024-05-09T06:36:33.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.212+0000 7fc14eb63640 1 -- start start 2024-05-09T06:36:33.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.213+0000 7fc14eb63640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc1340039b0 0x7fc1480b77f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:33.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.213+0000 7fc14eb63640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc14815d5b0 0x7fc14814cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:33.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.213+0000 7fc14eb63640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc14814fe50 0x7fc148160350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:33.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.213+0000 7fc14eb63640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fc1480d13f0 con 0x7fc14814fe50 2024-05-09T06:36:33.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.213+0000 7fc14eb63640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fc1480d1110 con 0x7fc1340039b0 2024-05-09T06:36:33.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.213+0000 7fc14eb63640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fc1480ccbe0 con 0x7fc14815d5b0 2024-05-09T06:36:33.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.213+0000 7fc14d100640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc14814fe50 0x7fc148160350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:33.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.213+0000 7fc14d100640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc14814fe50 0x7fc148160350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:35480/0 (socket says 172.21.15.3:35480) 2024-05-09T06:36:33.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.213+0000 7fc14d100640 1 -- 172.21.15.3:0/2172614525 learned_addr learned my addr 172.21.15.3:0/2172614525 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:36:33.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.213+0000 7fc147fff640 1 --2- 172.21.15.3:0/2172614525 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc14815d5b0 0x7fc14814cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:33.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.213+0000 7fc14c8ff640 1 --2- 172.21.15.3:0/2172614525 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc1340039b0 0x7fc1480b77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:33.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.214+0000 7fc14d100640 1 -- 172.21.15.3:0/2172614525 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc14815d5b0 msgr2=0x7fc14814cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:33.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.214+0000 7fc14d100640 1 --2- 172.21.15.3:0/2172614525 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc14815d5b0 0x7fc14814cbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:33.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.214+0000 7fc14d100640 1 -- 172.21.15.3:0/2172614525 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc1340039b0 msgr2=0x7fc1480b77f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:33.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.214+0000 7fc14d100640 1 --2- 172.21.15.3:0/2172614525 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc1340039b0 0x7fc1480b77f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:33.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.214+0000 7fc14d100640 1 -- 172.21.15.3:0/2172614525 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc14814d0f0 con 0x7fc14814fe50 2024-05-09T06:36:33.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.214+0000 7fc14d100640 1 --2- 172.21.15.3:0/2172614525 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc14814fe50 0x7fc148160350 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fc1480b4bf0 tx=0x7fc130011920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:33.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.214+0000 7fc1457fa640 1 -- 172.21.15.3:0/2172614525 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc130003610 con 0x7fc14814fe50 2024-05-09T06:36:33.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.214+0000 7fc1457fa640 1 -- 172.21.15.3:0/2172614525 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc13001b070 con 0x7fc14814fe50 2024-05-09T06:36:33.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.214+0000 7fc1457fa640 1 -- 172.21.15.3:0/2172614525 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1300039f0 con 0x7fc14814fe50 2024-05-09T06:36:33.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.214+0000 7fc14eb63640 1 -- 172.21.15.3:0/2172614525 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1480b6750 con 0x7fc14814fe50 2024-05-09T06:36:33.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.214+0000 7fc14eb63640 1 -- 172.21.15.3:0/2172614525 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1480b6d10 con 0x7fc14814fe50 2024-05-09T06:36:33.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.215+0000 7fc14eb63640 1 -- 172.21.15.3:0/2172614525 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc148069550 con 0x7fc14814fe50 2024-05-09T06:36:33.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.215+0000 7fc1457fa640 1 -- 172.21.15.3:0/2172614525 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc130003b90 con 0x7fc14814fe50 2024-05-09T06:36:33.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.216+0000 7fc1457fa640 1 --2- 172.21.15.3:0/2172614525 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fc12806be30 0x7fc12806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:33.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.216+0000 7fc1457fa640 1 -- 172.21.15.3:0/2172614525 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fc13008ad20 con 0x7fc14814fe50 2024-05-09T06:36:33.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.216+0000 7fc14c8ff640 1 --2- 172.21.15.3:0/2172614525 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fc12806be30 0x7fc12806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:33.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.217+0000 7fc14c8ff640 1 --2- 172.21.15.3:0/2172614525 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fc12806be30 0x7fc12806e2f0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fc140033000 tx=0x7fc140008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:33.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.220+0000 7fc1457fa640 1 -- 172.21.15.3:0/2172614525 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fc130058fe0 con 0x7fc14814fe50 2024-05-09T06:36:33.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.403+0000 7fc14eb63640 1 -- 172.21.15.3:0/2172614525 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fc14806cb50 con 0x7fc14814fe50 2024-05-09T06:36:33.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.403+0000 7fc1457fa640 1 -- 172.21.15.3:0/2172614525 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7fc13005d2e0 con 0x7fc14814fe50 2024-05-09T06:36:33.383 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 64 2024-05-09T06:36:33.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.405+0000 7fc14eb63640 1 -- 172.21.15.3:0/2172614525 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fc12806be30 msgr2=0x7fc12806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:33.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.405+0000 7fc14eb63640 1 --2- 172.21.15.3:0/2172614525 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fc12806be30 0x7fc12806e2f0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fc140033000 tx=0x7fc140008030 comp rx=0 tx=0).stop 2024-05-09T06:36:33.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.405+0000 7fc14eb63640 1 -- 172.21.15.3:0/2172614525 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc14814fe50 msgr2=0x7fc148160350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:33.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.405+0000 7fc14eb63640 1 --2- 172.21.15.3:0/2172614525 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc14814fe50 0x7fc148160350 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fc1480b4bf0 tx=0x7fc130011920 comp rx=0 tx=0).stop 2024-05-09T06:36:33.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.406+0000 7fc14eb63640 1 -- 172.21.15.3:0/2172614525 shutdown_connections 2024-05-09T06:36:33.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.406+0000 7fc14eb63640 1 --2- 172.21.15.3:0/2172614525 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fc12806be30 0x7fc12806e2f0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:33.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.406+0000 7fc14eb63640 1 --2- 172.21.15.3:0/2172614525 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc14814fe50 0x7fc148160350 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:33.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.406+0000 7fc14eb63640 1 --2- 172.21.15.3:0/2172614525 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc14815d5b0 0x7fc14814cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:33.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.406+0000 7fc14eb63640 1 --2- 172.21.15.3:0/2172614525 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc1340039b0 0x7fc1480b77f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:33.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.406+0000 7fc14eb63640 1 -- 172.21.15.3:0/2172614525 >> 172.21.15.3:0/2172614525 conn(0x7fc1480b3af0 msgr2=0x7fc1480ccf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:33.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.406+0000 7fc14eb63640 1 -- 172.21.15.3:0/2172614525 shutdown_connections 2024-05-09T06:36:33.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.406+0000 7fc14eb63640 1 -- 172.21.15.3:0/2172614525 wait complete. 2024-05-09T06:36:33.397 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T06:36:33.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.521+0000 7fcdc3b7e640 1 Processor -- start 2024-05-09T06:36:33.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.521+0000 7fcdc3b7e640 1 -- start start 2024-05-09T06:36:33.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.522+0000 7fcdc3b7e640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fcdbc15e100 0x7fcdbc0ae2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:33.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.522+0000 7fcdc3b7e640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcdbc06d8f0 0x7fcdbc0ae7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:33.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.522+0000 7fcdc3b7e640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcdbc06cab0 con 0x7fcdbc06bcf0 2024-05-09T06:36:33.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.522+0000 7fcdc3b7e640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7fcdbc06c890 con 0x7fcdbc15e100 2024-05-09T06:36:33.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.522+0000 7fcdc3b7e640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fcdbc0b3ed0 con 0x7fcdbc06d8f0 2024-05-09T06:36:33.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.522+0000 7fcdc1119640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcdbc06d8f0 0x7fcdbc0ae7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:33.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.522+0000 7fcdc191a640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fcdbc15e100 0x7fcdbc0ae2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:33.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.522+0000 7fcdc1119640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcdbc06d8f0 0x7fcdbc0ae7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.3:45932/0 (socket says 172.21.15.3:45932) 2024-05-09T06:36:33.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.522+0000 7fcdc1119640 1 -- 172.21.15.3:0/2158774164 learned_addr learned my addr 172.21.15.3:0/2158774164 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:36:33.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.522+0000 7fcdc1119640 1 -- 172.21.15.3:0/2158774164 >> v2:172.21.15.190:3300/0 conn(0x7fcdbc15e100 msgr2=0x7fcdbc0ae2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:33.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.522+0000 7fcdc1119640 1 --2- 172.21.15.3:0/2158774164 >> v2:172.21.15.190:3300/0 conn(0x7fcdbc15e100 0x7fcdbc0ae2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:33.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.522+0000 7fcdc1119640 1 -- 172.21.15.3:0/2158774164 >> v1:172.21.15.190:6789/0 conn(0x7fcdbc06bcf0 legacy=0x7fcdbc06c180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:33.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.522+0000 7fcdc1119640 1 -- 172.21.15.3:0/2158774164 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcdbc0b4120 con 0x7fcdbc06d8f0 2024-05-09T06:36:33.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.522+0000 7fcdc191a640 1 --2- 172.21.15.3:0/2158774164 >> v2:172.21.15.190:3300/0 conn(0x7fcdbc15e100 0x7fcdbc0ae2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T06:36:33.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.523+0000 7fcdc1119640 1 --2- 172.21.15.3:0/2158774164 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcdbc06d8f0 0x7fcdbc0ae7f0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fcdac009870 tx=0x7fcdac02a1c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e504929f0a5d7c03 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:33.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.523+0000 7fcdc0918640 1 -- 172.21.15.3:0/2158774164 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdac037070 con 0x7fcdbc06d8f0 2024-05-09T06:36:33.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.523+0000 7fcdc0918640 1 -- 172.21.15.3:0/2158774164 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcdac032440 con 0x7fcdbc06d8f0 2024-05-09T06:36:33.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.523+0000 7fcdc0918640 1 -- 172.21.15.3:0/2158774164 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdac032810 con 0x7fcdbc06d8f0 2024-05-09T06:36:33.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.524+0000 7fcdc3b7e640 1 -- 172.21.15.3:0/2158774164 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcdbc06d8f0 msgr2=0x7fcdbc0ae7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:33.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.524+0000 7fcdc3b7e640 1 --2- 172.21.15.3:0/2158774164 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcdbc06d8f0 0x7fcdbc0ae7f0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fcdac009870 tx=0x7fcdac02a1c0 comp rx=0 tx=0).stop 2024-05-09T06:36:33.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.524+0000 7fcdc3b7e640 1 -- 172.21.15.3:0/2158774164 shutdown_connections 2024-05-09T06:36:33.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.524+0000 7fcdc3b7e640 1 --2- 172.21.15.3:0/2158774164 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcdbc06d8f0 0x7fcdbc0ae7f0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:33.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.524+0000 7fcdc3b7e640 1 --2- 172.21.15.3:0/2158774164 >> v2:172.21.15.190:3300/0 conn(0x7fcdbc15e100 0x7fcdbc0ae2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:33.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.524+0000 7fcdc3b7e640 1 -- 172.21.15.3:0/2158774164 >> 172.21.15.3:0/2158774164 conn(0x7fcdbc0d05d0 msgr2=0x7fcdbc0d09e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:33.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.524+0000 7fcdc3b7e640 1 -- 172.21.15.3:0/2158774164 shutdown_connections 2024-05-09T06:36:33.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.524+0000 7fcdc3b7e640 1 -- 172.21.15.3:0/2158774164 wait complete. 2024-05-09T06:36:33.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.525+0000 7fcdc3b7e640 1 Processor -- start 2024-05-09T06:36:33.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.525+0000 7fcdc3b7e640 1 -- start start 2024-05-09T06:36:33.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.526+0000 7fcdc3b7e640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcdbc06bcf0 0x7fcdbc14ec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:33.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.526+0000 7fcdc3b7e640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcdbc06d8f0 0x7fcdbc0ce3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:33.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.526+0000 7fcdc3b7e640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcdbc15e100 0x7fcdbc0ce920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:33.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.526+0000 7fcdc3b7e640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fcdbc15e4e0 con 0x7fcdbc06bcf0 2024-05-09T06:36:33.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.526+0000 7fcdc3b7e640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fcdbc0b4770 con 0x7fcdbc15e100 2024-05-09T06:36:33.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.526+0000 7fcdc3b7e640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fcdbc15e660 con 0x7fcdbc06d8f0 2024-05-09T06:36:33.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.526+0000 7fcdc191a640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcdbc06bcf0 0x7fcdbc14ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:33.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.526+0000 7fcdc191a640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcdbc06bcf0 0x7fcdbc14ec70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:35482/0 (socket says 172.21.15.3:35482) 2024-05-09T06:36:33.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.526+0000 7fcdc191a640 1 -- 172.21.15.3:0/481729666 learned_addr learned my addr 172.21.15.3:0/481729666 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:36:33.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.526+0000 7fcdc1119640 1 --2- 172.21.15.3:0/481729666 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcdbc06d8f0 0x7fcdbc0ce3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:33.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.526+0000 7fcdc211b640 1 --2- 172.21.15.3:0/481729666 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcdbc15e100 0x7fcdbc0ce920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:33.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.526+0000 7fcdc191a640 1 -- 172.21.15.3:0/481729666 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcdbc06d8f0 msgr2=0x7fcdbc0ce3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:33.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.526+0000 7fcdc191a640 1 --2- 172.21.15.3:0/481729666 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcdbc06d8f0 0x7fcdbc0ce3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:33.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.526+0000 7fcdc191a640 1 -- 172.21.15.3:0/481729666 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcdbc15e100 msgr2=0x7fcdbc0ce920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:33.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.526+0000 7fcdc191a640 1 --2- 172.21.15.3:0/481729666 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcdbc15e100 0x7fcdbc0ce920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:33.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.526+0000 7fcdc191a640 1 -- 172.21.15.3:0/481729666 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcdbc15e7e0 con 0x7fcdbc06bcf0 2024-05-09T06:36:33.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.526+0000 7fcdc191a640 1 --2- 172.21.15.3:0/481729666 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcdbc06bcf0 0x7fcdbc14ec70 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fcdb800e960 tx=0x7fcdb800ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:33.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.527+0000 7fcdb27fc640 1 -- 172.21.15.3:0/481729666 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdb8017070 con 0x7fcdbc06bcf0 2024-05-09T06:36:33.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.527+0000 7fcdb27fc640 1 -- 172.21.15.3:0/481729666 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcdb800ca00 con 0x7fcdbc06bcf0 2024-05-09T06:36:33.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.527+0000 7fcdc3b7e640 1 -- 172.21.15.3:0/481729666 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcdbc15eac0 con 0x7fcdbc06bcf0 2024-05-09T06:36:33.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.527+0000 7fcdb27fc640 1 -- 172.21.15.3:0/481729666 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdb800cdd0 con 0x7fcdbc06bcf0 2024-05-09T06:36:33.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.527+0000 7fcdc3b7e640 1 -- 172.21.15.3:0/481729666 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcdbc15f080 con 0x7fcdbc06bcf0 2024-05-09T06:36:33.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.528+0000 7fcdc3b7e640 1 -- 172.21.15.3:0/481729666 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcdbc04f660 con 0x7fcdbc06bcf0 2024-05-09T06:36:33.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.528+0000 7fcdb27fc640 1 -- 172.21.15.3:0/481729666 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fcdb801a070 con 0x7fcdbc06bcf0 2024-05-09T06:36:33.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.529+0000 7fcdb27fc640 1 --2- 172.21.15.3:0/481729666 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fcda006be30 0x7fcda006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:33.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.529+0000 7fcdb27fc640 1 -- 172.21.15.3:0/481729666 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fcdb808b0d0 con 0x7fcdbc06bcf0 2024-05-09T06:36:33.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.529+0000 7fcdc1119640 1 --2- 172.21.15.3:0/481729666 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fcda006be30 0x7fcda006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:33.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.529+0000 7fcdc1119640 1 --2- 172.21.15.3:0/481729666 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fcda006be30 0x7fcda006e2f0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fcdac034000 tx=0x7fcdac02e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:33.513 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.533+0000 7fcdb27fc640 1 -- 172.21.15.3:0/481729666 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fcdb8059a80 con 0x7fcdbc06bcf0 2024-05-09T06:36:33.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.718+0000 7fcdc3b7e640 1 -- 172.21.15.3:0/481729666 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fcdbc0b3cb0 con 0x7fcdbc06bcf0 2024-05-09T06:36:33.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.719+0000 7fcdb27fc640 1 -- 172.21.15.3:0/481729666 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16521 (secure 0 0 0) 0x7fcdb8059380 con 0x7fcdbc06bcf0 2024-05-09T06:36:33.699 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-09T06:36:33.699 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":20,"fsid":"16ed737d-2665-431a-8ac6-ef242be3975a","created":"2024-05-09T06:35:42.541368+0000","modified":"2024-05-09T06:36:08.856391+0000","last_up_change":"2024-05-09T06:35:53.778274+0000","last_in_change":"2024-05-09T06:35:44.780586+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T06:35:54.819999+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T06:36:04.931314+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T06:36:06.200433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fae3a8db-4826-4f3c-b33a-12c09bb5df6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":1897849827},{"type":"v1","addr":"172.21.15.3:6817","nonce":1897849827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":1897849827},{"type":"v1","addr":"172.21.15.3:6819","nonce":1897849827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":1897849827},{"type":"v1","addr":"172.21.15.3:6823","nonce":1897849827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":1897849827},{"type":"v1","addr":"172.21.15.3:6821","nonce":1897849827}]},"public_addr":"172.21.15.3:6817/1897849827","cluster_addr":"172.21.15.3:6819/1897849827","heartbeat_back_addr":"172.21.15.3:6823/1897849827","heartbeat_front_addr":"172.21.15.3:6821/1897849827","state":["exists","up"]},{"osd":1,"uuid":"646e882d-4ce4-4720-8c84-f71df563a35e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":926805498},{"type":"v1","addr":"172.21.15.3:6801","nonce":926805498}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":926805498},{"type":"v1","addr":"172.21.15.3:6803","nonce":926805498}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":926805498},{"type":"v1","addr":"172.21.15.3:6807","nonce":926805498}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":926805498},{"type":"v1","addr":"172.21.15.3:6805","nonce":926805498}]},"public_addr":"172.21.15.3:6801/926805498","cluster_addr":"172.21.15.3:6803/926805498","heartbeat_back_addr":"172.21.15.3:6807/926805498","heartbeat_front_addr":"172.21.15.3:6805/926805498","state":["exists","up"]},{"osd":2,"uuid":"1abbadb5-a570-4359-9643-acff3ede35a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":801295451},{"type":"v1","addr":"172.21.15.3:6809","nonce":801295451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":801295451},{"type":"v1","addr":"172.21.15.3:6811","nonce":801295451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":801295451},{"type":"v1","addr":"172.21.15.3:6815","nonce":801295451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":801295451},{"type":"v1","addr":"172.21.15.3:6813","nonce":801295451}]},"public_addr":"172.21.15.3:6809/801295451","cluster_addr":"172.21.15.3:6811/801295451","heartbeat_back_addr":"172.21.15.3:6815/801295451","heartbeat_front_addr":"172.21.15.3:6813/801295451","state":["exists","up"]},{"osd":3,"uuid":"62ac4dc1-2371-4dd2-acd3-e2dbb11aedf5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":1903389857},{"type":"v1","addr":"172.21.15.3:6825","nonce":1903389857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":1903389857},{"type":"v1","addr":"172.21.15.3:6827","nonce":1903389857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":1903389857},{"type":"v1","addr":"172.21.15.3:6831","nonce":1903389857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":1903389857},{"type":"v1","addr":"172.21.15.3:6829","nonce":1903389857}]},"public_addr":"172.21.15.3:6825/1903389857","cluster_addr":"172.21.15.3:6827/1903389857","heartbeat_back_addr":"172.21.15.3:6831/1903389857","heartbeat_front_addr":"172.21.15.3:6829/1903389857","state":["exists","up"]},{"osd":4,"uuid":"ae808398-8326-4d7c-9319-a2a0eafffb59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":2143624576},{"type":"v1","addr":"172.21.15.190:6801","nonce":2143624576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":2143624576},{"type":"v1","addr":"172.21.15.190:6803","nonce":2143624576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":2143624576},{"type":"v1","addr":"172.21.15.190:6807","nonce":2143624576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":2143624576},{"type":"v1","addr":"172.21.15.190:6805","nonce":2143624576}]},"public_addr":"172.21.15.190:6801/2143624576","cluster_addr":"172.21.15.190:6803/2143624576","heartbeat_back_addr":"172.21.15.190:6807/2143624576","heartbeat_front_addr":"172.21.15.190:6805/2143624576","state":["exists","up"]},{"osd":5,"uuid":"918adf50-c440-48ac-9a8d-4837562d7275","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":1766923126},{"type":"v1","addr":"172.21.15.190:6814","nonce":1766923126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":1766923126},{"type":"v1","addr":"172.21.15.190:6818","nonce":1766923126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":1766923126},{"type":"v1","addr":"172.21.15.190:6823","nonce":1766923126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":1766923126},{"type":"v1","addr":"172.21.15.190:6821","nonce":1766923126}]},"public_addr":"172.21.15.190:6814/1766923126","cluster_addr":"172.21.15.190:6818/1766923126","heartbeat_back_addr":"172.21.15.190:6823/1766923126","heartbeat_front_addr":"172.21.15.190:6821/1766923126","state":["exists","up"]},{"osd":6,"uuid":"ac2882bc-2a7c-463c-8f6c-76ff49828adc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":342414248},{"type":"v1","addr":"172.21.15.190:6825","nonce":342414248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":342414248},{"type":"v1","addr":"172.21.15.190:6827","nonce":342414248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":342414248},{"type":"v1","addr":"172.21.15.190:6831","nonce":342414248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":342414248},{"type":"v1","addr":"172.21.15.190:6829","nonce":342414248}]},"public_addr":"172.21.15.190:6825/342414248","cluster_addr":"172.21.15.190:6827/342414248","heartbeat_back_addr":"172.21.15.190:6831/342414248","heartbeat_front_addr":"172.21.15.190:6829/342414248","state":["exists","up"]},{"osd":7,"uuid":"dd4e767a-4730-4c70-8bd5-3f088140a5a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":537849891},{"type":"v1","addr":"172.21.15.190:6809","nonce":537849891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":537849891},{"type":"v1","addr":"172.21.15.190:6811","nonce":537849891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6817","nonce":537849891},{"type":"v1","addr":"172.21.15.190:6819","nonce":537849891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6813","nonce":537849891},{"type":"v1","addr":"172.21.15.190:6815","nonce":537849891}]},"public_addr":"172.21.15.190:6809/537849891","cluster_addr":"172.21.15.190:6811/537849891","heartbeat_back_addr":"172.21.15.190:6819/537849891","heartbeat_front_addr":"172.21.15.190:6815/537849891","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.889784+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.974002+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.973549+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.955396+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.928471+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.902418+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.958696+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.926474+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T06:36:33.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.721+0000 7fcdc3b7e640 1 -- 172.21.15.3:0/481729666 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fcda006be30 msgr2=0x7fcda006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:33.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.721+0000 7fcdc3b7e640 1 --2- 172.21.15.3:0/481729666 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fcda006be30 0x7fcda006e2f0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fcdac034000 tx=0x7fcdac02e000 comp rx=0 tx=0).stop 2024-05-09T06:36:33.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.721+0000 7fcdc3b7e640 1 -- 172.21.15.3:0/481729666 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcdbc06bcf0 msgr2=0x7fcdbc14ec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:33.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.721+0000 7fcdc3b7e640 1 --2- 172.21.15.3:0/481729666 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcdbc06bcf0 0x7fcdbc14ec70 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fcdb800e960 tx=0x7fcdb800ee20 comp rx=0 tx=0).stop 2024-05-09T06:36:33.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.721+0000 7fcdc3b7e640 1 -- 172.21.15.3:0/481729666 shutdown_connections 2024-05-09T06:36:33.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.721+0000 7fcdc3b7e640 1 --2- 172.21.15.3:0/481729666 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fcda006be30 0x7fcda006e2f0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:33.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.721+0000 7fcdc3b7e640 1 --2- 172.21.15.3:0/481729666 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcdbc15e100 0x7fcdbc0ce920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:33.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.721+0000 7fcdc3b7e640 1 --2- 172.21.15.3:0/481729666 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcdbc06d8f0 0x7fcdbc0ce3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:33.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.721+0000 7fcdc3b7e640 1 --2- 172.21.15.3:0/481729666 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcdbc06bcf0 0x7fcdbc14ec70 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:33.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.721+0000 7fcdc3b7e640 1 -- 172.21.15.3:0/481729666 >> 172.21.15.3:0/481729666 conn(0x7fcdbc0d05d0 msgr2=0x7fcdbc0b0470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:33.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.721+0000 7fcdc3b7e640 1 -- 172.21.15.3:0/481729666 shutdown_connections 2024-05-09T06:36:33.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.721+0000 7fcdc3b7e640 1 -- 172.21.15.3:0/481729666 wait complete. 2024-05-09T06:36:33.712 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-09T06:35:54.819999+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-09T06:36:04.931314+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-09T06:36:06.200433+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-09T06:36:33.712 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-09T06:36:33.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.840+0000 7fa0a99c1640 1 Processor -- start 2024-05-09T06:36:33.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.840+0000 7fa0a99c1640 1 -- start start 2024-05-09T06:36:33.821 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.841+0000 7fa0a99c1640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa0a4144910 0x7fa0a40cf8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:33.821 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.841+0000 7fa0a99c1640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa0a4072970 0x7fa0a4071080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:33.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.841+0000 7fa0a99c1640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa0a4144cf0 con 0x7fa0a40b4130 2024-05-09T06:36:33.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.841+0000 7fa0a99c1640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fa0a40724e0 con 0x7fa0a4072970 2024-05-09T06:36:33.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.841+0000 7fa0a99c1640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fa0a40718d0 con 0x7fa0a4144910 2024-05-09T06:36:33.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.841+0000 7fa0a27fc640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa0a4072970 0x7fa0a4071080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:33.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.841+0000 7fa0a27fc640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa0a4072970 0x7fa0a4071080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:35488/0 (socket says 172.21.15.3:35488) 2024-05-09T06:36:33.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.841+0000 7fa0a27fc640 1 -- 172.21.15.3:0/3271352859 learned_addr learned my addr 172.21.15.3:0/3271352859 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:36:33.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.841+0000 7fa0a37fe640 1 --1- 172.21.15.3:0/3271352859 >> v1:172.21.15.3:6789/0 conn(0x7fa0a40b4130 0x7fa0a406e250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:58950/0 (socket says 172.21.15.3:58950) 2024-05-09T06:36:33.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.841+0000 7fa0a2ffd640 1 --2- 172.21.15.3:0/3271352859 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa0a4144910 0x7fa0a40cf8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:33.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.841+0000 7fa0a27fc640 1 -- 172.21.15.3:0/3271352859 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa0a4144910 msgr2=0x7fa0a40cf8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:33.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.841+0000 7fa0a27fc640 1 --2- 172.21.15.3:0/3271352859 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa0a4144910 0x7fa0a40cf8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:33.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.841+0000 7fa0a27fc640 1 -- 172.21.15.3:0/3271352859 >> v1:172.21.15.3:6789/0 conn(0x7fa0a40b4130 legacy=0x7fa0a406e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:33.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.841+0000 7fa0a27fc640 1 -- 172.21.15.3:0/3271352859 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0a4071bb0 con 0x7fa0a4072970 2024-05-09T06:36:33.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.842+0000 7fa0a27fc640 1 --2- 172.21.15.3:0/3271352859 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa0a4072970 0x7fa0a4071080 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fa08c009870 tx=0x7fa08c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc5443838eebe51f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:33.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.842+0000 7fa0a1ffb640 1 -- 172.21.15.3:0/3271352859 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa08c037070 con 0x7fa0a4072970 2024-05-09T06:36:33.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.842+0000 7fa0a1ffb640 1 -- 172.21.15.3:0/3271352859 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa08c032440 con 0x7fa0a4072970 2024-05-09T06:36:33.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.842+0000 7fa0a1ffb640 1 -- 172.21.15.3:0/3271352859 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa08c032810 con 0x7fa0a4072970 2024-05-09T06:36:33.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.843+0000 7fa0a99c1640 1 -- 172.21.15.3:0/3271352859 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa0a4072970 msgr2=0x7fa0a4071080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:33.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.843+0000 7fa0a99c1640 1 --2- 172.21.15.3:0/3271352859 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa0a4072970 0x7fa0a4071080 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fa08c009870 tx=0x7fa08c02a270 comp rx=0 tx=0).stop 2024-05-09T06:36:33.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.843+0000 7fa0a99c1640 1 -- 172.21.15.3:0/3271352859 shutdown_connections 2024-05-09T06:36:33.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.843+0000 7fa0a99c1640 1 --2- 172.21.15.3:0/3271352859 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa0a4072970 0x7fa0a4071080 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:33.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.843+0000 7fa0a99c1640 1 --2- 172.21.15.3:0/3271352859 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa0a4144910 0x7fa0a40cf8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:33.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.843+0000 7fa0a99c1640 1 -- 172.21.15.3:0/3271352859 >> 172.21.15.3:0/3271352859 conn(0x7fa0a40b1c20 msgr2=0x7fa0a40b2030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:33.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.843+0000 7fa0a99c1640 1 -- 172.21.15.3:0/3271352859 shutdown_connections 2024-05-09T06:36:33.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.843+0000 7fa0a99c1640 1 -- 172.21.15.3:0/3271352859 wait complete. 2024-05-09T06:36:33.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.844+0000 7fa0a99c1640 1 Processor -- start 2024-05-09T06:36:33.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.844+0000 7fa0a99c1640 1 -- start start 2024-05-09T06:36:33.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.845+0000 7fa0a99c1640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa0a4072970 0x7fa0a4151880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:33.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.845+0000 7fa0a99c1640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa0a40b4130 0x7fa0a414fe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:33.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.845+0000 7fa0a99c1640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa0a4144910 0x7fa0a4140db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:33.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.845+0000 7fa0a99c1640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fa0a40715c0 con 0x7fa0a4144910 2024-05-09T06:36:33.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.845+0000 7fa0a99c1640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fa0a4072170 con 0x7fa0a40b4130 2024-05-09T06:36:33.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.845+0000 7fa0a99c1640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fa0a40b45c0 con 0x7fa0a4072970 2024-05-09T06:36:33.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.845+0000 7fa0a37fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa0a4144910 0x7fa0a4140db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:33.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.845+0000 7fa0a37fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa0a4144910 0x7fa0a4140db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:35494/0 (socket says 172.21.15.3:35494) 2024-05-09T06:36:33.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.845+0000 7fa0a37fe640 1 -- 172.21.15.3:0/1608681826 learned_addr learned my addr 172.21.15.3:0/1608681826 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:36:33.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.845+0000 7fa0a2ffd640 1 --2- 172.21.15.3:0/1608681826 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa0a4072970 0x7fa0a4151880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:33.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.845+0000 7fa0a27fc640 1 --2- 172.21.15.3:0/1608681826 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa0a40b4130 0x7fa0a414fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:33.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.845+0000 7fa0a37fe640 1 -- 172.21.15.3:0/1608681826 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa0a4072970 msgr2=0x7fa0a4151880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:33.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.845+0000 7fa0a37fe640 1 --2- 172.21.15.3:0/1608681826 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa0a4072970 0x7fa0a4151880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:33.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.845+0000 7fa0a37fe640 1 -- 172.21.15.3:0/1608681826 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa0a40b4130 msgr2=0x7fa0a414fe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:33.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.845+0000 7fa0a37fe640 1 --2- 172.21.15.3:0/1608681826 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa0a40b4130 0x7fa0a414fe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:33.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.845+0000 7fa0a37fe640 1 -- 172.21.15.3:0/1608681826 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0a4151dc0 con 0x7fa0a4144910 2024-05-09T06:36:33.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.846+0000 7fa0a37fe640 1 --2- 172.21.15.3:0/1608681826 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa0a4144910 0x7fa0a4140db0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fa09400c780 tx=0x7fa09400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:33.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.846+0000 7fa09bfff640 1 -- 172.21.15.3:0/1608681826 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa094017070 con 0x7fa0a4144910 2024-05-09T06:36:33.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.846+0000 7fa09bfff640 1 -- 172.21.15.3:0/1608681826 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa09400a640 con 0x7fa0a4144910 2024-05-09T06:36:33.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.846+0000 7fa0a99c1640 1 -- 172.21.15.3:0/1608681826 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0a41503d0 con 0x7fa0a4144910 2024-05-09T06:36:33.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.846+0000 7fa09bfff640 1 -- 172.21.15.3:0/1608681826 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa09400aa30 con 0x7fa0a4144910 2024-05-09T06:36:33.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.846+0000 7fa0a99c1640 1 -- 172.21.15.3:0/1608681826 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0a4150ac0 con 0x7fa0a4144910 2024-05-09T06:36:33.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.847+0000 7fa0a99c1640 1 -- 172.21.15.3:0/1608681826 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0a4069550 con 0x7fa0a4144910 2024-05-09T06:36:33.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.847+0000 7fa09bfff640 1 -- 172.21.15.3:0/1608681826 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa09401a070 con 0x7fa0a4144910 2024-05-09T06:36:33.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.848+0000 7fa09bfff640 1 --2- 172.21.15.3:0/1608681826 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fa08406be30 0x7fa08406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:33.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.848+0000 7fa09bfff640 1 -- 172.21.15.3:0/1608681826 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fa09408a380 con 0x7fa0a4144910 2024-05-09T06:36:33.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.848+0000 7fa0a2ffd640 1 --2- 172.21.15.3:0/1608681826 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fa08406be30 0x7fa08406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:33.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.848+0000 7fa0a2ffd640 1 --2- 172.21.15.3:0/1608681826 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fa08406be30 0x7fa08406e2f0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fa09c0079a0 tx=0x7fa09c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:33.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:33.852+0000 7fa09bfff640 1 -- 172.21.15.3:0/1608681826 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fa0940586d0 con 0x7fa0a4144910 2024-05-09T06:36:34.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:34.035+0000 7fa0a99c1640 1 -- 172.21.15.3:0/1608681826 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fa0a40b3de0 con 0x7fa0a4144910 2024-05-09T06:36:34.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:34.035+0000 7fa09bfff640 1 -- 172.21.15.3:0/1608681826 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7fa09405c9d0 con 0x7fa0a4144910 2024-05-09T06:36:34.015 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 1 2024-05-09T06:36:34.017 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:34.037+0000 7fa0a99c1640 1 -- 172.21.15.3:0/1608681826 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fa08406be30 msgr2=0x7fa08406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:34.017 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:34.037+0000 7fa0a99c1640 1 --2- 172.21.15.3:0/1608681826 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fa08406be30 0x7fa08406e2f0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fa09c0079a0 tx=0x7fa09c008040 comp rx=0 tx=0).stop 2024-05-09T06:36:34.017 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:34.037+0000 7fa0a99c1640 1 -- 172.21.15.3:0/1608681826 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa0a4144910 msgr2=0x7fa0a4140db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:34.017 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:34.037+0000 7fa0a99c1640 1 --2- 172.21.15.3:0/1608681826 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa0a4144910 0x7fa0a4140db0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fa09400c780 tx=0x7fa09400cc40 comp rx=0 tx=0).stop 2024-05-09T06:36:34.017 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:34.037+0000 7fa0a99c1640 1 -- 172.21.15.3:0/1608681826 shutdown_connections 2024-05-09T06:36:34.018 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:34.037+0000 7fa0a99c1640 1 --2- 172.21.15.3:0/1608681826 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fa08406be30 0x7fa08406e2f0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:34.018 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:34.037+0000 7fa0a99c1640 1 --2- 172.21.15.3:0/1608681826 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa0a4144910 0x7fa0a4140db0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:34.018 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:34.037+0000 7fa0a99c1640 1 --2- 172.21.15.3:0/1608681826 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa0a40b4130 0x7fa0a414fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:34.018 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:34.037+0000 7fa0a99c1640 1 --2- 172.21.15.3:0/1608681826 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa0a4072970 0x7fa0a4151880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:34.018 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:34.037+0000 7fa0a99c1640 1 -- 172.21.15.3:0/1608681826 >> 172.21.15.3:0/1608681826 conn(0x7fa0a40b1c20 msgr2=0x7fa0a406d240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:34.018 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:34.037+0000 7fa0a99c1640 1 -- 172.21.15.3:0/1608681826 shutdown_connections 2024-05-09T06:36:34.018 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:34.037+0000 7fa0a99c1640 1 -- 172.21.15.3:0/1608681826 wait complete. 2024-05-09T06:36:34.028 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-09T06:36:34.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:34.155+0000 7f42bb361640 1 Processor -- start 2024-05-09T06:36:34.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:34.155+0000 7f42bb361640 1 -- start start 2024-05-09T06:36:34.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:34.156+0000 7f42bb361640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f42b4072970 0x7f42b40cf8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:34.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:34.156+0000 7f42bb361640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f42b4144cf0 con 0x7f42b40b4130 2024-05-09T06:36:34.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:34.156+0000 7f42bb361640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f42b40718d0 con 0x7f42b4144910 2024-05-09T06:36:34.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:34.156+0000 7f42bb361640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f42b40724e0 con 0x7f42b4072970 2024-05-09T06:36:34.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:34.156+0000 7f42b88fc640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f42b4072970 0x7f42b40cf8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:34.137 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:34.156+0000 7f42b98fe640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f42b40b4130 0x7f42b415b5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:58952/0 (socket says 172.21.15.3:58952) 2024-05-09T06:36:34.137 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:34.156+0000 7f42b88fc640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f42b4072970 0x7f42b40cf8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:35498/0 (socket says 172.21.15.3:35498) 2024-05-09T06:36:34.137 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:34.156+0000 7f42b98fe640 1 -- 172.21.15.3:0/1015677708 learned_addr learned my addr 172.21.15.3:0/1015677708 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-05-09T06:36:34.137 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:34.157+0000 7f42b88fc640 1 -- 172.21.15.3:0/1015677708 >> v1:172.21.15.190:6789/0 conn(0x7f42b4144910 legacy=0x7f42b4071080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:34.137 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:34.157+0000 7f42b88fc640 1 -- 172.21.15.3:0/1015677708 >> v1:172.21.15.3:6789/0 conn(0x7f42b40b4130 legacy=0x7f42b415b5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:34.137 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:34.157+0000 7f42b88fc640 1 -- 172.21.15.3:0/1015677708 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42b4071bb0 con 0x7f42b4072970 2024-05-09T06:36:34.137 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:34.157+0000 7f42b3fff640 1 -- 172.21.15.3:0/1015677708 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 939377482 0 0) 0x7f42b4144cf0 con 0x7f42b40b4130 2024-05-09T06:36:34.138 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:34.157+0000 7f42b88fc640 1 --2- 172.21.15.3:0/1015677708 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f42b4072970 0x7f42b40cf8c0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f42a80097a0 tx=0x7f42a802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fb2b1931a883e7e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:34.138 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:34.157+0000 7f42b3fff640 1 -- 172.21.15.3:0/1015677708 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42a8037070 con 0x7f42b4072970 2024-05-09T06:36:34.138 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:34.157+0000 7f42b3fff640 1 -- 172.21.15.3:0/1015677708 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f42a8032440 con 0x7f42b4072970 2024-05-09T06:36:34.138 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:34.157+0000 7f42b3fff640 1 -- 172.21.15.3:0/1015677708 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42a8032810 con 0x7f42b4072970 2024-05-09T06:36:34.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:34.158+0000 7f42bb361640 1 -- 172.21.15.3:0/1015677708 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f42b4072970 msgr2=0x7f42b40cf8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:34.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:34.158+0000 7f42bb361640 1 --2- 172.21.15.3:0/1015677708 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f42b4072970 0x7f42b40cf8c0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f42a80097a0 tx=0x7f42a802a1e0 comp rx=0 tx=0).stop 2024-05-09T06:36:34.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:34.158+0000 7f42bb361640 1 -- 172.21.15.3:0/1015677708 shutdown_connections 2024-05-09T06:36:34.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:34.158+0000 7f42bb361640 1 --2- 172.21.15.3:0/1015677708 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f42b4072970 0x7f42b40cf8c0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:34.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:34.158+0000 7f42bb361640 1 -- 172.21.15.3:0/1015677708 >> 172.21.15.3:0/1015677708 conn(0x7f42b40b1c20 msgr2=0x7f42b40b2030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:34.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:34.158+0000 7f42bb361640 1 -- 172.21.15.3:0/1015677708 shutdown_connections 2024-05-09T06:36:34.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:34.158+0000 7f42bb361640 1 -- 172.21.15.3:0/1015677708 wait complete. 2024-05-09T06:36:34.141 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:34.159+0000 7f42bb361640 1 Processor -- start 2024-05-09T06:36:34.141 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:34.159+0000 7f42bb361640 1 -- start start 2024-05-09T06:36:34.141 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:34.160+0000 7f42bb361640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f42b4072970 0x7f42b415a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:34.142 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:34.160+0000 7f42bb361640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f42b40b4130 0x7f42b40ad560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:34.142 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:34.160+0000 7f42bb361640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f42b4144910 0x7f42b40c4e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:34.142 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:34.160+0000 7f42bb361640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f42b40b4560 con 0x7f42b4072970 2024-05-09T06:36:34.142 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:34.160+0000 7f42bb361640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f42b4072170 con 0x7f42b4144910 2024-05-09T06:36:34.142 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:34.160+0000 7f42bb361640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f42b40b46e0 con 0x7f42b40b4130 2024-05-09T06:36:34.142 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:34.160+0000 7f42b90fd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f42b4072970 0x7f42b415a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:34.142 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:34.160+0000 7f42b90fd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f42b4072970 0x7f42b415a6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:35506/0 (socket says 172.21.15.3:35506) 2024-05-09T06:36:34.142 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:34.160+0000 7f42b90fd640 1 -- 172.21.15.3:0/2434264129 learned_addr learned my addr 172.21.15.3:0/2434264129 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:36:34.142 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:34.160+0000 7f42b98fe640 1 --2- 172.21.15.3:0/2434264129 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f42b4144910 0x7f42b40c4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:34.142 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:34.160+0000 7f42b90fd640 1 -- 172.21.15.3:0/2434264129 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f42b40b4130 msgr2=0x7f42b40ad560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:34.142 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:34.160+0000 7f42b88fc640 1 --2- 172.21.15.3:0/2434264129 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f42b40b4130 0x7f42b40ad560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:34.142 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:34.160+0000 7f42b90fd640 1 --2- 172.21.15.3:0/2434264129 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f42b40b4130 0x7f42b40ad560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:34.142 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:34.161+0000 7f42b90fd640 1 -- 172.21.15.3:0/2434264129 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f42b4144910 msgr2=0x7f42b40c4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:34.142 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:34.161+0000 7f42b90fd640 1 --2- 172.21.15.3:0/2434264129 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f42b4144910 0x7f42b40c4e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:34.142 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:34.161+0000 7f42b90fd640 1 -- 172.21.15.3:0/2434264129 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42b40adf20 con 0x7f42b4072970 2024-05-09T06:36:34.143 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:34.161+0000 7f42b98fe640 1 --2- 172.21.15.3:0/2434264129 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f42b4144910 0x7f42b40c4e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:36:34.143 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:34.161+0000 7f42b88fc640 1 --2- 172.21.15.3:0/2434264129 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f42b40b4130 0x7f42b40ad560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:36:34.143 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:34.161+0000 7f42b90fd640 1 --2- 172.21.15.3:0/2434264129 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f42b4072970 0x7f42b415a6c0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f42ac00b3f0 tx=0x7f42ac00b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:34.143 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:34.161+0000 7f42b1ffb640 1 -- 172.21.15.3:0/2434264129 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42ac015020 con 0x7f42b4072970 2024-05-09T06:36:34.143 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:34.161+0000 7f42b1ffb640 1 -- 172.21.15.3:0/2434264129 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f42ac011830 con 0x7f42b4072970 2024-05-09T06:36:34.143 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:34.161+0000 7f42bb361640 1 -- 172.21.15.3:0/2434264129 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42b40ae200 con 0x7f42b4072970 2024-05-09T06:36:34.143 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:34.161+0000 7f42b1ffb640 1 -- 172.21.15.3:0/2434264129 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42ac011c20 con 0x7f42b4072970 2024-05-09T06:36:34.143 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:34.161+0000 7f42bb361640 1 -- 172.21.15.3:0/2434264129 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42b40ae7c0 con 0x7f42b4072970 2024-05-09T06:36:34.144 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:34.162+0000 7f42bb361640 1 -- 172.21.15.3:0/2434264129 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42b4069550 con 0x7f42b4072970 2024-05-09T06:36:34.144 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:34.163+0000 7f42b1ffb640 1 -- 172.21.15.3:0/2434264129 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f42ac018020 con 0x7f42b4072970 2024-05-09T06:36:34.144 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:34.163+0000 7f42b1ffb640 1 --2- 172.21.15.3:0/2434264129 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f42a006be30 0x7f42a006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:34.144 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:34.163+0000 7f42b1ffb640 1 -- 172.21.15.3:0/2434264129 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f42ac08aae0 con 0x7f42b4072970 2024-05-09T06:36:34.144 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:34.163+0000 7f42b88fc640 1 --2- 172.21.15.3:0/2434264129 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f42a006be30 0x7f42a006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:34.144 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:34.164+0000 7f42b88fc640 1 --2- 172.21.15.3:0/2434264129 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f42a006be30 0x7f42a006e2f0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f42a8034000 tx=0x7f42a802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:34.147 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:34.168+0000 7f42b1ffb640 1 -- 172.21.15.3:0/2434264129 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f42ac01f030 con 0x7f42b4072970 2024-05-09T06:36:34.336 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:34.356+0000 7f42bb361640 1 -- 172.21.15.3:0/2434264129 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f42b4142730 con 0x7f42b4072970 2024-05-09T06:36:34.336 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:34.356+0000 7f42b1ffb640 1 -- 172.21.15.3:0/2434264129 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7f42ac059490 con 0x7f42b4072970 2024-05-09T06:36:34.336 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 32 2024-05-09T06:36:34.338 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:34.358+0000 7f42bb361640 1 -- 172.21.15.3:0/2434264129 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f42a006be30 msgr2=0x7f42a006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:34.338 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:34.358+0000 7f42bb361640 1 --2- 172.21.15.3:0/2434264129 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f42a006be30 0x7f42a006e2f0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f42a8034000 tx=0x7f42a802e000 comp rx=0 tx=0).stop 2024-05-09T06:36:34.338 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:34.358+0000 7f42bb361640 1 -- 172.21.15.3:0/2434264129 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f42b4072970 msgr2=0x7f42b415a6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:34.338 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:34.358+0000 7f42bb361640 1 --2- 172.21.15.3:0/2434264129 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f42b4072970 0x7f42b415a6c0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f42ac00b3f0 tx=0x7f42ac00b8b0 comp rx=0 tx=0).stop 2024-05-09T06:36:34.338 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:34.358+0000 7f42bb361640 1 -- 172.21.15.3:0/2434264129 shutdown_connections 2024-05-09T06:36:34.338 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:34.358+0000 7f42bb361640 1 --2- 172.21.15.3:0/2434264129 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f42a006be30 0x7f42a006e2f0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:34.339 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:34.358+0000 7f42bb361640 1 --2- 172.21.15.3:0/2434264129 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f42b4144910 0x7f42b40c4e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:34.339 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:34.358+0000 7f42bb361640 1 --2- 172.21.15.3:0/2434264129 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f42b40b4130 0x7f42b40ad560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:34.339 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:34.358+0000 7f42bb361640 1 --2- 172.21.15.3:0/2434264129 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f42b4072970 0x7f42b415a6c0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:34.339 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:34.358+0000 7f42bb361640 1 -- 172.21.15.3:0/2434264129 >> 172.21.15.3:0/2434264129 conn(0x7f42b40b1c20 msgr2=0x7f42b40b3020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:34.339 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:34.358+0000 7f42bb361640 1 -- 172.21.15.3:0/2434264129 shutdown_connections 2024-05-09T06:36:34.339 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:34.358+0000 7f42bb361640 1 -- 172.21.15.3:0/2434264129 wait complete. 2024-05-09T06:36:34.349 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-09T06:36:34.456 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:34.476+0000 7f53c7d42640 1 Processor -- start 2024-05-09T06:36:34.456 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:34.476+0000 7f53c7d42640 1 -- start start 2024-05-09T06:36:34.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:34.477+0000 7f53c7d42640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f53c015e490 0x7f53c0162590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:34.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:34.477+0000 7f53c7d42640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f53c006b250 0x7f53c006b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:34.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:34.477+0000 7f53c7d42640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f53c006cad0 con 0x7f53c015e0b0 2024-05-09T06:36:34.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:34.477+0000 7f53c7d42640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f53c00b3eb0 con 0x7f53c015e490 2024-05-09T06:36:34.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:34.477+0000 7f53c7d42640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f53c00b4190 con 0x7f53c006b250 2024-05-09T06:36:34.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:34.477+0000 7f53c52dd640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f53c015e490 0x7f53c0162590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:34.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:34.477+0000 7f53c52dd640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f53c015e490 0x7f53c0162590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:35516/0 (socket says 172.21.15.3:35516) 2024-05-09T06:36:34.458 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:34.477+0000 7f53c52dd640 1 -- 172.21.15.3:0/114596253 learned_addr learned my addr 172.21.15.3:0/114596253 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:36:34.458 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:34.477+0000 7f53c62df640 1 --2- 172.21.15.3:0/114596253 >> v2:172.21.15.190:3300/0 conn(0x7f53c006b250 0x7f53c006b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:34.458 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:34.477+0000 7f53c52dd640 1 -- 172.21.15.3:0/114596253 >> v2:172.21.15.190:3300/0 conn(0x7f53c006b250 msgr2=0x7f53c006b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:34.458 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:34.477+0000 7f53c52dd640 1 --2- 172.21.15.3:0/114596253 >> v2:172.21.15.190:3300/0 conn(0x7f53c006b250 0x7f53c006b6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:34.458 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:34.477+0000 7f53c52dd640 1 -- 172.21.15.3:0/114596253 >> v1:172.21.15.190:6789/0 conn(0x7f53c015e0b0 legacy=0x7f53c0161e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:34.458 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:34.477+0000 7f53c52dd640 1 -- 172.21.15.3:0/114596253 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53c00b4470 con 0x7f53c015e490 2024-05-09T06:36:34.458 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:34.477+0000 7f53c62df640 1 --2- 172.21.15.3:0/114596253 >> v2:172.21.15.190:3300/0 conn(0x7f53c006b250 0x7f53c006b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:36:34.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:34.478+0000 7f53c52dd640 1 --2- 172.21.15.3:0/114596253 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f53c015e490 0x7f53c0162590 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f53c00aed50 tx=0x7f53b002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99d2aa834428fdd7 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:34.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:34.478+0000 7f53c4adc640 1 -- 172.21.15.3:0/114596253 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53b0037070 con 0x7f53c015e490 2024-05-09T06:36:34.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:34.478+0000 7f53c4adc640 1 -- 172.21.15.3:0/114596253 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f53b0032440 con 0x7f53c015e490 2024-05-09T06:36:34.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:34.478+0000 7f53c4adc640 1 -- 172.21.15.3:0/114596253 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53b0032810 con 0x7f53c015e490 2024-05-09T06:36:34.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:34.479+0000 7f53c7d42640 1 -- 172.21.15.3:0/114596253 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f53c015e490 msgr2=0x7f53c0162590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:34.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:34.479+0000 7f53c7d42640 1 --2- 172.21.15.3:0/114596253 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f53c015e490 0x7f53c0162590 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f53c00aed50 tx=0x7f53b002a230 comp rx=0 tx=0).stop 2024-05-09T06:36:34.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:34.479+0000 7f53c7d42640 1 -- 172.21.15.3:0/114596253 shutdown_connections 2024-05-09T06:36:34.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:34.479+0000 7f53c7d42640 1 --2- 172.21.15.3:0/114596253 >> v2:172.21.15.190:3300/0 conn(0x7f53c006b250 0x7f53c006b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:34.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:34.479+0000 7f53c7d42640 1 --2- 172.21.15.3:0/114596253 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f53c015e490 0x7f53c0162590 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:34.460 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:34.479+0000 7f53c7d42640 1 -- 172.21.15.3:0/114596253 >> 172.21.15.3:0/114596253 conn(0x7f53c00d07b0 msgr2=0x7f53c00d0bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:34.460 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:34.479+0000 7f53c7d42640 1 -- 172.21.15.3:0/114596253 shutdown_connections 2024-05-09T06:36:34.460 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:34.479+0000 7f53c7d42640 1 -- 172.21.15.3:0/114596253 wait complete. 2024-05-09T06:36:34.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:34.480+0000 7f53c7d42640 1 Processor -- start 2024-05-09T06:36:34.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:34.480+0000 7f53c7d42640 1 -- start start 2024-05-09T06:36:34.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:34.481+0000 7f53c7d42640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f53c006b250 0x7f53c015e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:34.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:34.481+0000 7f53c7d42640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f53c015e0b0 0x7f53c015ff90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:34.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:34.481+0000 7f53c7d42640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f53c015e490 0x7f53c0143090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:34.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:34.481+0000 7f53c7d42640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f53c006da70 con 0x7f53c015e490 2024-05-09T06:36:34.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:34.481+0000 7f53c7d42640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f53c006d8f0 con 0x7f53c006b250 2024-05-09T06:36:34.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:34.481+0000 7f53c7d42640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f53c0162ad0 con 0x7f53c015e0b0 2024-05-09T06:36:34.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:34.481+0000 7f53c62df640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f53c015e490 0x7f53c0143090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:34.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:34.481+0000 7f53c62df640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f53c015e490 0x7f53c0143090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:35520/0 (socket says 172.21.15.3:35520) 2024-05-09T06:36:34.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:34.481+0000 7f53c62df640 1 -- 172.21.15.3:0/3585985638 learned_addr learned my addr 172.21.15.3:0/3585985638 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:36:34.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:34.481+0000 7f53c5ade640 1 --2- 172.21.15.3:0/3585985638 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f53c006b250 0x7f53c015e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:34.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:34.481+0000 7f53c52dd640 1 --2- 172.21.15.3:0/3585985638 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f53c015e0b0 0x7f53c015ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:34.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:34.481+0000 7f53c62df640 1 -- 172.21.15.3:0/3585985638 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f53c015e0b0 msgr2=0x7f53c015ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:34.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:34.481+0000 7f53c62df640 1 --2- 172.21.15.3:0/3585985638 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f53c015e0b0 0x7f53c015ff90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:34.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:34.481+0000 7f53c62df640 1 -- 172.21.15.3:0/3585985638 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f53c006b250 msgr2=0x7f53c015e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:34.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:34.481+0000 7f53c62df640 1 --2- 172.21.15.3:0/3585985638 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f53c006b250 0x7f53c015e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:34.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:34.481+0000 7f53c62df640 1 -- 172.21.15.3:0/3585985638 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53c015cd10 con 0x7f53c015e490 2024-05-09T06:36:34.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:34.481+0000 7f53c5ade640 1 --2- 172.21.15.3:0/3585985638 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f53c006b250 0x7f53c015e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:36:34.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:34.482+0000 7f53c62df640 1 --2- 172.21.15.3:0/3585985638 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f53c015e490 0x7f53c0143090 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f53bc00cf20 tx=0x7f53bc00a7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:34.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:34.482+0000 7f53b67fc640 1 -- 172.21.15.3:0/3585985638 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53bc01a070 con 0x7f53c015e490 2024-05-09T06:36:34.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:34.482+0000 7f53b67fc640 1 -- 172.21.15.3:0/3585985638 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f53bc00d840 con 0x7f53c015e490 2024-05-09T06:36:34.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:34.482+0000 7f53c7d42640 1 -- 172.21.15.3:0/3585985638 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53c015cff0 con 0x7f53c015e490 2024-05-09T06:36:34.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:34.482+0000 7f53b67fc640 1 -- 172.21.15.3:0/3585985638 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53bc00dc30 con 0x7f53c015e490 2024-05-09T06:36:34.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:34.482+0000 7f53c7d42640 1 -- 172.21.15.3:0/3585985638 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53c015d5b0 con 0x7f53c015e490 2024-05-09T06:36:34.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:34.483+0000 7f53c7d42640 1 -- 172.21.15.3:0/3585985638 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53c0069550 con 0x7f53c015e490 2024-05-09T06:36:34.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:34.483+0000 7f53b67fc640 1 -- 172.21.15.3:0/3585985638 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f53bc01d070 con 0x7f53c015e490 2024-05-09T06:36:34.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:34.484+0000 7f53b67fc640 1 --2- 172.21.15.3:0/3585985638 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f53a406be60 0x7f53a406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:34.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:34.484+0000 7f53b67fc640 1 -- 172.21.15.3:0/3585985638 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f53bc08b6d0 con 0x7f53c015e490 2024-05-09T06:36:34.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:34.484+0000 7f53c5ade640 1 --2- 172.21.15.3:0/3585985638 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f53a406be60 0x7f53a406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:34.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:34.484+0000 7f53c5ade640 1 --2- 172.21.15.3:0/3585985638 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f53a406be60 0x7f53a406e320 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f53b8003eb0 tx=0x7f53b8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:34.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:34.488+0000 7f53b67fc640 1 -- 172.21.15.3:0/3585985638 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f53bc05a080 con 0x7f53c015e490 2024-05-09T06:36:34.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:34.670+0000 7f53c7d42640 1 -- 172.21.15.3:0/3585985638 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f53c0152090 con 0x7f53c015e490 2024-05-09T06:36:34.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:34.670+0000 7f53b67fc640 1 -- 172.21.15.3:0/3585985638 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7f53bc059980 con 0x7f53c015e490 2024-05-09T06:36:34.651 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 64 2024-05-09T06:36:34.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:34.672+0000 7f53c7d42640 1 -- 172.21.15.3:0/3585985638 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f53a406be60 msgr2=0x7f53a406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:34.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:34.672+0000 7f53c7d42640 1 --2- 172.21.15.3:0/3585985638 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f53a406be60 0x7f53a406e320 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f53b8003eb0 tx=0x7f53b8009040 comp rx=0 tx=0).stop 2024-05-09T06:36:34.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:34.672+0000 7f53c7d42640 1 -- 172.21.15.3:0/3585985638 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f53c015e490 msgr2=0x7f53c0143090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:34.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:34.672+0000 7f53c7d42640 1 --2- 172.21.15.3:0/3585985638 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f53c015e490 0x7f53c0143090 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f53bc00cf20 tx=0x7f53bc00a7d0 comp rx=0 tx=0).stop 2024-05-09T06:36:34.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:34.672+0000 7f53c7d42640 1 -- 172.21.15.3:0/3585985638 shutdown_connections 2024-05-09T06:36:34.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:34.672+0000 7f53c7d42640 1 --2- 172.21.15.3:0/3585985638 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f53a406be60 0x7f53a406e320 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:34.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:34.672+0000 7f53c7d42640 1 --2- 172.21.15.3:0/3585985638 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f53c015e490 0x7f53c0143090 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:34.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:34.672+0000 7f53c7d42640 1 --2- 172.21.15.3:0/3585985638 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f53c015e0b0 0x7f53c015ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:34.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:34.672+0000 7f53c7d42640 1 --2- 172.21.15.3:0/3585985638 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f53c006b250 0x7f53c015e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:34.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:34.672+0000 7f53c7d42640 1 -- 172.21.15.3:0/3585985638 >> 172.21.15.3:0/3585985638 conn(0x7f53c00d07b0 msgr2=0x7f53c006f580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:34.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:34.672+0000 7f53c7d42640 1 -- 172.21.15.3:0/3585985638 shutdown_connections 2024-05-09T06:36:34.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:34.672+0000 7f53c7d42640 1 -- 172.21.15.3:0/3585985638 wait complete. 2024-05-09T06:36:34.934 INFO:tasks.cephfs_test_runner:Starting test: test_volumes_module_finisher_thread (tasks.cephfs.test_volumes.TestPerModuleFinsherThread) 2024-05-09T06:36:34.935 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T06:36:35.040 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:35.060+0000 7f20efefc640 1 Processor -- start 2024-05-09T06:36:35.041 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:35.061+0000 7f20efefc640 1 -- start start 2024-05-09T06:36:35.043 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:35.062+0000 7f20efefc640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f20e815e100 0x7f20e80ae2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:35.043 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:35.062+0000 7f20efefc640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f20e806d8f0 0x7f20e80ae7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:35.043 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:35.062+0000 7f20efefc640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f20e806c830 con 0x7f20e806bcf0 2024-05-09T06:36:35.043 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:35.062+0000 7f20efefc640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f20e80b3ed0 con 0x7f20e815e100 2024-05-09T06:36:35.043 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:35.062+0000 7f20efefc640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f20e80b4770 con 0x7f20e806d8f0 2024-05-09T06:36:35.043 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:35.062+0000 7f20edc98640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f20e815e100 0x7f20e80ae2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:35.043 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:35.062+0000 7f20ee499640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f20e806bcf0 0x7f20e806c120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:58956/0 (socket says 172.21.15.3:58956) 2024-05-09T06:36:35.043 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:35.062+0000 7f20ee499640 1 -- 172.21.15.3:0/3335928825 learned_addr learned my addr 172.21.15.3:0/3335928825 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-05-09T06:36:35.043 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:35.062+0000 7f20ed497640 1 --2- 172.21.15.3:0/3335928825 >> v2:172.21.15.190:3300/0 conn(0x7f20e806d8f0 0x7f20e80ae7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:35.043 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:35.062+0000 7f20edc98640 1 -- 172.21.15.3:0/3335928825 >> v2:172.21.15.190:3300/0 conn(0x7f20e806d8f0 msgr2=0x7f20e80ae7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:35.043 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:35.062+0000 7f20edc98640 1 --2- 172.21.15.3:0/3335928825 >> v2:172.21.15.190:3300/0 conn(0x7f20e806d8f0 0x7f20e80ae7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:35.043 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:35.062+0000 7f20edc98640 1 -- 172.21.15.3:0/3335928825 >> v1:172.21.15.3:6789/0 conn(0x7f20e806bcf0 legacy=0x7f20e806c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:35.043 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:35.062+0000 7f20ecc96640 1 -- 172.21.15.3:0/3335928825 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 953702919 0 0) 0x7f20e0001110 con 0x7f20e806bcf0 2024-05-09T06:36:35.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:35.062+0000 7f20edc98640 1 -- 172.21.15.3:0/3335928825 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20e806cab0 con 0x7f20e815e100 2024-05-09T06:36:35.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:35.063+0000 7f20edc98640 1 --2- 172.21.15.3:0/3335928825 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f20e815e100 0x7f20e80ae2b0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f20e8072d70 tx=0x7f20d802a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a770de81a87f4ee0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:35.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:35.063+0000 7f20ecc96640 1 -- 172.21.15.3:0/3335928825 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20d8007740 con 0x7f20e815e100 2024-05-09T06:36:35.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:35.063+0000 7f20ecc96640 1 -- 172.21.15.3:0/3335928825 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f20d80078e0 con 0x7f20e815e100 2024-05-09T06:36:35.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:35.063+0000 7f20ecc96640 1 -- 172.21.15.3:0/3335928825 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20d8007cb0 con 0x7f20e815e100 2024-05-09T06:36:35.046 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:35.063+0000 7f20efefc640 1 -- 172.21.15.3:0/3335928825 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f20e815e100 msgr2=0x7f20e80ae2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:35.046 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:35.063+0000 7f20efefc640 1 --2- 172.21.15.3:0/3335928825 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f20e815e100 0x7f20e80ae2b0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f20e8072d70 tx=0x7f20d802a5f0 comp rx=0 tx=0).stop 2024-05-09T06:36:35.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:35.064+0000 7f20efefc640 1 -- 172.21.15.3:0/3335928825 shutdown_connections 2024-05-09T06:36:35.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:35.064+0000 7f20efefc640 1 --2- 172.21.15.3:0/3335928825 >> v2:172.21.15.190:3300/0 conn(0x7f20e806d8f0 0x7f20e80ae7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:35.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:35.064+0000 7f20efefc640 1 --2- 172.21.15.3:0/3335928825 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f20e815e100 0x7f20e80ae2b0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:35.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:35.064+0000 7f20efefc640 1 -- 172.21.15.3:0/3335928825 >> 172.21.15.3:0/3335928825 conn(0x7f20e80d05d0 msgr2=0x7f20e80d09e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:35.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:35.064+0000 7f20efefc640 1 -- 172.21.15.3:0/3335928825 shutdown_connections 2024-05-09T06:36:35.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:35.064+0000 7f20efefc640 1 -- 172.21.15.3:0/3335928825 wait complete. 2024-05-09T06:36:35.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:35.065+0000 7f20efefc640 1 Processor -- start 2024-05-09T06:36:35.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:35.065+0000 7f20efefc640 1 -- start start 2024-05-09T06:36:35.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:35.065+0000 7f20efefc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f20e806bcf0 0x7f20e80ccd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:35.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:35.066+0000 7f20efefc640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f20e806d8f0 0x7f20e8160240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:35.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:35.066+0000 7f20efefc640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f20e815e100 0x7f20e8160780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:35.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:35.066+0000 7f20efefc640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f20e815e4e0 con 0x7f20e806bcf0 2024-05-09T06:36:35.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:35.066+0000 7f20efefc640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f20e80b4400 con 0x7f20e806d8f0 2024-05-09T06:36:35.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:35.066+0000 7f20efefc640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f20e815e660 con 0x7f20e815e100 2024-05-09T06:36:35.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:35.066+0000 7f20edc98640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f20e806bcf0 0x7f20e80ccd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:35.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:35.066+0000 7f20edc98640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f20e806bcf0 0x7f20e80ccd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:35546/0 (socket says 172.21.15.3:35546) 2024-05-09T06:36:35.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:35.066+0000 7f20edc98640 1 -- 172.21.15.3:0/2362643654 learned_addr learned my addr 172.21.15.3:0/2362643654 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:36:35.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:35.066+0000 7f20ed497640 1 --2- 172.21.15.3:0/2362643654 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f20e806d8f0 0x7f20e8160240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:35.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:35.066+0000 7f20ee499640 1 --2- 172.21.15.3:0/2362643654 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f20e815e100 0x7f20e8160780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:35.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:35.066+0000 7f20edc98640 1 -- 172.21.15.3:0/2362643654 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f20e815e100 msgr2=0x7f20e8160780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:35.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:35.066+0000 7f20edc98640 1 --2- 172.21.15.3:0/2362643654 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f20e815e100 0x7f20e8160780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:35.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:35.066+0000 7f20edc98640 1 -- 172.21.15.3:0/2362643654 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f20e806d8f0 msgr2=0x7f20e8160240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:35.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:35.066+0000 7f20edc98640 1 --2- 172.21.15.3:0/2362643654 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f20e806d8f0 0x7f20e8160240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:35.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:35.066+0000 7f20edc98640 1 -- 172.21.15.3:0/2362643654 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20e8160e50 con 0x7f20e806bcf0 2024-05-09T06:36:35.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:35.066+0000 7f20edc98640 1 --2- 172.21.15.3:0/2362643654 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f20e806bcf0 0x7f20e80ccd70 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f20d8009e80 tx=0x7f20d8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:35.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:35.066+0000 7f20e67fc640 1 -- 172.21.15.3:0/2362643654 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20d803f070 con 0x7f20e806bcf0 2024-05-09T06:36:35.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:35.066+0000 7f20e67fc640 1 -- 172.21.15.3:0/2362643654 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f20d8031c40 con 0x7f20e806bcf0 2024-05-09T06:36:35.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:35.066+0000 7f20efefc640 1 -- 172.21.15.3:0/2362643654 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20e8161130 con 0x7f20e806bcf0 2024-05-09T06:36:35.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:35.066+0000 7f20e67fc640 1 -- 172.21.15.3:0/2362643654 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20d8009450 con 0x7f20e806bcf0 2024-05-09T06:36:35.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:35.066+0000 7f20efefc640 1 -- 172.21.15.3:0/2362643654 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20e8141b80 con 0x7f20e806bcf0 2024-05-09T06:36:35.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:35.067+0000 7f20efefc640 1 -- 172.21.15.3:0/2362643654 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20e804f660 con 0x7f20e806bcf0 2024-05-09T06:36:35.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:35.068+0000 7f20e67fc640 1 -- 172.21.15.3:0/2362643654 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f20d8043410 con 0x7f20e806bcf0 2024-05-09T06:36:35.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:35.068+0000 7f20e67fc640 1 --2- 172.21.15.3:0/2362643654 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f20d406be30 0x7f20d406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:35.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:35.069+0000 7f20e67fc640 1 -- 172.21.15.3:0/2362643654 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f20d80ac430 con 0x7f20e806bcf0 2024-05-09T06:36:35.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:35.069+0000 7f20ed497640 1 --2- 172.21.15.3:0/2362643654 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f20d406be30 0x7f20d406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:35.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:35.069+0000 7f20ed497640 1 --2- 172.21.15.3:0/2362643654 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f20d406be30 0x7f20d406e2f0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f20e814f820 tx=0x7f20dc008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:35.052 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:35.073+0000 7f20e67fc640 1 -- 172.21.15.3:0/2362643654 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f20d807ad40 con 0x7f20e806bcf0 2024-05-09T06:36:35.238 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:35.258+0000 7f20efefc640 1 -- 172.21.15.3:0/2362643654 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f20e8158380 con 0x7f20e806bcf0 2024-05-09T06:36:35.238 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:35.259+0000 7f20e67fc640 1 -- 172.21.15.3:0/2362643654 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16521 (secure 0 0 0) 0x7f20d807a640 con 0x7f20e806bcf0 2024-05-09T06:36:35.239 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-09T06:36:35.239 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":20,"fsid":"16ed737d-2665-431a-8ac6-ef242be3975a","created":"2024-05-09T06:35:42.541368+0000","modified":"2024-05-09T06:36:08.856391+0000","last_up_change":"2024-05-09T06:35:53.778274+0000","last_in_change":"2024-05-09T06:35:44.780586+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T06:35:54.819999+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T06:36:04.931314+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T06:36:06.200433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fae3a8db-4826-4f3c-b33a-12c09bb5df6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":1897849827},{"type":"v1","addr":"172.21.15.3:6817","nonce":1897849827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":1897849827},{"type":"v1","addr":"172.21.15.3:6819","nonce":1897849827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":1897849827},{"type":"v1","addr":"172.21.15.3:6823","nonce":1897849827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":1897849827},{"type":"v1","addr":"172.21.15.3:6821","nonce":1897849827}]},"public_addr":"172.21.15.3:6817/1897849827","cluster_addr":"172.21.15.3:6819/1897849827","heartbeat_back_addr":"172.21.15.3:6823/1897849827","heartbeat_front_addr":"172.21.15.3:6821/1897849827","state":["exists","up"]},{"osd":1,"uuid":"646e882d-4ce4-4720-8c84-f71df563a35e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":926805498},{"type":"v1","addr":"172.21.15.3:6801","nonce":926805498}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":926805498},{"type":"v1","addr":"172.21.15.3:6803","nonce":926805498}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":926805498},{"type":"v1","addr":"172.21.15.3:6807","nonce":926805498}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":926805498},{"type":"v1","addr":"172.21.15.3:6805","nonce":926805498}]},"public_addr":"172.21.15.3:6801/926805498","cluster_addr":"172.21.15.3:6803/926805498","heartbeat_back_addr":"172.21.15.3:6807/926805498","heartbeat_front_addr":"172.21.15.3:6805/926805498","state":["exists","up"]},{"osd":2,"uuid":"1abbadb5-a570-4359-9643-acff3ede35a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":801295451},{"type":"v1","addr":"172.21.15.3:6809","nonce":801295451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":801295451},{"type":"v1","addr":"172.21.15.3:6811","nonce":801295451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":801295451},{"type":"v1","addr":"172.21.15.3:6815","nonce":801295451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":801295451},{"type":"v1","addr":"172.21.15.3:6813","nonce":801295451}]},"public_addr":"172.21.15.3:6809/801295451","cluster_addr":"172.21.15.3:6811/801295451","heartbeat_back_addr":"172.21.15.3:6815/801295451","heartbeat_front_addr":"172.21.15.3:6813/801295451","state":["exists","up"]},{"osd":3,"uuid":"62ac4dc1-2371-4dd2-acd3-e2dbb11aedf5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":1903389857},{"type":"v1","addr":"172.21.15.3:6825","nonce":1903389857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":1903389857},{"type":"v1","addr":"172.21.15.3:6827","nonce":1903389857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":1903389857},{"type":"v1","addr":"172.21.15.3:6831","nonce":1903389857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":1903389857},{"type":"v1","addr":"172.21.15.3:6829","nonce":1903389857}]},"public_addr":"172.21.15.3:6825/1903389857","cluster_addr":"172.21.15.3:6827/1903389857","heartbeat_back_addr":"172.21.15.3:6831/1903389857","heartbeat_front_addr":"172.21.15.3:6829/1903389857","state":["exists","up"]},{"osd":4,"uuid":"ae808398-8326-4d7c-9319-a2a0eafffb59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":2143624576},{"type":"v1","addr":"172.21.15.190:6801","nonce":2143624576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":2143624576},{"type":"v1","addr":"172.21.15.190:6803","nonce":2143624576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":2143624576},{"type":"v1","addr":"172.21.15.190:6807","nonce":2143624576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":2143624576},{"type":"v1","addr":"172.21.15.190:6805","nonce":2143624576}]},"public_addr":"172.21.15.190:6801/2143624576","cluster_addr":"172.21.15.190:6803/2143624576","heartbeat_back_addr":"172.21.15.190:6807/2143624576","heartbeat_front_addr":"172.21.15.190:6805/2143624576","state":["exists","up"]},{"osd":5,"uuid":"918adf50-c440-48ac-9a8d-4837562d7275","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":1766923126},{"type":"v1","addr":"172.21.15.190:6814","nonce":1766923126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":1766923126},{"type":"v1","addr":"172.21.15.190:6818","nonce":1766923126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":1766923126},{"type":"v1","addr":"172.21.15.190:6823","nonce":1766923126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":1766923126},{"type":"v1","addr":"172.21.15.190:6821","nonce":1766923126}]},"public_addr":"172.21.15.190:6814/1766923126","cluster_addr":"172.21.15.190:6818/1766923126","heartbeat_back_addr":"172.21.15.190:6823/1766923126","heartbeat_front_addr":"172.21.15.190:6821/1766923126","state":["exists","up"]},{"osd":6,"uuid":"ac2882bc-2a7c-463c-8f6c-76ff49828adc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":342414248},{"type":"v1","addr":"172.21.15.190:6825","nonce":342414248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":342414248},{"type":"v1","addr":"172.21.15.190:6827","nonce":342414248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":342414248},{"type":"v1","addr":"172.21.15.190:6831","nonce":342414248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":342414248},{"type":"v1","addr":"172.21.15.190:6829","nonce":342414248}]},"public_addr":"172.21.15.190:6825/342414248","cluster_addr":"172.21.15.190:6827/342414248","heartbeat_back_addr":"172.21.15.190:6831/342414248","heartbeat_front_addr":"172.21.15.190:6829/342414248","state":["exists","up"]},{"osd":7,"uuid":"dd4e767a-4730-4c70-8bd5-3f088140a5a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":537849891},{"type":"v1","addr":"172.21.15.190:6809","nonce":537849891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":537849891},{"type":"v1","addr":"172.21.15.190:6811","nonce":537849891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6817","nonce":537849891},{"type":"v1","addr":"172.21.15.190:6819","nonce":537849891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6813","nonce":537849891},{"type":"v1","addr":"172.21.15.190:6815","nonce":537849891}]},"public_addr":"172.21.15.190:6809/537849891","cluster_addr":"172.21.15.190:6811/537849891","heartbeat_back_addr":"172.21.15.190:6819/537849891","heartbeat_front_addr":"172.21.15.190:6815/537849891","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.889784+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.974002+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.973549+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.955396+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.928471+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.902418+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.958696+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.926474+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T06:36:35.241 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:35.261+0000 7f20efefc640 1 -- 172.21.15.3:0/2362643654 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f20d406be30 msgr2=0x7f20d406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:35.241 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:35.261+0000 7f20efefc640 1 --2- 172.21.15.3:0/2362643654 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f20d406be30 0x7f20d406e2f0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f20e814f820 tx=0x7f20dc008040 comp rx=0 tx=0).stop 2024-05-09T06:36:35.241 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:35.261+0000 7f20efefc640 1 -- 172.21.15.3:0/2362643654 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f20e806bcf0 msgr2=0x7f20e80ccd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:35.241 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:35.261+0000 7f20efefc640 1 --2- 172.21.15.3:0/2362643654 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f20e806bcf0 0x7f20e80ccd70 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f20d8009e80 tx=0x7f20d8004a40 comp rx=0 tx=0).stop 2024-05-09T06:36:35.241 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:35.261+0000 7f20efefc640 1 -- 172.21.15.3:0/2362643654 shutdown_connections 2024-05-09T06:36:35.241 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:35.261+0000 7f20efefc640 1 --2- 172.21.15.3:0/2362643654 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f20d406be30 0x7f20d406e2f0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:35.241 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:35.261+0000 7f20efefc640 1 --2- 172.21.15.3:0/2362643654 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f20e815e100 0x7f20e8160780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:35.241 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:35.261+0000 7f20efefc640 1 --2- 172.21.15.3:0/2362643654 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f20e806d8f0 0x7f20e8160240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:35.241 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:35.261+0000 7f20efefc640 1 --2- 172.21.15.3:0/2362643654 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f20e806bcf0 0x7f20e80ccd70 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:35.241 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:35.261+0000 7f20efefc640 1 -- 172.21.15.3:0/2362643654 >> 172.21.15.3:0/2362643654 conn(0x7f20e80d05d0 msgr2=0x7f20e80b1b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:35.242 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:35.261+0000 7f20efefc640 1 -- 172.21.15.3:0/2362643654 shutdown_connections 2024-05-09T06:36:35.242 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:35.261+0000 7f20efefc640 1 -- 172.21.15.3:0/2362643654 wait complete. 2024-05-09T06:36:35.252 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-09T06:35:54.819999+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-09T06:36:04.931314+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-09T06:36:06.200433+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-09T06:36:35.252 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-09T06:36:35.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:35.379+0000 7f1a9f8f5640 1 Processor -- start 2024-05-09T06:36:35.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:35.380+0000 7f1a9f8f5640 1 -- start start 2024-05-09T06:36:35.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:35.380+0000 7f1a9f8f5640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f1a9815e490 0x7f1a98162590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:35.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:35.380+0000 7f1a9f8f5640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1a9806cad0 con 0x7f1a9806b250 2024-05-09T06:36:35.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:35.380+0000 7f1a9f8f5640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1a980b3eb0 con 0x7f1a9815e0b0 2024-05-09T06:36:35.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:35.380+0000 7f1a9f8f5640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f1a980b4190 con 0x7f1a9815e490 2024-05-09T06:36:35.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:35.381+0000 7f1a9ce90640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f1a9815e490 0x7f1a98162590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:35.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:35.381+0000 7f1a9ce90640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f1a9815e490 0x7f1a98162590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:35560/0 (socket says 172.21.15.3:35560) 2024-05-09T06:36:35.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:35.381+0000 7f1a9de92640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f1a9806b250 0x7f1a9806b6c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:58958/0 (socket says 172.21.15.3:58958) 2024-05-09T06:36:35.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:35.381+0000 7f1a9ce90640 1 -- 172.21.15.3:0/1323513209 learned_addr learned my addr 172.21.15.3:0/1323513209 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:36:35.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:35.381+0000 7f1a9ce90640 1 -- 172.21.15.3:0/1323513209 >> v1:172.21.15.190:6789/0 conn(0x7f1a9815e0b0 legacy=0x7f1a98161e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:35.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:35.381+0000 7f1a9ce90640 1 -- 172.21.15.3:0/1323513209 >> v1:172.21.15.3:6789/0 conn(0x7f1a9806b250 legacy=0x7f1a9806b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:35.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:35.381+0000 7f1a9ce90640 1 -- 172.21.15.3:0/1323513209 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a980b4470 con 0x7f1a9815e490 2024-05-09T06:36:35.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:35.381+0000 7f1a8ffff640 1 -- 172.21.15.3:0/1323513209 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3778972814 0 0) 0x7f1a9806cad0 con 0x7f1a9806b250 2024-05-09T06:36:35.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:35.382+0000 7f1a9ce90640 1 --2- 172.21.15.3:0/1323513209 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1a9815e490 0x7f1a98162590 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f1a88009870 tx=0x7f1a8802a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fe1d41cc6d496ed server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:35.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:35.382+0000 7f1a8ffff640 1 -- 172.21.15.3:0/1323513209 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a88037070 con 0x7f1a9815e490 2024-05-09T06:36:35.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:35.382+0000 7f1a8ffff640 1 -- 172.21.15.3:0/1323513209 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a8802ae50 con 0x7f1a9815e490 2024-05-09T06:36:35.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:35.382+0000 7f1a8ffff640 1 -- 172.21.15.3:0/1323513209 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a88032860 con 0x7f1a9815e490 2024-05-09T06:36:35.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:35.382+0000 7f1a9f8f5640 1 -- 172.21.15.3:0/1323513209 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1a9815e490 msgr2=0x7f1a98162590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:35.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:35.382+0000 7f1a9f8f5640 1 --2- 172.21.15.3:0/1323513209 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1a9815e490 0x7f1a98162590 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f1a88009870 tx=0x7f1a8802a300 comp rx=0 tx=0).stop 2024-05-09T06:36:35.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:35.382+0000 7f1a9f8f5640 1 -- 172.21.15.3:0/1323513209 shutdown_connections 2024-05-09T06:36:35.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:35.382+0000 7f1a9f8f5640 1 --2- 172.21.15.3:0/1323513209 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1a9815e490 0x7f1a98162590 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:35.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:35.382+0000 7f1a9f8f5640 1 -- 172.21.15.3:0/1323513209 >> 172.21.15.3:0/1323513209 conn(0x7f1a980d07b0 msgr2=0x7f1a980d0bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:35.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:35.382+0000 7f1a9f8f5640 1 -- 172.21.15.3:0/1323513209 shutdown_connections 2024-05-09T06:36:35.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:35.382+0000 7f1a9f8f5640 1 -- 172.21.15.3:0/1323513209 wait complete. 2024-05-09T06:36:35.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:35.384+0000 7f1a9f8f5640 1 Processor -- start 2024-05-09T06:36:35.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:35.384+0000 7f1a9f8f5640 1 -- start start 2024-05-09T06:36:35.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:35.384+0000 7f1a9f8f5640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1a9806b250 0x7f1a9813fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:35.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:35.384+0000 7f1a9f8f5640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1a9815e0b0 0x7f1a9815c040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:35.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:35.384+0000 7f1a9f8f5640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1a9815e490 0x7f1a980b1050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:35.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:35.384+0000 7f1a9f8f5640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f1a9806da70 con 0x7f1a9815e490 2024-05-09T06:36:35.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:35.384+0000 7f1a9f8f5640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f1a9806d8f0 con 0x7f1a9815e0b0 2024-05-09T06:36:35.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:35.384+0000 7f1a9f8f5640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f1a98162ad0 con 0x7f1a9806b250 2024-05-09T06:36:35.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:35.384+0000 7f1a9de92640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1a9815e490 0x7f1a980b1050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:35.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:35.385+0000 7f1a9de92640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1a9815e490 0x7f1a980b1050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:35572/0 (socket says 172.21.15.3:35572) 2024-05-09T06:36:35.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:35.385+0000 7f1a9de92640 1 -- 172.21.15.3:0/3390240207 learned_addr learned my addr 172.21.15.3:0/3390240207 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:36:35.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:35.385+0000 7f1a9ce90640 1 --2- 172.21.15.3:0/3390240207 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1a9815e0b0 0x7f1a9815c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:35.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:35.385+0000 7f1a9d691640 1 --2- 172.21.15.3:0/3390240207 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1a9806b250 0x7f1a9813fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:35.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:35.385+0000 7f1a9de92640 1 -- 172.21.15.3:0/3390240207 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1a9806b250 msgr2=0x7f1a9813fef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:35.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:35.385+0000 7f1a9de92640 1 --2- 172.21.15.3:0/3390240207 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1a9806b250 0x7f1a9813fef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:35.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:35.385+0000 7f1a9de92640 1 -- 172.21.15.3:0/3390240207 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1a9815e0b0 msgr2=0x7f1a9815c040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:35.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:35.385+0000 7f1a9de92640 1 --2- 172.21.15.3:0/3390240207 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1a9815e0b0 0x7f1a9815c040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:35.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:35.385+0000 7f1a9de92640 1 -- 172.21.15.3:0/3390240207 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a981537e0 con 0x7f1a9815e490 2024-05-09T06:36:35.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:35.385+0000 7f1a9de92640 1 --2- 172.21.15.3:0/3390240207 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1a9815e490 0x7f1a980b1050 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f1a9000c7a0 tx=0x7f1a9000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:35.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:35.385+0000 7f1a8dffb640 1 -- 172.21.15.3:0/3390240207 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a90018070 con 0x7f1a9815e490 2024-05-09T06:36:35.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:35.385+0000 7f1a8dffb640 1 -- 172.21.15.3:0/3390240207 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a9000a640 con 0x7f1a9815e490 2024-05-09T06:36:35.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:35.385+0000 7f1a8dffb640 1 -- 172.21.15.3:0/3390240207 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a9000aab0 con 0x7f1a9815e490 2024-05-09T06:36:35.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:35.385+0000 7f1a9f8f5640 1 -- 172.21.15.3:0/3390240207 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a98153ac0 con 0x7f1a9815e490 2024-05-09T06:36:35.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:35.385+0000 7f1a9f8f5640 1 -- 172.21.15.3:0/3390240207 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a98140430 con 0x7f1a9815e490 2024-05-09T06:36:35.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:35.386+0000 7f1a9f8f5640 1 -- 172.21.15.3:0/3390240207 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a98069550 con 0x7f1a9815e490 2024-05-09T06:36:35.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:35.387+0000 7f1a8dffb640 1 -- 172.21.15.3:0/3390240207 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1a9001b070 con 0x7f1a9815e490 2024-05-09T06:36:35.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:35.387+0000 7f1a8dffb640 1 --2- 172.21.15.3:0/3390240207 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f1a7c06be60 0x7f1a7c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:35.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:35.387+0000 7f1a8dffb640 1 -- 172.21.15.3:0/3390240207 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f1a9008a460 con 0x7f1a9815e490 2024-05-09T06:36:35.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:35.388+0000 7f1a9d691640 1 --2- 172.21.15.3:0/3390240207 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f1a7c06be60 0x7f1a7c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:35.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:35.388+0000 7f1a9d691640 1 --2- 172.21.15.3:0/3390240207 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f1a7c06be60 0x7f1a7c06e320 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f1a94000f60 tx=0x7f1a94009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:35.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:35.392+0000 7f1a8dffb640 1 -- 172.21.15.3:0/3390240207 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f1a90058720 con 0x7f1a9815e490 2024-05-09T06:36:35.554 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:35.574+0000 7f1a9f8f5640 1 -- 172.21.15.3:0/3390240207 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f1a980d0280 con 0x7f1a9815e490 2024-05-09T06:36:35.554 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:35.575+0000 7f1a8dffb640 1 -- 172.21.15.3:0/3390240207 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7f1a9005ca20 con 0x7f1a9815e490 2024-05-09T06:36:35.555 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 1 2024-05-09T06:36:35.557 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:35.577+0000 7f1a9f8f5640 1 -- 172.21.15.3:0/3390240207 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f1a7c06be60 msgr2=0x7f1a7c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:35.557 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:35.577+0000 7f1a9f8f5640 1 --2- 172.21.15.3:0/3390240207 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f1a7c06be60 0x7f1a7c06e320 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f1a94000f60 tx=0x7f1a94009040 comp rx=0 tx=0).stop 2024-05-09T06:36:35.557 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:35.577+0000 7f1a9f8f5640 1 -- 172.21.15.3:0/3390240207 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1a9815e490 msgr2=0x7f1a980b1050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:35.557 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:35.577+0000 7f1a9f8f5640 1 --2- 172.21.15.3:0/3390240207 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1a9815e490 0x7f1a980b1050 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f1a9000c7a0 tx=0x7f1a9000cc60 comp rx=0 tx=0).stop 2024-05-09T06:36:35.557 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:35.577+0000 7f1a9f8f5640 1 -- 172.21.15.3:0/3390240207 shutdown_connections 2024-05-09T06:36:35.557 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:35.577+0000 7f1a9f8f5640 1 --2- 172.21.15.3:0/3390240207 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f1a7c06be60 0x7f1a7c06e320 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:35.557 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:35.577+0000 7f1a9f8f5640 1 --2- 172.21.15.3:0/3390240207 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1a9815e490 0x7f1a980b1050 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:35.557 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:35.577+0000 7f1a9f8f5640 1 --2- 172.21.15.3:0/3390240207 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1a9815e0b0 0x7f1a9815c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:35.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:35.577+0000 7f1a9f8f5640 1 --2- 172.21.15.3:0/3390240207 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1a9806b250 0x7f1a9813fef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:35.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:35.577+0000 7f1a9f8f5640 1 -- 172.21.15.3:0/3390240207 >> 172.21.15.3:0/3390240207 conn(0x7f1a980d07b0 msgr2=0x7f1a98161e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:35.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:35.577+0000 7f1a9f8f5640 1 -- 172.21.15.3:0/3390240207 shutdown_connections 2024-05-09T06:36:35.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:35.577+0000 7f1a9f8f5640 1 -- 172.21.15.3:0/3390240207 wait complete. 2024-05-09T06:36:35.568 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-09T06:36:35.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:35.695+0000 7f1157293640 1 Processor -- start 2024-05-09T06:36:35.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:35.695+0000 7f1157293640 1 -- start start 2024-05-09T06:36:35.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:35.696+0000 7f1157293640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f11500b4520 0x7f11500ae600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:35.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:35.696+0000 7f1157293640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1150162280 con 0x7f115015dff0 2024-05-09T06:36:35.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:35.696+0000 7f1157293640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1150162560 con 0x7f1150161df0 2024-05-09T06:36:35.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:35.696+0000 7f1157293640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f1150162840 con 0x7f11500b4520 2024-05-09T06:36:35.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:35.696+0000 7f1155830640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f11500b4520 0x7f11500ae600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:35.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:35.696+0000 7f115502f640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f115015dff0 0x7f115006e600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:58964/0 (socket says 172.21.15.3:58964) 2024-05-09T06:36:35.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:35.696+0000 7f1155830640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f11500b4520 0x7f11500ae600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:35584/0 (socket says 172.21.15.3:35584) 2024-05-09T06:36:35.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:35.696+0000 7f115502f640 1 -- 172.21.15.3:0/1485680310 learned_addr learned my addr 172.21.15.3:0/1485680310 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-05-09T06:36:35.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:35.696+0000 7f1155830640 1 -- 172.21.15.3:0/1485680310 >> v1:172.21.15.190:6789/0 conn(0x7f1150161df0 legacy=0x7f11500728f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:35.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:35.696+0000 7f1155830640 1 -- 172.21.15.3:0/1485680310 >> v1:172.21.15.3:6789/0 conn(0x7f115015dff0 legacy=0x7f115006e600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:35.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:35.696+0000 7f1147fff640 1 -- 172.21.15.3:0/1485680310 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1978375278 0 0) 0x7f1150162280 con 0x7f115015dff0 2024-05-09T06:36:35.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:35.696+0000 7f1155830640 1 -- 172.21.15.3:0/1485680310 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1150162b20 con 0x7f11500b4520 2024-05-09T06:36:35.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:35.697+0000 7f1155830640 1 --2- 172.21.15.3:0/1485680310 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f11500b4520 0x7f11500ae600 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f1148009d90 tx=0x7f114802a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bdcffcc04c8a962 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:35.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:35.697+0000 7f1147fff640 1 -- 172.21.15.3:0/1485680310 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1148007740 con 0x7f11500b4520 2024-05-09T06:36:35.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:35.697+0000 7f1147fff640 1 -- 172.21.15.3:0/1485680310 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f11480078e0 con 0x7f11500b4520 2024-05-09T06:36:35.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:35.697+0000 7f1147fff640 1 -- 172.21.15.3:0/1485680310 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1148007cb0 con 0x7f11500b4520 2024-05-09T06:36:35.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:35.698+0000 7f1157293640 1 -- 172.21.15.3:0/1485680310 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f11500b4520 msgr2=0x7f11500ae600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:35.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:35.698+0000 7f1157293640 1 --2- 172.21.15.3:0/1485680310 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f11500b4520 0x7f11500ae600 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f1148009d90 tx=0x7f114802a660 comp rx=0 tx=0).stop 2024-05-09T06:36:35.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:35.698+0000 7f1157293640 1 -- 172.21.15.3:0/1485680310 shutdown_connections 2024-05-09T06:36:35.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:35.698+0000 7f1157293640 1 --2- 172.21.15.3:0/1485680310 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f11500b4520 0x7f11500ae600 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:35.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:35.698+0000 7f1157293640 1 -- 172.21.15.3:0/1485680310 >> 172.21.15.3:0/1485680310 conn(0x7f11500b6530 msgr2=0x7f11500b6910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:35.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:35.698+0000 7f1157293640 1 -- 172.21.15.3:0/1485680310 shutdown_connections 2024-05-09T06:36:35.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:35.698+0000 7f1157293640 1 -- 172.21.15.3:0/1485680310 wait complete. 2024-05-09T06:36:35.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:35.699+0000 7f1157293640 1 Processor -- start 2024-05-09T06:36:35.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:35.699+0000 7f1157293640 1 -- start start 2024-05-09T06:36:35.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:35.700+0000 7f1157293640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f11500b4520 0x7f11501447d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:35.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:35.700+0000 7f1157293640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f115015dff0 0x7f115015e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:35.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:35.700+0000 7f1157293640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f11500b3100 0x7f1150160800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:35.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:35.700+0000 7f1157293640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f115015e3d0 con 0x7f11500b4520 2024-05-09T06:36:35.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:35.700+0000 7f1157293640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f115015e550 con 0x7f11500b3100 2024-05-09T06:36:35.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:35.700+0000 7f1157293640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f115015e6d0 con 0x7f115015dff0 2024-05-09T06:36:35.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:35.700+0000 7f115502f640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f11500b4520 0x7f11501447d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:35.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:35.700+0000 7f115502f640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f11500b4520 0x7f11501447d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:35594/0 (socket says 172.21.15.3:35594) 2024-05-09T06:36:35.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:35.700+0000 7f115502f640 1 -- 172.21.15.3:0/1680910425 learned_addr learned my addr 172.21.15.3:0/1680910425 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:36:35.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:35.700+0000 7f115482e640 1 --2- 172.21.15.3:0/1680910425 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f115015dff0 0x7f115015e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:35.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:35.700+0000 7f1155830640 1 --2- 172.21.15.3:0/1680910425 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f11500b3100 0x7f1150160800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:35.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:35.700+0000 7f115502f640 1 -- 172.21.15.3:0/1680910425 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f115015dff0 msgr2=0x7f115015e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:35.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:35.700+0000 7f115502f640 1 --2- 172.21.15.3:0/1680910425 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f115015dff0 0x7f115015e850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:35.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:35.700+0000 7f115502f640 1 -- 172.21.15.3:0/1680910425 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f11500b3100 msgr2=0x7f1150160800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:35.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:35.700+0000 7f115502f640 1 --2- 172.21.15.3:0/1680910425 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f11500b3100 0x7f1150160800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:35.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:35.700+0000 7f115502f640 1 -- 172.21.15.3:0/1680910425 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f115015f1c0 con 0x7f11500b4520 2024-05-09T06:36:35.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:35.700+0000 7f1155830640 1 --2- 172.21.15.3:0/1680910425 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f11500b3100 0x7f1150160800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:36:35.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:35.700+0000 7f115482e640 1 --2- 172.21.15.3:0/1680910425 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f115015dff0 0x7f115015e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:36:35.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:35.700+0000 7f115502f640 1 --2- 172.21.15.3:0/1680910425 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f11500b4520 0x7f11501447d0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f114c00c4d0 tx=0x7f114c00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:35.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:35.701+0000 7f1145ffb640 1 -- 172.21.15.3:0/1680910425 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f114c018070 con 0x7f11500b4520 2024-05-09T06:36:35.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:35.701+0000 7f1145ffb640 1 -- 172.21.15.3:0/1680910425 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f114c0074e0 con 0x7f11500b4520 2024-05-09T06:36:35.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:35.701+0000 7f1157293640 1 -- 172.21.15.3:0/1680910425 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f115015f4a0 con 0x7f11500b4520 2024-05-09T06:36:35.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:35.701+0000 7f1145ffb640 1 -- 172.21.15.3:0/1680910425 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f114c004a20 con 0x7f11500b4520 2024-05-09T06:36:35.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:35.701+0000 7f1157293640 1 -- 172.21.15.3:0/1680910425 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f115015fa60 con 0x7f11500b4520 2024-05-09T06:36:35.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:35.702+0000 7f1157293640 1 -- 172.21.15.3:0/1680910425 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1150069550 con 0x7f11500b4520 2024-05-09T06:36:35.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:35.702+0000 7f1145ffb640 1 -- 172.21.15.3:0/1680910425 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f114c01b070 con 0x7f11500b4520 2024-05-09T06:36:35.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:35.703+0000 7f1145ffb640 1 --2- 172.21.15.3:0/1680910425 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f113406be30 0x7f113406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:35.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:35.703+0000 7f1145ffb640 1 -- 172.21.15.3:0/1680910425 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f114c08a170 con 0x7f11500b4520 2024-05-09T06:36:35.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:35.703+0000 7f115482e640 1 --2- 172.21.15.3:0/1680910425 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f113406be30 0x7f113406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:35.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:35.703+0000 7f115482e640 1 --2- 172.21.15.3:0/1680910425 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f113406be30 0x7f113406e2f0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f115006bca0 tx=0x7f1140009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:35.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:35.707+0000 7f1145ffb640 1 -- 172.21.15.3:0/1680910425 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f114c058b20 con 0x7f11500b4520 2024-05-09T06:36:35.869 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:35.889+0000 7f1157293640 1 -- 172.21.15.3:0/1680910425 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f11500afec0 con 0x7f11500b4520 2024-05-09T06:36:35.869 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:35.889+0000 7f1145ffb640 1 -- 172.21.15.3:0/1680910425 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7f114c058420 con 0x7f11500b4520 2024-05-09T06:36:35.869 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 32 2024-05-09T06:36:35.871 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:35.891+0000 7f1157293640 1 -- 172.21.15.3:0/1680910425 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f113406be30 msgr2=0x7f113406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:35.871 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:35.891+0000 7f1157293640 1 --2- 172.21.15.3:0/1680910425 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f113406be30 0x7f113406e2f0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f115006bca0 tx=0x7f1140009040 comp rx=0 tx=0).stop 2024-05-09T06:36:35.871 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:35.891+0000 7f1157293640 1 -- 172.21.15.3:0/1680910425 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f11500b4520 msgr2=0x7f11501447d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:35.871 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:35.891+0000 7f1157293640 1 --2- 172.21.15.3:0/1680910425 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f11500b4520 0x7f11501447d0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f114c00c4d0 tx=0x7f114c00c990 comp rx=0 tx=0).stop 2024-05-09T06:36:35.871 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:35.891+0000 7f1157293640 1 -- 172.21.15.3:0/1680910425 shutdown_connections 2024-05-09T06:36:35.871 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:35.891+0000 7f1157293640 1 --2- 172.21.15.3:0/1680910425 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f113406be30 0x7f113406e2f0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:35.871 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:35.891+0000 7f1157293640 1 --2- 172.21.15.3:0/1680910425 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f11500b3100 0x7f1150160800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:35.871 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:35.891+0000 7f1157293640 1 --2- 172.21.15.3:0/1680910425 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f115015dff0 0x7f115015e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:35.872 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:35.891+0000 7f1157293640 1 --2- 172.21.15.3:0/1680910425 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f11500b4520 0x7f11501447d0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:35.872 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:35.891+0000 7f1157293640 1 -- 172.21.15.3:0/1680910425 >> 172.21.15.3:0/1680910425 conn(0x7f11500b6530 msgr2=0x7f1150158f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:35.872 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:35.891+0000 7f1157293640 1 -- 172.21.15.3:0/1680910425 shutdown_connections 2024-05-09T06:36:35.872 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:35.891+0000 7f1157293640 1 -- 172.21.15.3:0/1680910425 wait complete. 2024-05-09T06:36:35.882 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-09T06:36:35.989 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:36.009+0000 7fac41bf9640 1 Processor -- start 2024-05-09T06:36:35.989 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:36.009+0000 7fac41bf9640 1 -- start start 2024-05-09T06:36:35.991 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:36.010+0000 7fac41bf9640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fac3c06cd70 0x7fac3c162a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:35.991 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:36.010+0000 7fac41bf9640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fac3c0d02b0 0x7fac3c06d150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:35.991 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:36.010+0000 7fac41bf9640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fac3c162fa0 con 0x7fac3c09aa30 2024-05-09T06:36:35.991 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:36.010+0000 7fac41bf9640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fac3c161e40 con 0x7fac3c06cd70 2024-05-09T06:36:35.991 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:36.010+0000 7fac41bf9640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7fac3c06d690 con 0x7fac3c0d02b0 2024-05-09T06:36:35.991 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:36.010+0000 7fac3b7fe640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fac3c06cd70 0x7fac3c162a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:35.991 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:36.010+0000 7fac3b7fe640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fac3c06cd70 0x7fac3c162a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:35600/0 (socket says 172.21.15.3:35600) 2024-05-09T06:36:35.991 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:36.010+0000 7fac3b7fe640 1 -- 172.21.15.3:0/593726996 learned_addr learned my addr 172.21.15.3:0/593726996 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:36:35.991 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:36.010+0000 7fac3b7fe640 1 -- 172.21.15.3:0/593726996 >> v2:172.21.15.190:3300/0 conn(0x7fac3c0d02b0 msgr2=0x7fac3c06d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:35.991 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:36.010+0000 7fac3b7fe640 1 --2- 172.21.15.3:0/593726996 >> v2:172.21.15.190:3300/0 conn(0x7fac3c0d02b0 0x7fac3c06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:35.992 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:36.010+0000 7fac3b7fe640 1 -- 172.21.15.3:0/593726996 >> v1:172.21.15.190:6789/0 conn(0x7fac3c09aa30 legacy=0x7fac3c09aea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:35.992 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:36.010+0000 7fac3b7fe640 1 -- 172.21.15.3:0/593726996 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac3c0d0870 con 0x7fac3c06cd70 2024-05-09T06:36:35.992 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:36.011+0000 7fac3b7fe640 1 --2- 172.21.15.3:0/593726996 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fac3c06cd70 0x7fac3c162a60 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fac3c072860 tx=0x7fac2c02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e6a9566f5449e47 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:35.992 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:36.011+0000 7fac3a7fc640 1 -- 172.21.15.3:0/593726996 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac2c037070 con 0x7fac3c06cd70 2024-05-09T06:36:35.992 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:36.011+0000 7fac3a7fc640 1 -- 172.21.15.3:0/593726996 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac2c032440 con 0x7fac3c06cd70 2024-05-09T06:36:35.992 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:36.011+0000 7fac3a7fc640 1 -- 172.21.15.3:0/593726996 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac2c032810 con 0x7fac3c06cd70 2024-05-09T06:36:35.993 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:36.012+0000 7fac41bf9640 1 -- 172.21.15.3:0/593726996 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fac3c06cd70 msgr2=0x7fac3c162a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:35.993 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:36.012+0000 7fac41bf9640 1 --2- 172.21.15.3:0/593726996 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fac3c06cd70 0x7fac3c162a60 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fac3c072860 tx=0x7fac2c02a210 comp rx=0 tx=0).stop 2024-05-09T06:36:35.993 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:36.012+0000 7fac41bf9640 1 -- 172.21.15.3:0/593726996 shutdown_connections 2024-05-09T06:36:35.993 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:36.012+0000 7fac41bf9640 1 --2- 172.21.15.3:0/593726996 >> v2:172.21.15.190:3300/0 conn(0x7fac3c0d02b0 0x7fac3c06d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:35.993 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:36.012+0000 7fac41bf9640 1 --2- 172.21.15.3:0/593726996 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fac3c06cd70 0x7fac3c162a60 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:35.993 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:36.012+0000 7fac41bf9640 1 -- 172.21.15.3:0/593726996 >> 172.21.15.3:0/593726996 conn(0x7fac3c0b3af0 msgr2=0x7fac3c0b3f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:35.993 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:36.012+0000 7fac41bf9640 1 -- 172.21.15.3:0/593726996 shutdown_connections 2024-05-09T06:36:35.993 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:36.012+0000 7fac41bf9640 1 -- 172.21.15.3:0/593726996 wait complete. 2024-05-09T06:36:35.994 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:36.013+0000 7fac41bf9640 1 Processor -- start 2024-05-09T06:36:35.994 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:36.013+0000 7fac41bf9640 1 -- start start 2024-05-09T06:36:35.996 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:36.014+0000 7fac41bf9640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fac3c06cd70 0x7fac3c15c960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:35.996 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:36.014+0000 7fac41bf9640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fac3c09aa30 0x7fac3c0c7d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:35.996 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:36.014+0000 7fac41bf9640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fac3c0d02b0 0x7fac3c0c82a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:35.997 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:36.014+0000 7fac41bf9640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fac3c0d1110 con 0x7fac3c0d02b0 2024-05-09T06:36:35.997 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:36.014+0000 7fac41bf9640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fac3c0d0e30 con 0x7fac3c09aa30 2024-05-09T06:36:35.997 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:36.014+0000 7fac41bf9640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fac3c0d13f0 con 0x7fac3c06cd70 2024-05-09T06:36:35.997 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:36.014+0000 7fac3bfff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fac3c0d02b0 0x7fac3c0c82a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:35.997 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:36.014+0000 7fac3bfff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fac3c0d02b0 0x7fac3c0c82a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:35602/0 (socket says 172.21.15.3:35602) 2024-05-09T06:36:35.997 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:36.014+0000 7fac3bfff640 1 -- 172.21.15.3:0/3906770862 learned_addr learned my addr 172.21.15.3:0/3906770862 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:36:35.997 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:36.014+0000 7fac3affd640 1 --2- 172.21.15.3:0/3906770862 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fac3c09aa30 0x7fac3c0c7d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:35.997 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:36.014+0000 7fac3b7fe640 1 --2- 172.21.15.3:0/3906770862 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fac3c06cd70 0x7fac3c15c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:35.997 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:36.014+0000 7fac3bfff640 1 -- 172.21.15.3:0/3906770862 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fac3c06cd70 msgr2=0x7fac3c15c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:35.997 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:36.014+0000 7fac3bfff640 1 --2- 172.21.15.3:0/3906770862 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fac3c06cd70 0x7fac3c15c960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:35.997 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:36.014+0000 7fac3bfff640 1 -- 172.21.15.3:0/3906770862 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fac3c09aa30 msgr2=0x7fac3c0c7d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:35.997 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:36.014+0000 7fac3bfff640 1 --2- 172.21.15.3:0/3906770862 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fac3c09aa30 0x7fac3c0c7d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:35.997 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:36.014+0000 7fac3bfff640 1 -- 172.21.15.3:0/3906770862 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac3c0b5af0 con 0x7fac3c0d02b0 2024-05-09T06:36:35.997 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:36.014+0000 7fac3b7fe640 1 --2- 172.21.15.3:0/3906770862 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fac3c06cd70 0x7fac3c15c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:36:35.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:36.014+0000 7fac3bfff640 1 --2- 172.21.15.3:0/3906770862 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fac3c0d02b0 0x7fac3c0c82a0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fac3400b990 tx=0x7fac3400be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:35.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:36.015+0000 7fac23fff640 1 -- 172.21.15.3:0/3906770862 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac34019070 con 0x7fac3c0d02b0 2024-05-09T06:36:35.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:36.015+0000 7fac23fff640 1 -- 172.21.15.3:0/3906770862 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac3400cba0 con 0x7fac3c0d02b0 2024-05-09T06:36:35.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:36.015+0000 7fac41bf9640 1 -- 172.21.15.3:0/3906770862 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac3c0af080 con 0x7fac3c0d02b0 2024-05-09T06:36:35.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:36.015+0000 7fac23fff640 1 -- 172.21.15.3:0/3906770862 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac34014440 con 0x7fac3c0d02b0 2024-05-09T06:36:35.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:36.015+0000 7fac41bf9640 1 -- 172.21.15.3:0/3906770862 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac3c0af640 con 0x7fac3c0d02b0 2024-05-09T06:36:35.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:36.016+0000 7fac41bf9640 1 -- 172.21.15.3:0/3906770862 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac3c069550 con 0x7fac3c0d02b0 2024-05-09T06:36:35.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:36.016+0000 7fac23fff640 1 -- 172.21.15.3:0/3906770862 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fac3401c070 con 0x7fac3c0d02b0 2024-05-09T06:36:35.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:36.017+0000 7fac23fff640 1 --2- 172.21.15.3:0/3906770862 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fac2806be30 0x7fac2806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:35.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:36.017+0000 7fac23fff640 1 -- 172.21.15.3:0/3906770862 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fac3408ab80 con 0x7fac3c0d02b0 2024-05-09T06:36:35.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:36.017+0000 7fac3b7fe640 1 --2- 172.21.15.3:0/3906770862 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fac2806be30 0x7fac2806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:35.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:36.017+0000 7fac3b7fe640 1 --2- 172.21.15.3:0/3906770862 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fac2806be30 0x7fac2806e2f0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fac2c034000 tx=0x7fac2c02e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:36.000 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:36.021+0000 7fac23fff640 1 -- 172.21.15.3:0/3906770862 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fac3c0b5af0 con 0x7fac3c0d02b0 2024-05-09T06:36:36.186 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:36.206+0000 7fac41bf9640 1 -- 172.21.15.3:0/3906770862 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fac3c06cb50 con 0x7fac3c0d02b0 2024-05-09T06:36:36.186 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:36.206+0000 7fac23fff640 1 -- 172.21.15.3:0/3906770862 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7fac340595c0 con 0x7fac3c0d02b0 2024-05-09T06:36:36.186 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 64 2024-05-09T06:36:36.189 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:36.209+0000 7fac41bf9640 1 -- 172.21.15.3:0/3906770862 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fac2806be30 msgr2=0x7fac2806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:36.189 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:36.209+0000 7fac41bf9640 1 --2- 172.21.15.3:0/3906770862 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fac2806be30 0x7fac2806e2f0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fac2c034000 tx=0x7fac2c02e000 comp rx=0 tx=0).stop 2024-05-09T06:36:36.189 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:36.209+0000 7fac41bf9640 1 -- 172.21.15.3:0/3906770862 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fac3c0d02b0 msgr2=0x7fac3c0c82a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:36.189 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:36.209+0000 7fac41bf9640 1 --2- 172.21.15.3:0/3906770862 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fac3c0d02b0 0x7fac3c0c82a0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fac3400b990 tx=0x7fac3400be50 comp rx=0 tx=0).stop 2024-05-09T06:36:36.189 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:36.209+0000 7fac41bf9640 1 -- 172.21.15.3:0/3906770862 shutdown_connections 2024-05-09T06:36:36.189 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:36.209+0000 7fac41bf9640 1 --2- 172.21.15.3:0/3906770862 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fac2806be30 0x7fac2806e2f0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:36.189 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:36.209+0000 7fac41bf9640 1 --2- 172.21.15.3:0/3906770862 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fac3c0d02b0 0x7fac3c0c82a0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:36.189 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:36.209+0000 7fac41bf9640 1 --2- 172.21.15.3:0/3906770862 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fac3c09aa30 0x7fac3c0c7d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:36.189 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:36.209+0000 7fac41bf9640 1 --2- 172.21.15.3:0/3906770862 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fac3c06cd70 0x7fac3c15c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:36.189 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:36.209+0000 7fac41bf9640 1 -- 172.21.15.3:0/3906770862 >> 172.21.15.3:0/3906770862 conn(0x7fac3c0b3af0 msgr2=0x7fac3c15d690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:36.189 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:36.209+0000 7fac41bf9640 1 -- 172.21.15.3:0/3906770862 shutdown_connections 2024-05-09T06:36:36.190 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:36.209+0000 7fac41bf9640 1 -- 172.21.15.3:0/3906770862 wait complete. 2024-05-09T06:36:36.200 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestPerModuleFinsherThread.test_volumes_module_finisher_thread' 2024-05-09T06:36:36.294 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-09T06:36:36.297 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-09T06:36:36.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:36.326+0000 7f415a75a640 1 Processor -- start 2024-05-09T06:36:36.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:36.326+0000 7f415a75a640 1 -- start start 2024-05-09T06:36:36.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:36.327+0000 7f415a75a640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f41540afc20 0x7f415415d330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:36.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:36.327+0000 7f415a75a640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f41540cf010 0x7f41540cd090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:36.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:36.327+0000 7f415a75a640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f41540cf8f0 con 0x7f41540cd5d0 2024-05-09T06:36:36.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:36.327+0000 7f415a75a640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f41540cda00 con 0x7f41540cf010 2024-05-09T06:36:36.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:36.327+0000 7f415a75a640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f41540cfbd0 con 0x7f41540afc20 2024-05-09T06:36:36.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:36.327+0000 7f4158cf7640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f41540cd5d0 0x7f415415fd10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:58980/0 (socket says 172.21.15.3:58980) 2024-05-09T06:36:36.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:36.327+0000 7f4158cf7640 1 -- 172.21.15.3:0/1690926894 learned_addr learned my addr 172.21.15.3:0/1690926894 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-05-09T06:36:36.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:36.328+0000 7f41537fe640 1 --2- 172.21.15.3:0/1690926894 >> v2:172.21.15.190:3300/0 conn(0x7f41540cf010 0x7f41540cd090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:36.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:36.328+0000 7f4153fff640 1 --2- 172.21.15.3:0/1690926894 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f41540afc20 0x7f415415d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:36.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:36.328+0000 7f4152ffd640 1 -- 172.21.15.3:0/1690926894 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2923799982 0 0) 0x7f41540cf8f0 con 0x7f41540cd5d0 2024-05-09T06:36:36.309 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:36.328+0000 7f4152ffd640 1 -- 172.21.15.3:0/1690926894 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f413c003520 con 0x7f41540cd5d0 2024-05-09T06:36:36.309 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:36.328+0000 7f41537fe640 1 -- 172.21.15.3:0/1690926894 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f41540afc20 msgr2=0x7f415415d330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:36.309 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:36.328+0000 7f41537fe640 1 --2- 172.21.15.3:0/1690926894 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f41540afc20 0x7f415415d330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:36.309 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:36.328+0000 7f41537fe640 1 -- 172.21.15.3:0/1690926894 >> v1:172.21.15.3:6789/0 conn(0x7f41540cd5d0 legacy=0x7f415415fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:36.309 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:36.328+0000 7f41537fe640 1 -- 172.21.15.3:0/1690926894 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41540cfeb0 con 0x7f41540cf010 2024-05-09T06:36:36.309 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:36.328+0000 7f4153fff640 1 --2- 172.21.15.3:0/1690926894 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f41540afc20 0x7f415415d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T06:36:36.309 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:36.329+0000 7f41537fe640 1 --2- 172.21.15.3:0/1690926894 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f41540cf010 0x7f41540cd090 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f41480097a0 tx=0x7f414802a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=77ba5f797abc88ad server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:36.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:36.329+0000 7f4152ffd640 1 -- 172.21.15.3:0/1690926894 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4148037070 con 0x7f41540cf010 2024-05-09T06:36:36.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:36.329+0000 7f4152ffd640 1 -- 172.21.15.3:0/1690926894 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4148032440 con 0x7f41540cf010 2024-05-09T06:36:36.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:36.329+0000 7f4152ffd640 1 -- 172.21.15.3:0/1690926894 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4148032810 con 0x7f41540cf010 2024-05-09T06:36:36.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:36.330+0000 7f415a75a640 1 -- 172.21.15.3:0/1690926894 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f41540cf010 msgr2=0x7f41540cd090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:36.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:36.330+0000 7f415a75a640 1 --2- 172.21.15.3:0/1690926894 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f41540cf010 0x7f41540cd090 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f41480097a0 tx=0x7f414802a180 comp rx=0 tx=0).stop 2024-05-09T06:36:36.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:36.330+0000 7f415a75a640 1 -- 172.21.15.3:0/1690926894 shutdown_connections 2024-05-09T06:36:36.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:36.330+0000 7f415a75a640 1 --2- 172.21.15.3:0/1690926894 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f41540cf010 0x7f41540cd090 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:36.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:36.330+0000 7f415a75a640 1 --2- 172.21.15.3:0/1690926894 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f41540afc20 0x7f415415d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:36.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:36.330+0000 7f415a75a640 1 -- 172.21.15.3:0/1690926894 >> 172.21.15.3:0/1690926894 conn(0x7f41540d0840 msgr2=0x7f41540d0c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:36.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:36.330+0000 7f415a75a640 1 -- 172.21.15.3:0/1690926894 shutdown_connections 2024-05-09T06:36:36.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:36.330+0000 7f415a75a640 1 -- 172.21.15.3:0/1690926894 wait complete. 2024-05-09T06:36:36.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:36.332+0000 7f415a75a640 1 Processor -- start 2024-05-09T06:36:36.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:36.332+0000 7f415a75a640 1 -- start start 2024-05-09T06:36:36.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:36.333+0000 7f415a75a640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f41540afc20 0x7f4154141730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:36.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:36.333+0000 7f415a75a640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f41540cd5d0 0x7f415413eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:36.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:36.333+0000 7f415a75a640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f41540cf010 0x7f4154140360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:36.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:36.333+0000 7f415a75a640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f41540cf490 con 0x7f41540afc20 2024-05-09T06:36:36.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:36.333+0000 7f415a75a640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f41540b0000 con 0x7f41540cf010 2024-05-09T06:36:36.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:36.333+0000 7f415a75a640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f41540cf610 con 0x7f41540cd5d0 2024-05-09T06:36:36.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:36.333+0000 7f4153fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f41540afc20 0x7f4154141730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:36.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:36.333+0000 7f4153fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f41540afc20 0x7f4154141730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:35618/0 (socket says 172.21.15.3:35618) 2024-05-09T06:36:36.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:36.333+0000 7f4153fff640 1 -- 172.21.15.3:0/2657421369 learned_addr learned my addr 172.21.15.3:0/2657421369 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:36:36.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:36.333+0000 7f41537fe640 1 --2- 172.21.15.3:0/2657421369 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f41540cd5d0 0x7f415413eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:36.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:36.333+0000 7f4158cf7640 1 --2- 172.21.15.3:0/2657421369 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f41540cf010 0x7f4154140360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:36.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:36.333+0000 7f4153fff640 1 -- 172.21.15.3:0/2657421369 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f41540cd5d0 msgr2=0x7f415413eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:36.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:36.333+0000 7f4153fff640 1 --2- 172.21.15.3:0/2657421369 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f41540cd5d0 0x7f415413eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:36.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:36.333+0000 7f4153fff640 1 -- 172.21.15.3:0/2657421369 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f41540cf010 msgr2=0x7f4154140360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:36.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:36.333+0000 7f4153fff640 1 --2- 172.21.15.3:0/2657421369 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f41540cf010 0x7f4154140360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:36.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:36.333+0000 7f4153fff640 1 -- 172.21.15.3:0/2657421369 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f415413f2c0 con 0x7f41540afc20 2024-05-09T06:36:36.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:36.333+0000 7f4153fff640 1 --2- 172.21.15.3:0/2657421369 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f41540afc20 0x7f4154141730 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f414400ebe0 tx=0x7f414400c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:36.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:36.334+0000 7f4150ff9640 1 -- 172.21.15.3:0/2657421369 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4144017070 con 0x7f41540afc20 2024-05-09T06:36:36.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:36.334+0000 7f4150ff9640 1 -- 172.21.15.3:0/2657421369 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f414400cc30 con 0x7f41540afc20 2024-05-09T06:36:36.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:36.334+0000 7f415a75a640 1 -- 172.21.15.3:0/2657421369 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f415413f5a0 con 0x7f41540afc20 2024-05-09T06:36:36.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:36.334+0000 7f4150ff9640 1 -- 172.21.15.3:0/2657421369 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4144013470 con 0x7f41540afc20 2024-05-09T06:36:36.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:36.334+0000 7f415a75a640 1 -- 172.21.15.3:0/2657421369 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f415413fb60 con 0x7f41540afc20 2024-05-09T06:36:36.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:36.335+0000 7f415a75a640 1 -- 172.21.15.3:0/2657421369 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f415404f660 con 0x7f41540afc20 2024-05-09T06:36:36.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:36.335+0000 7f4150ff9640 1 -- 172.21.15.3:0/2657421369 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f414401a070 con 0x7f41540afc20 2024-05-09T06:36:36.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:36.336+0000 7f4150ff9640 1 --2- 172.21.15.3:0/2657421369 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f414006be30 0x7f414006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:36.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:36.336+0000 7f4150ff9640 1 -- 172.21.15.3:0/2657421369 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f414408b020 con 0x7f41540afc20 2024-05-09T06:36:36.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:36.336+0000 7f41537fe640 1 --2- 172.21.15.3:0/2657421369 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f414006be30 0x7f414006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:36.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:36.337+0000 7f41537fe640 1 --2- 172.21.15.3:0/2657421369 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f414006be30 0x7f414006e2f0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f4148034000 tx=0x7f414802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:36.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:36.340+0000 7f4150ff9640 1 -- 172.21.15.3:0/2657421369 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f4144059410 con 0x7f41540afc20 2024-05-09T06:36:36.576 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:36.596+0000 7f415a75a640 1 -- 172.21.15.3:0/2657421369 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestPerModuleFinsherThread.test_volumes_module_finisher_thread"]} v 0) -- 0x7f41540b1dc0 con 0x7f41540afc20 2024-05-09T06:36:36.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:36.989+0000 7f4150ff9640 1 -- 172.21.15.3:0/2657421369 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestPerModuleFinsherThread.test_volumes_module_finisher_thread"]}]=0 v54) ==== 170+0+0 (secure 0 0 0) 0x7f414405d710 con 0x7f41540afc20 2024-05-09T06:36:36.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:36.991+0000 7f415a75a640 1 -- 172.21.15.3:0/2657421369 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f414006be30 msgr2=0x7f414006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:36.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:36.991+0000 7f415a75a640 1 --2- 172.21.15.3:0/2657421369 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f414006be30 0x7f414006e2f0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f4148034000 tx=0x7f414802e000 comp rx=0 tx=0).stop 2024-05-09T06:36:36.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:36.991+0000 7f415a75a640 1 -- 172.21.15.3:0/2657421369 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f41540afc20 msgr2=0x7f4154141730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:36.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:36.991+0000 7f415a75a640 1 --2- 172.21.15.3:0/2657421369 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f41540afc20 0x7f4154141730 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f414400ebe0 tx=0x7f414400c520 comp rx=0 tx=0).stop 2024-05-09T06:36:36.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:36.991+0000 7f415a75a640 1 -- 172.21.15.3:0/2657421369 shutdown_connections 2024-05-09T06:36:36.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:36.991+0000 7f415a75a640 1 --2- 172.21.15.3:0/2657421369 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f414006be30 0x7f414006e2f0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:36.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:36.991+0000 7f415a75a640 1 --2- 172.21.15.3:0/2657421369 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f41540cf010 0x7f4154140360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:36.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:36.991+0000 7f415a75a640 1 --2- 172.21.15.3:0/2657421369 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f41540cd5d0 0x7f415413eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:36.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:36.991+0000 7f415a75a640 1 --2- 172.21.15.3:0/2657421369 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f41540afc20 0x7f4154141730 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:36.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:36.991+0000 7f415a75a640 1 -- 172.21.15.3:0/2657421369 >> 172.21.15.3:0/2657421369 conn(0x7f41540d0840 msgr2=0x7f415406bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:36.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:36.991+0000 7f415a75a640 1 -- 172.21.15.3:0/2657421369 shutdown_connections 2024-05-09T06:36:36.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:36.991+0000 7f415a75a640 1 -- 172.21.15.3:0/2657421369 wait complete. 2024-05-09T06:36:36.982 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-05-09T06:36:37.090 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:37.110+0000 7f2344f14640 1 Processor -- start 2024-05-09T06:36:37.090 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:37.110+0000 7f2344f14640 1 -- start start 2024-05-09T06:36:37.091 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:37.111+0000 7f2344f14640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f234015e5c0 0x7f23400d2800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:37.091 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:37.111+0000 7f2344f14640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f23400d4c40 0x7f23400d0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:37.091 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:37.111+0000 7f2344f14640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f23400d1130 0x7f23400ced30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:37.091 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:37.111+0000 7f2344f14640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f234015e9a0 con 0x7f23400d4c40 2024-05-09T06:36:37.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:37.111+0000 7f2344f14640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f23400d31c0 con 0x7f23400d1130 2024-05-09T06:36:37.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:37.111+0000 7f2344f14640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f23400d50a0 con 0x7f234015e5c0 2024-05-09T06:36:37.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:37.111+0000 7f233dd9b640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f23400d4c40 0x7f23400d0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:37.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:37.111+0000 7f233dd9b640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f23400d4c40 0x7f23400d0bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:35628/0 (socket says 172.21.15.3:35628) 2024-05-09T06:36:37.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:37.111+0000 7f233dd9b640 1 -- 172.21.15.3:0/2069498767 learned_addr learned my addr 172.21.15.3:0/2069498767 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:36:37.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:37.111+0000 7f233ed9d640 1 --2- 172.21.15.3:0/2069498767 >> v2:172.21.15.190:3300/0 conn(0x7f23400d1130 0x7f23400ced30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:37.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:37.111+0000 7f233e59c640 1 --2- 172.21.15.3:0/2069498767 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f234015e5c0 0x7f23400d2800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:37.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:37.111+0000 7f233dd9b640 1 -- 172.21.15.3:0/2069498767 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f234015e5c0 msgr2=0x7f23400d2800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:37.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:37.111+0000 7f233dd9b640 1 --2- 172.21.15.3:0/2069498767 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f234015e5c0 0x7f23400d2800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:37.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:37.111+0000 7f233dd9b640 1 -- 172.21.15.3:0/2069498767 >> v2:172.21.15.190:3300/0 conn(0x7f23400d1130 msgr2=0x7f23400ced30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:37.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:37.111+0000 7f233dd9b640 1 --2- 172.21.15.3:0/2069498767 >> v2:172.21.15.190:3300/0 conn(0x7f23400d1130 0x7f23400ced30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:37.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:37.111+0000 7f233dd9b640 1 -- 172.21.15.3:0/2069498767 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23400d2ee0 con 0x7f23400d4c40 2024-05-09T06:36:37.093 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:37.111+0000 7f233ed9d640 1 --2- 172.21.15.3:0/2069498767 >> v2:172.21.15.190:3300/0 conn(0x7f23400d1130 0x7f23400ced30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:36:37.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:37.112+0000 7f233dd9b640 1 --2- 172.21.15.3:0/2069498767 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f23400d4c40 0x7f23400d0bf0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f23400cb250 tx=0x7f233402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5497c3309cc19ae3 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:37.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:37.112+0000 7f233d59a640 1 -- 172.21.15.3:0/2069498767 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2334037070 con 0x7f23400d4c40 2024-05-09T06:36:37.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:37.112+0000 7f233d59a640 1 -- 172.21.15.3:0/2069498767 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2334032440 con 0x7f23400d4c40 2024-05-09T06:36:37.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:37.112+0000 7f233d59a640 1 -- 172.21.15.3:0/2069498767 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2334032810 con 0x7f23400d4c40 2024-05-09T06:36:37.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:37.113+0000 7f2344f14640 1 -- 172.21.15.3:0/2069498767 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f23400d4c40 msgr2=0x7f23400d0bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:37.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:37.113+0000 7f2344f14640 1 --2- 172.21.15.3:0/2069498767 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f23400d4c40 0x7f23400d0bf0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f23400cb250 tx=0x7f233402a1e0 comp rx=0 tx=0).stop 2024-05-09T06:36:37.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:37.113+0000 7f2344f14640 1 -- 172.21.15.3:0/2069498767 shutdown_connections 2024-05-09T06:36:37.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:37.113+0000 7f2344f14640 1 --2- 172.21.15.3:0/2069498767 >> v2:172.21.15.190:3300/0 conn(0x7f23400d1130 0x7f23400ced30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:37.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:37.113+0000 7f2344f14640 1 --2- 172.21.15.3:0/2069498767 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f23400d4c40 0x7f23400d0bf0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:37.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:37.113+0000 7f2344f14640 1 --2- 172.21.15.3:0/2069498767 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f234015e5c0 0x7f23400d2800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:37.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:37.113+0000 7f2344f14640 1 -- 172.21.15.3:0/2069498767 >> 172.21.15.3:0/2069498767 conn(0x7f234009d500 msgr2=0x7f23400d4220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:37.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:37.113+0000 7f2344f14640 1 -- 172.21.15.3:0/2069498767 shutdown_connections 2024-05-09T06:36:37.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:37.113+0000 7f2344f14640 1 -- 172.21.15.3:0/2069498767 wait complete. 2024-05-09T06:36:37.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:37.114+0000 7f2344f14640 1 Processor -- start 2024-05-09T06:36:37.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:37.115+0000 7f2344f14640 1 -- start start 2024-05-09T06:36:37.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:37.115+0000 7f2344f14640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f23400d1130 0x7f23401f4650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:37.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:37.115+0000 7f2344f14640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f23400d4c40 0x7f23400cbf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:37.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:37.115+0000 7f2344f14640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f234015e5c0 0x7f23400b6e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:37.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:37.115+0000 7f2344f14640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f23400b42c0 con 0x7f23400d1130 2024-05-09T06:36:37.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:37.115+0000 7f2344f14640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f23400b3fe0 con 0x7f23400d4c40 2024-05-09T06:36:37.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:37.115+0000 7f2344f14640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f23400cf270 con 0x7f234015e5c0 2024-05-09T06:36:37.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:37.115+0000 7f233e59c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f23400d1130 0x7f23401f4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:37.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:37.115+0000 7f233e59c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f23400d1130 0x7f23401f4650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:35642/0 (socket says 172.21.15.3:35642) 2024-05-09T06:36:37.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:37.115+0000 7f233e59c640 1 -- 172.21.15.3:0/140028349 learned_addr learned my addr 172.21.15.3:0/140028349 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:36:37.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:37.116+0000 7f233ed9d640 1 --2- 172.21.15.3:0/140028349 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f234015e5c0 0x7f23400b6e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:37.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:37.116+0000 7f233dd9b640 1 --2- 172.21.15.3:0/140028349 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f23400d4c40 0x7f23400cbf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:37.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:37.116+0000 7f233e59c640 1 -- 172.21.15.3:0/140028349 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f234015e5c0 msgr2=0x7f23400b6e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:37.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:37.116+0000 7f233e59c640 1 --2- 172.21.15.3:0/140028349 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f234015e5c0 0x7f23400b6e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:37.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:37.116+0000 7f233e59c640 1 -- 172.21.15.3:0/140028349 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f23400d4c40 msgr2=0x7f23400cbf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:37.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:37.116+0000 7f233e59c640 1 --2- 172.21.15.3:0/140028349 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f23400d4c40 0x7f23400cbf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:37.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:37.116+0000 7f233e59c640 1 -- 172.21.15.3:0/140028349 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23401f4b90 con 0x7f23400d1130 2024-05-09T06:36:37.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:37.116+0000 7f233e59c640 1 --2- 172.21.15.3:0/140028349 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f23400d1130 0x7f23401f4650 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f233000c7f0 tx=0x7f233000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:37.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:37.116+0000 7f232effd640 1 -- 172.21.15.3:0/140028349 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2330015070 con 0x7f23400d1130 2024-05-09T06:36:37.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:37.116+0000 7f232effd640 1 -- 172.21.15.3:0/140028349 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2330004220 con 0x7f23400d1130 2024-05-09T06:36:37.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:37.116+0000 7f2344f14640 1 -- 172.21.15.3:0/140028349 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23400cb490 con 0x7f23400d1130 2024-05-09T06:36:37.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:37.116+0000 7f232effd640 1 -- 172.21.15.3:0/140028349 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2330007620 con 0x7f23400d1130 2024-05-09T06:36:37.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:37.116+0000 7f2344f14640 1 -- 172.21.15.3:0/140028349 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23400cb730 con 0x7f23400d1130 2024-05-09T06:36:37.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:37.117+0000 7f2344f14640 1 -- 172.21.15.3:0/140028349 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2340069550 con 0x7f23400d1130 2024-05-09T06:36:37.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:37.118+0000 7f232effd640 1 -- 172.21.15.3:0/140028349 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f23300079d0 con 0x7f23400d1130 2024-05-09T06:36:37.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:37.118+0000 7f232effd640 1 --2- 172.21.15.3:0/140028349 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f232006be30 0x7f232006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:37.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:37.118+0000 7f232effd640 1 -- 172.21.15.3:0/140028349 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f233008b1c0 con 0x7f23400d1130 2024-05-09T06:36:37.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:37.118+0000 7f233dd9b640 1 --2- 172.21.15.3:0/140028349 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f232006be30 0x7f232006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:37.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:37.119+0000 7f233dd9b640 1 --2- 172.21.15.3:0/140028349 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f232006be30 0x7f232006e2f0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f2334034000 tx=0x7f233402e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:37.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:37.123+0000 7f232effd640 1 -- 172.21.15.3:0/140028349 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f23401f4b90 con 0x7f23400d1130 2024-05-09T06:36:37.285 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:37.305+0000 7f2344f14640 1 -- 172.21.15.3:0/140028349 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f23400b9070 con 0x7f23400d1130 2024-05-09T06:36:37.288 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:37.308+0000 7f232effd640 1 -- 172.21.15.3:0/140028349 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) ==== 122+0+0 (secure 0 0 0) 0x7f2330059af0 con 0x7f23400d1130 2024-05-09T06:36:37.290 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:37.310+0000 7f2344f14640 1 -- 172.21.15.3:0/140028349 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f232006be30 msgr2=0x7f232006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:37.290 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:37.310+0000 7f2344f14640 1 --2- 172.21.15.3:0/140028349 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f232006be30 0x7f232006e2f0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f2334034000 tx=0x7f233402e000 comp rx=0 tx=0).stop 2024-05-09T06:36:37.291 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:37.310+0000 7f2344f14640 1 -- 172.21.15.3:0/140028349 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f23400d1130 msgr2=0x7f23401f4650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:37.291 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:37.310+0000 7f2344f14640 1 --2- 172.21.15.3:0/140028349 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f23400d1130 0x7f23401f4650 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f233000c7f0 tx=0x7f233000ccb0 comp rx=0 tx=0).stop 2024-05-09T06:36:37.291 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:37.310+0000 7f2344f14640 1 -- 172.21.15.3:0/140028349 shutdown_connections 2024-05-09T06:36:37.291 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:37.310+0000 7f2344f14640 1 --2- 172.21.15.3:0/140028349 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f232006be30 0x7f232006e2f0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:37.291 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:37.310+0000 7f2344f14640 1 --2- 172.21.15.3:0/140028349 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f234015e5c0 0x7f23400b6e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:37.291 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:37.310+0000 7f2344f14640 1 --2- 172.21.15.3:0/140028349 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f23400d4c40 0x7f23400cbf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:37.291 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:37.310+0000 7f2344f14640 1 --2- 172.21.15.3:0/140028349 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f23400d1130 0x7f23401f4650 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:37.291 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:37.310+0000 7f2344f14640 1 -- 172.21.15.3:0/140028349 >> 172.21.15.3:0/140028349 conn(0x7f234009d500 msgr2=0x7f234015d870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:37.291 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:37.310+0000 7f2344f14640 1 -- 172.21.15.3:0/140028349 shutdown_connections 2024-05-09T06:36:37.291 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:37.310+0000 7f2344f14640 1 -- 172.21.15.3:0/140028349 wait complete. 2024-05-09T06:36:37.301 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-05-09T06:36:37.302 DEBUG:teuthology.orchestra.run.smithi003:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-09T06:36:37.336 DEBUG:teuthology.orchestra.run.smithi190:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-09T06:36:37.378 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-05-09T06:36:37.382 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-05-09T06:36:37.382 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T06:36:37.399 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-05-09T06:36:37.399 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T06:36:37.456 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-05-09T06:36:37.456 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T06:36:37.514 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T06:36:37.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:37.678+0000 7fe2478f5640 1 Processor -- start 2024-05-09T06:36:37.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:37.678+0000 7fe2478f5640 1 -- start start 2024-05-09T06:36:37.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:37.679+0000 7fe2478f5640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe240154cc0 0x7fe24015ee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:37.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:37.679+0000 7fe2478f5640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe2400cfc80 con 0x7fe24015b6f0 2024-05-09T06:36:37.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:37.679+0000 7fe2478f5640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe24015e390 con 0x7fe24006e400 2024-05-09T06:36:37.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:37.679+0000 7fe2478f5640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fe2400cf9a0 con 0x7fe240154cc0 2024-05-09T06:36:37.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:37.679+0000 7fe244e90640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fe24015b6f0 0x7fe24006dcf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:58990/0 (socket says 172.21.15.3:58990) 2024-05-09T06:36:37.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:37.679+0000 7fe244e90640 1 -- 172.21.15.3:0/182341384 learned_addr learned my addr 172.21.15.3:0/182341384 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-05-09T06:36:37.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:37.679+0000 7fe245691640 1 --2- 172.21.15.3:0/182341384 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe240154cc0 0x7fe24015ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:37.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:37.679+0000 7fe237fff640 1 -- 172.21.15.3:0/182341384 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4020789097 0 0) 0x7fe2400cfc80 con 0x7fe24015b6f0 2024-05-09T06:36:37.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:37.679+0000 7fe237fff640 1 -- 172.21.15.3:0/182341384 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe22c003520 con 0x7fe24015b6f0 2024-05-09T06:36:37.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:37.680+0000 7fe237fff640 1 -- 172.21.15.3:0/182341384 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3632402669 0 0) 0x7fe24015e390 con 0x7fe24006e400 2024-05-09T06:36:37.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:37.680+0000 7fe237fff640 1 -- 172.21.15.3:0/182341384 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe2400cfc80 con 0x7fe24006e400 2024-05-09T06:36:37.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:37.680+0000 7fe237fff640 1 -- 172.21.15.3:0/182341384 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3739456307 0 0) 0x7fe22c003520 con 0x7fe24015b6f0 2024-05-09T06:36:37.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:37.680+0000 7fe237fff640 1 -- 172.21.15.3:0/182341384 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe240154cc0 msgr2=0x7fe24015ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:37.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:37.680+0000 7fe237fff640 1 --2- 172.21.15.3:0/182341384 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe240154cc0 0x7fe24015ee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:37.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:37.680+0000 7fe237fff640 1 -- 172.21.15.3:0/182341384 >> v1:172.21.15.190:6789/0 conn(0x7fe24006e400 legacy=0x7fe2400cdeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:37.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:37.680+0000 7fe237fff640 1 -- 172.21.15.3:0/182341384 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe24015e630 con 0x7fe24015b6f0 2024-05-09T06:36:37.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:37.680+0000 7fe237fff640 1 -- 172.21.15.3:0/182341384 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 563312031 0 0) 0x7fe230001670 con 0x7fe24015b6f0 2024-05-09T06:36:37.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:37.680+0000 7fe237fff640 1 -- 172.21.15.3:0/182341384 >> v1:172.21.15.3:6789/0 conn(0x7fe24015b6f0 legacy=0x7fe24006dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:37.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:37.680+0000 7fe237fff640 1 --2- 172.21.15.3:0/182341384 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe22c003b70 0x7fe22c003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:37.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:37.680+0000 7fe237fff640 1 --2- 172.21.15.3:0/182341384 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe240154cc0 0x7fe22c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:37.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:37.680+0000 7fe237fff640 1 --2- 172.21.15.3:0/182341384 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe22c004d40 0x7fe22c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:37.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:37.680+0000 7fe237fff640 1 -- 172.21.15.3:0/182341384 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fe2400cf9a0 con 0x7fe22c003b70 2024-05-09T06:36:37.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:37.680+0000 7fe237fff640 1 -- 172.21.15.3:0/182341384 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fe22c00dcd0 con 0x7fe22c004d40 2024-05-09T06:36:37.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:37.680+0000 7fe237fff640 1 -- 172.21.15.3:0/182341384 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fe22c00ded0 con 0x7fe240154cc0 2024-05-09T06:36:37.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:37.680+0000 7fe245691640 1 --2- 172.21.15.3:0/182341384 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe22c003b70 0x7fe22c003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:37.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:37.680+0000 7fe244e90640 1 --2- 172.21.15.3:0/182341384 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe240154cc0 0x7fe22c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:37.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:37.680+0000 7fe245e92640 1 --2- 172.21.15.3:0/182341384 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe22c004d40 0x7fe22c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:37.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:37.680+0000 7fe245691640 1 -- 172.21.15.3:0/182341384 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe240154cc0 msgr2=0x7fe22c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:37.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:37.680+0000 7fe245691640 1 --2- 172.21.15.3:0/182341384 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe240154cc0 0x7fe22c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:37.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:37.680+0000 7fe245691640 1 -- 172.21.15.3:0/182341384 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe22c004d40 msgr2=0x7fe22c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:37.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:37.680+0000 7fe245691640 1 --2- 172.21.15.3:0/182341384 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe22c004d40 0x7fe22c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:37.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:37.680+0000 7fe245691640 1 -- 172.21.15.3:0/182341384 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe22c003520 con 0x7fe22c003b70 2024-05-09T06:36:37.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:37.680+0000 7fe245e92640 1 --2- 172.21.15.3:0/182341384 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe22c004d40 0x7fe22c0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:36:37.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:37.681+0000 7fe245691640 1 --2- 172.21.15.3:0/182341384 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe22c003b70 0x7fe22c003f70 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fe23c00aa40 tx=0x7fe22c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=705ab1f9634bda1d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:37.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:37.681+0000 7fe237fff640 1 -- 172.21.15.3:0/182341384 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe23c036070 con 0x7fe22c003b70 2024-05-09T06:36:37.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:37.681+0000 7fe237fff640 1 -- 172.21.15.3:0/182341384 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe23c031600 con 0x7fe22c003b70 2024-05-09T06:36:37.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:37.681+0000 7fe237fff640 1 -- 172.21.15.3:0/182341384 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe23c00b040 con 0x7fe22c003b70 2024-05-09T06:36:37.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:37.681+0000 7fe2478f5640 1 -- 172.21.15.3:0/182341384 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe22c003b70 msgr2=0x7fe22c003f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:37.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:37.681+0000 7fe2478f5640 1 --2- 172.21.15.3:0/182341384 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe22c003b70 0x7fe22c003f70 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fe23c00aa40 tx=0x7fe22c005830 comp rx=0 tx=0).stop 2024-05-09T06:36:37.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:37.681+0000 7fe245e92640 1 -- 172.21.15.3:0/182341384 reap_dead start 2024-05-09T06:36:37.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:37.681+0000 7fe2478f5640 1 -- 172.21.15.3:0/182341384 shutdown_connections 2024-05-09T06:36:37.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:37.681+0000 7fe2478f5640 1 -- 172.21.15.3:0/182341384 >> 172.21.15.3:0/182341384 conn(0x7fe24006b430 msgr2=0x7fe24006b810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:37.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:37.682+0000 7fe2478f5640 1 -- 172.21.15.3:0/182341384 shutdown_connections 2024-05-09T06:36:37.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:37.682+0000 7fe2478f5640 1 -- 172.21.15.3:0/182341384 wait complete. 2024-05-09T06:36:37.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:37.683+0000 7fe2478f5640 1 Processor -- start 2024-05-09T06:36:37.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:37.683+0000 7fe2478f5640 1 -- start start 2024-05-09T06:36:37.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:37.683+0000 7fe2478f5640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe22c003b70 0x7fe24014d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:37.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:37.683+0000 7fe2478f5640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe240148970 0x7fe2401599e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:37.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:37.683+0000 7fe2478f5640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe240159f20 0x7fe2401f4290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:37.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:37.684+0000 7fe2478f5640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fe2401550a0 con 0x7fe240159f20 2024-05-09T06:36:37.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:37.684+0000 7fe2478f5640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fe2400cf690 con 0x7fe22c003b70 2024-05-09T06:36:37.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:37.684+0000 7fe2478f5640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fe24015bb70 con 0x7fe240148970 2024-05-09T06:36:37.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:37.684+0000 7fe245e92640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe240159f20 0x7fe2401f4290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:37.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:37.684+0000 7fe245e92640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe240159f20 0x7fe2401f4290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:35666/0 (socket says 172.21.15.3:35666) 2024-05-09T06:36:37.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:37.684+0000 7fe245e92640 1 -- 172.21.15.3:0/4117952418 learned_addr learned my addr 172.21.15.3:0/4117952418 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:36:37.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:37.684+0000 7fe245691640 1 --2- 172.21.15.3:0/4117952418 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe22c003b70 0x7fe24014d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:37.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:37.684+0000 7fe244e90640 1 --2- 172.21.15.3:0/4117952418 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe240148970 0x7fe2401599e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:37.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:37.684+0000 7fe245e92640 1 -- 172.21.15.3:0/4117952418 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe240148970 msgr2=0x7fe2401599e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:37.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:37.684+0000 7fe245e92640 1 --2- 172.21.15.3:0/4117952418 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe240148970 0x7fe2401599e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:37.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:37.684+0000 7fe245e92640 1 -- 172.21.15.3:0/4117952418 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe22c003b70 msgr2=0x7fe24014d190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:37.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:37.684+0000 7fe245e92640 1 --2- 172.21.15.3:0/4117952418 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe22c003b70 0x7fe24014d190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:37.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:37.684+0000 7fe245e92640 1 -- 172.21.15.3:0/4117952418 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe240148df0 con 0x7fe240159f20 2024-05-09T06:36:37.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:37.684+0000 7fe245e92640 1 --2- 172.21.15.3:0/4117952418 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe240159f20 0x7fe2401f4290 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fe240073500 tx=0x7fe23800dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:37.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:37.684+0000 7fe235ffb640 1 -- 172.21.15.3:0/4117952418 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe238018070 con 0x7fe240159f20 2024-05-09T06:36:37.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:37.684+0000 7fe235ffb640 1 -- 172.21.15.3:0/4117952418 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe238003b70 con 0x7fe240159f20 2024-05-09T06:36:37.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:37.684+0000 7fe2478f5640 1 -- 172.21.15.3:0/4117952418 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2401f5230 con 0x7fe240159f20 2024-05-09T06:36:37.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:37.684+0000 7fe235ffb640 1 -- 172.21.15.3:0/4117952418 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe238005070 con 0x7fe240159f20 2024-05-09T06:36:37.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:37.685+0000 7fe2478f5640 1 -- 172.21.15.3:0/4117952418 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe24014c6b0 con 0x7fe240159f20 2024-05-09T06:36:37.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:37.686+0000 7fe2478f5640 1 -- 172.21.15.3:0/4117952418 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe24015a3a0 con 0x7fe240159f20 2024-05-09T06:36:37.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:37.686+0000 7fe235ffb640 1 -- 172.21.15.3:0/4117952418 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe238005210 con 0x7fe240159f20 2024-05-09T06:36:37.670 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:37.686+0000 7fe235ffb640 1 --2- 172.21.15.3:0/4117952418 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fe22406be60 0x7fe22406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:37.670 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:37.687+0000 7fe235ffb640 1 -- 172.21.15.3:0/4117952418 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fe23808a4f0 con 0x7fe240159f20 2024-05-09T06:36:37.670 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:37.687+0000 7fe245691640 1 --2- 172.21.15.3:0/4117952418 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fe22406be60 0x7fe22406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:37.670 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:37.687+0000 7fe245691640 1 --2- 172.21.15.3:0/4117952418 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fe22406be60 0x7fe22406e320 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fe23c033000 tx=0x7fe23c008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:37.670 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:37.691+0000 7fe235ffb640 1 -- 172.21.15.3:0/4117952418 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fe238058900 con 0x7fe240159f20 2024-05-09T06:36:37.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:37.875+0000 7fe2478f5640 1 -- 172.21.15.3:0/4117952418 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe24004f660 con 0x7fe240159f20 2024-05-09T06:36:37.856 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:37.877+0000 7fe235ffb640 1 -- 172.21.15.3:0/4117952418 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16521 (secure 0 0 0) 0x7fe23805cc00 con 0x7fe240159f20 2024-05-09T06:36:37.857 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-09T06:36:37.857 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":20,"fsid":"16ed737d-2665-431a-8ac6-ef242be3975a","created":"2024-05-09T06:35:42.541368+0000","modified":"2024-05-09T06:36:08.856391+0000","last_up_change":"2024-05-09T06:35:53.778274+0000","last_in_change":"2024-05-09T06:35:44.780586+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T06:35:54.819999+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T06:36:04.931314+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T06:36:06.200433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fae3a8db-4826-4f3c-b33a-12c09bb5df6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":1897849827},{"type":"v1","addr":"172.21.15.3:6817","nonce":1897849827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":1897849827},{"type":"v1","addr":"172.21.15.3:6819","nonce":1897849827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":1897849827},{"type":"v1","addr":"172.21.15.3:6823","nonce":1897849827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":1897849827},{"type":"v1","addr":"172.21.15.3:6821","nonce":1897849827}]},"public_addr":"172.21.15.3:6817/1897849827","cluster_addr":"172.21.15.3:6819/1897849827","heartbeat_back_addr":"172.21.15.3:6823/1897849827","heartbeat_front_addr":"172.21.15.3:6821/1897849827","state":["exists","up"]},{"osd":1,"uuid":"646e882d-4ce4-4720-8c84-f71df563a35e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":926805498},{"type":"v1","addr":"172.21.15.3:6801","nonce":926805498}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":926805498},{"type":"v1","addr":"172.21.15.3:6803","nonce":926805498}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":926805498},{"type":"v1","addr":"172.21.15.3:6807","nonce":926805498}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":926805498},{"type":"v1","addr":"172.21.15.3:6805","nonce":926805498}]},"public_addr":"172.21.15.3:6801/926805498","cluster_addr":"172.21.15.3:6803/926805498","heartbeat_back_addr":"172.21.15.3:6807/926805498","heartbeat_front_addr":"172.21.15.3:6805/926805498","state":["exists","up"]},{"osd":2,"uuid":"1abbadb5-a570-4359-9643-acff3ede35a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":801295451},{"type":"v1","addr":"172.21.15.3:6809","nonce":801295451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":801295451},{"type":"v1","addr":"172.21.15.3:6811","nonce":801295451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":801295451},{"type":"v1","addr":"172.21.15.3:6815","nonce":801295451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":801295451},{"type":"v1","addr":"172.21.15.3:6813","nonce":801295451}]},"public_addr":"172.21.15.3:6809/801295451","cluster_addr":"172.21.15.3:6811/801295451","heartbeat_back_addr":"172.21.15.3:6815/801295451","heartbeat_front_addr":"172.21.15.3:6813/801295451","state":["exists","up"]},{"osd":3,"uuid":"62ac4dc1-2371-4dd2-acd3-e2dbb11aedf5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":1903389857},{"type":"v1","addr":"172.21.15.3:6825","nonce":1903389857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":1903389857},{"type":"v1","addr":"172.21.15.3:6827","nonce":1903389857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":1903389857},{"type":"v1","addr":"172.21.15.3:6831","nonce":1903389857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":1903389857},{"type":"v1","addr":"172.21.15.3:6829","nonce":1903389857}]},"public_addr":"172.21.15.3:6825/1903389857","cluster_addr":"172.21.15.3:6827/1903389857","heartbeat_back_addr":"172.21.15.3:6831/1903389857","heartbeat_front_addr":"172.21.15.3:6829/1903389857","state":["exists","up"]},{"osd":4,"uuid":"ae808398-8326-4d7c-9319-a2a0eafffb59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":2143624576},{"type":"v1","addr":"172.21.15.190:6801","nonce":2143624576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":2143624576},{"type":"v1","addr":"172.21.15.190:6803","nonce":2143624576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":2143624576},{"type":"v1","addr":"172.21.15.190:6807","nonce":2143624576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":2143624576},{"type":"v1","addr":"172.21.15.190:6805","nonce":2143624576}]},"public_addr":"172.21.15.190:6801/2143624576","cluster_addr":"172.21.15.190:6803/2143624576","heartbeat_back_addr":"172.21.15.190:6807/2143624576","heartbeat_front_addr":"172.21.15.190:6805/2143624576","state":["exists","up"]},{"osd":5,"uuid":"918adf50-c440-48ac-9a8d-4837562d7275","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":1766923126},{"type":"v1","addr":"172.21.15.190:6814","nonce":1766923126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":1766923126},{"type":"v1","addr":"172.21.15.190:6818","nonce":1766923126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":1766923126},{"type":"v1","addr":"172.21.15.190:6823","nonce":1766923126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":1766923126},{"type":"v1","addr":"172.21.15.190:6821","nonce":1766923126}]},"public_addr":"172.21.15.190:6814/1766923126","cluster_addr":"172.21.15.190:6818/1766923126","heartbeat_back_addr":"172.21.15.190:6823/1766923126","heartbeat_front_addr":"172.21.15.190:6821/1766923126","state":["exists","up"]},{"osd":6,"uuid":"ac2882bc-2a7c-463c-8f6c-76ff49828adc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":342414248},{"type":"v1","addr":"172.21.15.190:6825","nonce":342414248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":342414248},{"type":"v1","addr":"172.21.15.190:6827","nonce":342414248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":342414248},{"type":"v1","addr":"172.21.15.190:6831","nonce":342414248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":342414248},{"type":"v1","addr":"172.21.15.190:6829","nonce":342414248}]},"public_addr":"172.21.15.190:6825/342414248","cluster_addr":"172.21.15.190:6827/342414248","heartbeat_back_addr":"172.21.15.190:6831/342414248","heartbeat_front_addr":"172.21.15.190:6829/342414248","state":["exists","up"]},{"osd":7,"uuid":"dd4e767a-4730-4c70-8bd5-3f088140a5a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":537849891},{"type":"v1","addr":"172.21.15.190:6809","nonce":537849891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":537849891},{"type":"v1","addr":"172.21.15.190:6811","nonce":537849891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6817","nonce":537849891},{"type":"v1","addr":"172.21.15.190:6819","nonce":537849891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6813","nonce":537849891},{"type":"v1","addr":"172.21.15.190:6815","nonce":537849891}]},"public_addr":"172.21.15.190:6809/537849891","cluster_addr":"172.21.15.190:6811/537849891","heartbeat_back_addr":"172.21.15.190:6819/537849891","heartbeat_front_addr":"172.21.15.190:6815/537849891","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.889784+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.974002+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.973549+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.955396+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.928471+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.902418+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.958696+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.926474+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T06:36:37.859 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:37.879+0000 7fe2478f5640 1 -- 172.21.15.3:0/4117952418 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fe22406be60 msgr2=0x7fe22406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:37.860 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:37.879+0000 7fe2478f5640 1 --2- 172.21.15.3:0/4117952418 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fe22406be60 0x7fe22406e320 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fe23c033000 tx=0x7fe23c008030 comp rx=0 tx=0).stop 2024-05-09T06:36:37.860 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:37.879+0000 7fe2478f5640 1 -- 172.21.15.3:0/4117952418 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe240159f20 msgr2=0x7fe2401f4290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:37.860 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:37.879+0000 7fe2478f5640 1 --2- 172.21.15.3:0/4117952418 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe240159f20 0x7fe2401f4290 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fe240073500 tx=0x7fe23800dc00 comp rx=0 tx=0).stop 2024-05-09T06:36:37.860 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:37.879+0000 7fe2478f5640 1 -- 172.21.15.3:0/4117952418 shutdown_connections 2024-05-09T06:36:37.860 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:37.879+0000 7fe2478f5640 1 --2- 172.21.15.3:0/4117952418 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fe22406be60 0x7fe22406e320 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:37.860 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:37.879+0000 7fe2478f5640 1 --2- 172.21.15.3:0/4117952418 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe240159f20 0x7fe2401f4290 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:37.860 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:37.879+0000 7fe2478f5640 1 --2- 172.21.15.3:0/4117952418 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe240148970 0x7fe2401599e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:37.860 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:37.879+0000 7fe2478f5640 1 --2- 172.21.15.3:0/4117952418 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe22c003b70 0x7fe24014d190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:37.860 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:37.879+0000 7fe2478f5640 1 -- 172.21.15.3:0/4117952418 >> 172.21.15.3:0/4117952418 conn(0x7fe24006b430 msgr2=0x7fe24015c9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:37.860 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:37.879+0000 7fe2478f5640 1 -- 172.21.15.3:0/4117952418 shutdown_connections 2024-05-09T06:36:37.860 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:37.879+0000 7fe2478f5640 1 -- 172.21.15.3:0/4117952418 wait complete. 2024-05-09T06:36:37.870 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-09T06:35:54.819999+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-09T06:36:04.931314+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-09T06:36:06.200433+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-09T06:36:37.871 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-09T06:36:37.978 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:37.999+0000 7fb2d89ee640 1 Processor -- start 2024-05-09T06:36:37.978 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:37.999+0000 7fb2d89ee640 1 -- start start 2024-05-09T06:36:37.980 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:37.999+0000 7fb2d89ee640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fb2d415e5c0 0x7fb2d40d2800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:37.980 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:37.999+0000 7fb2d89ee640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fb2d40d4c40 0x7fb2d40d0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:37.980 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:37.999+0000 7fb2d89ee640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb2d415e9a0 con 0x7fb2d40d1130 2024-05-09T06:36:37.980 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.000+0000 7fb2d89ee640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fb2d40d50d0 con 0x7fb2d415e5c0 2024-05-09T06:36:37.980 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.000+0000 7fb2d89ee640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7fb2d40d31c0 con 0x7fb2d40d4c40 2024-05-09T06:36:37.980 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.000+0000 7fb2d259c640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fb2d415e5c0 0x7fb2d40d2800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:37.980 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.000+0000 7fb2d2d9d640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fb2d40d1130 0x7fb2d40ced30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:58994/0 (socket says 172.21.15.3:58994) 2024-05-09T06:36:37.980 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.000+0000 7fb2d259c640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fb2d415e5c0 0x7fb2d40d2800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:35680/0 (socket says 172.21.15.3:35680) 2024-05-09T06:36:37.980 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.000+0000 7fb2d2d9d640 1 -- 172.21.15.3:0/1712501758 learned_addr learned my addr 172.21.15.3:0/1712501758 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-05-09T06:36:37.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.000+0000 7fb2d1d9b640 1 --2- 172.21.15.3:0/1712501758 >> v2:172.21.15.190:3300/0 conn(0x7fb2d40d4c40 0x7fb2d40d0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:37.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.000+0000 7fb2d259c640 1 -- 172.21.15.3:0/1712501758 >> v2:172.21.15.190:3300/0 conn(0x7fb2d40d4c40 msgr2=0x7fb2d40d0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:37.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.000+0000 7fb2d259c640 1 --2- 172.21.15.3:0/1712501758 >> v2:172.21.15.190:3300/0 conn(0x7fb2d40d4c40 0x7fb2d40d0bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:37.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.000+0000 7fb2d259c640 1 -- 172.21.15.3:0/1712501758 >> v1:172.21.15.3:6789/0 conn(0x7fb2d40d1130 legacy=0x7fb2d40ced30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:37.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.000+0000 7fb2d159a640 1 -- 172.21.15.3:0/1712501758 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 954250026 0 0) 0x7fb2d415e9a0 con 0x7fb2d40d1130 2024-05-09T06:36:37.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.000+0000 7fb2d259c640 1 -- 172.21.15.3:0/1712501758 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2d40d2ee0 con 0x7fb2d415e5c0 2024-05-09T06:36:37.984 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.001+0000 7fb2d259c640 1 --2- 172.21.15.3:0/1712501758 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb2d415e5c0 0x7fb2d40d2800 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fb2d408cb10 tx=0x7fb2cc02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=547f580e16dffb51 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:37.985 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.001+0000 7fb2d159a640 1 -- 172.21.15.3:0/1712501758 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2cc037070 con 0x7fb2d415e5c0 2024-05-09T06:36:37.985 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.001+0000 7fb2d159a640 1 -- 172.21.15.3:0/1712501758 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb2cc032440 con 0x7fb2d415e5c0 2024-05-09T06:36:37.985 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.001+0000 7fb2d159a640 1 -- 172.21.15.3:0/1712501758 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2cc032810 con 0x7fb2d415e5c0 2024-05-09T06:36:37.985 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.001+0000 7fb2d89ee640 1 -- 172.21.15.3:0/1712501758 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb2d415e5c0 msgr2=0x7fb2d40d2800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:37.985 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.001+0000 7fb2d89ee640 1 --2- 172.21.15.3:0/1712501758 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb2d415e5c0 0x7fb2d40d2800 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fb2d408cb10 tx=0x7fb2cc02a1a0 comp rx=0 tx=0).stop 2024-05-09T06:36:37.985 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.001+0000 7fb2d89ee640 1 -- 172.21.15.3:0/1712501758 shutdown_connections 2024-05-09T06:36:37.985 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.001+0000 7fb2d89ee640 1 --2- 172.21.15.3:0/1712501758 >> v2:172.21.15.190:3300/0 conn(0x7fb2d40d4c40 0x7fb2d40d0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:37.985 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.001+0000 7fb2d89ee640 1 --2- 172.21.15.3:0/1712501758 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb2d415e5c0 0x7fb2d40d2800 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:37.985 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.001+0000 7fb2d89ee640 1 -- 172.21.15.3:0/1712501758 >> 172.21.15.3:0/1712501758 conn(0x7fb2d409d500 msgr2=0x7fb2d40d4220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:37.985 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.002+0000 7fb2d89ee640 1 -- 172.21.15.3:0/1712501758 shutdown_connections 2024-05-09T06:36:37.985 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.002+0000 7fb2d89ee640 1 -- 172.21.15.3:0/1712501758 wait complete. 2024-05-09T06:36:37.985 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.003+0000 7fb2d89ee640 1 Processor -- start 2024-05-09T06:36:37.985 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.003+0000 7fb2d89ee640 1 -- start start 2024-05-09T06:36:37.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.004+0000 7fb2d89ee640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb2d40d1130 0x7fb2d41f70a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:37.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.004+0000 7fb2d89ee640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb2d40d4c40 0x7fb2d415f310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:37.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.004+0000 7fb2d89ee640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb2d415e5c0 0x7fb2d40cb2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:37.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.004+0000 7fb2d89ee640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fb2d40b42c0 con 0x7fb2d40d1130 2024-05-09T06:36:37.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.004+0000 7fb2d89ee640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fb2d40b3fe0 con 0x7fb2d415e5c0 2024-05-09T06:36:37.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.004+0000 7fb2d89ee640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fb2d40b57f0 con 0x7fb2d40d4c40 2024-05-09T06:36:37.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.004+0000 7fb2d259c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb2d40d1130 0x7fb2d41f70a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:37.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.004+0000 7fb2d259c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb2d40d1130 0x7fb2d41f70a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:35684/0 (socket says 172.21.15.3:35684) 2024-05-09T06:36:37.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.004+0000 7fb2d259c640 1 -- 172.21.15.3:0/599300384 learned_addr learned my addr 172.21.15.3:0/599300384 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:36:37.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.004+0000 7fb2d2d9d640 1 --2- 172.21.15.3:0/599300384 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb2d415e5c0 0x7fb2d40cb2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:37.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.004+0000 7fb2d1d9b640 1 --2- 172.21.15.3:0/599300384 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb2d40d4c40 0x7fb2d415f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:37.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.004+0000 7fb2d259c640 1 -- 172.21.15.3:0/599300384 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb2d40d4c40 msgr2=0x7fb2d415f310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:37.987 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.004+0000 7fb2d259c640 1 --2- 172.21.15.3:0/599300384 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb2d40d4c40 0x7fb2d415f310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:37.987 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.004+0000 7fb2d259c640 1 -- 172.21.15.3:0/599300384 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb2d415e5c0 msgr2=0x7fb2d40cb2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:37.987 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.004+0000 7fb2d259c640 1 --2- 172.21.15.3:0/599300384 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb2d415e5c0 0x7fb2d40cb2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:37.987 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.004+0000 7fb2d259c640 1 -- 172.21.15.3:0/599300384 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2d415fb90 con 0x7fb2d40d1130 2024-05-09T06:36:37.987 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.004+0000 7fb2d2d9d640 1 --2- 172.21.15.3:0/599300384 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb2d415e5c0 0x7fb2d40cb2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:36:37.987 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.004+0000 7fb2d259c640 1 --2- 172.21.15.3:0/599300384 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb2d40d1130 0x7fb2d41f70a0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fb2cc000c00 tx=0x7fb2cc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:37.987 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.004+0000 7fb2caffd640 1 -- 172.21.15.3:0/599300384 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2cc037040 con 0x7fb2d40d1130 2024-05-09T06:36:37.987 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.004+0000 7fb2caffd640 1 -- 172.21.15.3:0/599300384 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb2cc042070 con 0x7fb2d40d1130 2024-05-09T06:36:37.987 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.005+0000 7fb2d89ee640 1 -- 172.21.15.3:0/599300384 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2d415fe70 con 0x7fb2d40d1130 2024-05-09T06:36:37.987 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.005+0000 7fb2caffd640 1 -- 172.21.15.3:0/599300384 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2cc030bd0 con 0x7fb2d40d1130 2024-05-09T06:36:37.987 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.005+0000 7fb2d89ee640 1 -- 172.21.15.3:0/599300384 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2d4160430 con 0x7fb2d40d1130 2024-05-09T06:36:37.988 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.005+0000 7fb2d89ee640 1 -- 172.21.15.3:0/599300384 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2d40734b0 con 0x7fb2d40d1130 2024-05-09T06:36:37.988 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.006+0000 7fb2caffd640 1 -- 172.21.15.3:0/599300384 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb2cc047460 con 0x7fb2d40d1130 2024-05-09T06:36:37.989 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.006+0000 7fb2caffd640 1 --2- 172.21.15.3:0/599300384 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fb2b406be30 0x7fb2b406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:37.989 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.006+0000 7fb2caffd640 1 -- 172.21.15.3:0/599300384 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fb2cc0ab9c0 con 0x7fb2d40d1130 2024-05-09T06:36:37.989 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.007+0000 7fb2d1d9b640 1 --2- 172.21.15.3:0/599300384 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fb2b406be30 0x7fb2b406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:37.989 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.007+0000 7fb2d1d9b640 1 --2- 172.21.15.3:0/599300384 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fb2b406be30 0x7fb2b406e2f0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fb2d40d2330 tx=0x7fb2bc008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:37.990 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.011+0000 7fb2caffd640 1 -- 172.21.15.3:0/599300384 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fb2d40734b0 con 0x7fb2d40d1130 2024-05-09T06:36:38.174 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.193+0000 7fb2d89ee640 1 -- 172.21.15.3:0/599300384 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fb2d4069550 con 0x7fb2d40d1130 2024-05-09T06:36:38.174 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.194+0000 7fb2caffd640 1 -- 172.21.15.3:0/599300384 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7fb2cc0ade20 con 0x7fb2d40d1130 2024-05-09T06:36:38.174 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 1 2024-05-09T06:36:38.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.196+0000 7fb2d89ee640 1 -- 172.21.15.3:0/599300384 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fb2b406be30 msgr2=0x7fb2b406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:38.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.196+0000 7fb2d89ee640 1 --2- 172.21.15.3:0/599300384 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fb2b406be30 0x7fb2b406e2f0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fb2d40d2330 tx=0x7fb2bc008040 comp rx=0 tx=0).stop 2024-05-09T06:36:38.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.196+0000 7fb2d89ee640 1 -- 172.21.15.3:0/599300384 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb2d40d1130 msgr2=0x7fb2d41f70a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:38.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.196+0000 7fb2d89ee640 1 --2- 172.21.15.3:0/599300384 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb2d40d1130 0x7fb2d41f70a0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fb2cc000c00 tx=0x7fb2cc0046c0 comp rx=0 tx=0).stop 2024-05-09T06:36:38.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.196+0000 7fb2d89ee640 1 -- 172.21.15.3:0/599300384 shutdown_connections 2024-05-09T06:36:38.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.196+0000 7fb2d89ee640 1 --2- 172.21.15.3:0/599300384 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fb2b406be30 0x7fb2b406e2f0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:38.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.196+0000 7fb2d89ee640 1 --2- 172.21.15.3:0/599300384 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb2d415e5c0 0x7fb2d40cb2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:38.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.196+0000 7fb2d89ee640 1 --2- 172.21.15.3:0/599300384 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb2d40d4c40 0x7fb2d415f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:38.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.196+0000 7fb2d89ee640 1 --2- 172.21.15.3:0/599300384 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb2d40d1130 0x7fb2d41f70a0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:38.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.196+0000 7fb2d89ee640 1 -- 172.21.15.3:0/599300384 >> 172.21.15.3:0/599300384 conn(0x7fb2d409d500 msgr2=0x7fb2d415d800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:38.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.196+0000 7fb2d89ee640 1 -- 172.21.15.3:0/599300384 shutdown_connections 2024-05-09T06:36:38.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.196+0000 7fb2d89ee640 1 -- 172.21.15.3:0/599300384 wait complete. 2024-05-09T06:36:38.187 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-09T06:36:38.294 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.315+0000 7f14f6112640 1 Processor -- start 2024-05-09T06:36:38.295 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.315+0000 7f14f6112640 1 -- start start 2024-05-09T06:36:38.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.316+0000 7f14f6112640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f14f015e490 0x7f14f0162590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:38.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.316+0000 7f14f6112640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f14f006b250 0x7f14f006b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:38.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.316+0000 7f14f6112640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f14f006cad0 con 0x7f14f015e0b0 2024-05-09T06:36:38.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.316+0000 7f14f6112640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f14f00b3eb0 con 0x7f14f015e490 2024-05-09T06:36:38.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.316+0000 7f14f6112640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f14f00b4190 con 0x7f14f006b250 2024-05-09T06:36:38.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.316+0000 7f14ef7fe640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f14f015e0b0 0x7f14f0161e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:59006/0 (socket says 172.21.15.3:59006) 2024-05-09T06:36:38.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.316+0000 7f14ef7fe640 1 -- 172.21.15.3:0/1261891800 learned_addr learned my addr 172.21.15.3:0/1261891800 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-05-09T06:36:38.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.316+0000 7f14eeffd640 1 --2- 172.21.15.3:0/1261891800 >> v2:172.21.15.190:3300/0 conn(0x7f14f015e490 0x7f14f0162590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:38.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.316+0000 7f14effff640 1 --2- 172.21.15.3:0/1261891800 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f14f006b250 0x7f14f006b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:38.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.316+0000 7f14ee7fc640 1 -- 172.21.15.3:0/1261891800 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1797453856 0 0) 0x7f14f006cad0 con 0x7f14f015e0b0 2024-05-09T06:36:38.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.316+0000 7f14ee7fc640 1 -- 172.21.15.3:0/1261891800 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f14d8003520 con 0x7f14f015e0b0 2024-05-09T06:36:38.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.316+0000 7f14ee7fc640 1 -- 172.21.15.3:0/1261891800 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3719889672 0 0) 0x7f14d8003520 con 0x7f14f015e0b0 2024-05-09T06:36:38.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.316+0000 7f14ee7fc640 1 -- 172.21.15.3:0/1261891800 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f14f006b250 msgr2=0x7f14f006b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:38.301 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.316+0000 7f14ee7fc640 1 --2- 172.21.15.3:0/1261891800 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f14f006b250 0x7f14f006b6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:38.301 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.316+0000 7f14ee7fc640 1 -- 172.21.15.3:0/1261891800 >> v2:172.21.15.190:3300/0 conn(0x7f14f015e490 msgr2=0x7f14f0162590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:38.301 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.316+0000 7f14ee7fc640 1 --2- 172.21.15.3:0/1261891800 >> v2:172.21.15.190:3300/0 conn(0x7f14f015e490 0x7f14f0162590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:38.301 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.316+0000 7f14ee7fc640 1 -- 172.21.15.3:0/1261891800 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14f00b4470 con 0x7f14f015e0b0 2024-05-09T06:36:38.301 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.316+0000 7f14ee7fc640 1 -- 172.21.15.3:0/1261891800 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 563312031 0 0) 0x7f14e0001670 con 0x7f14f015e0b0 2024-05-09T06:36:38.301 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.316+0000 7f14ee7fc640 1 -- 172.21.15.3:0/1261891800 >> v1:172.21.15.3:6789/0 conn(0x7f14f015e0b0 legacy=0x7f14f0161e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:38.301 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.316+0000 7f14ee7fc640 1 --2- 172.21.15.3:0/1261891800 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f14d80039b0 0x7f14d8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:38.301 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.316+0000 7f14ee7fc640 1 --2- 172.21.15.3:0/1261891800 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f14f006b250 0x7f14d8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:38.301 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.316+0000 7f14ee7fc640 1 --2- 172.21.15.3:0/1261891800 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f14d8004d40 0x7f14d80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:38.301 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.316+0000 7f14ee7fc640 1 -- 172.21.15.3:0/1261891800 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f14f00b3eb0 con 0x7f14d80039b0 2024-05-09T06:36:38.301 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.316+0000 7f14ee7fc640 1 -- 172.21.15.3:0/1261891800 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f14f00b4190 con 0x7f14d8004d40 2024-05-09T06:36:38.301 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.316+0000 7f14ee7fc640 1 -- 172.21.15.3:0/1261891800 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f14d800b9c0 con 0x7f14f006b250 2024-05-09T06:36:38.301 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.317+0000 7f14ef7fe640 1 --2- 172.21.15.3:0/1261891800 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f14d80039b0 0x7f14d8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:38.301 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.317+0000 7f14eeffd640 1 --2- 172.21.15.3:0/1261891800 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f14f006b250 0x7f14d8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:38.301 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.317+0000 7f14effff640 1 --2- 172.21.15.3:0/1261891800 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f14d8004d40 0x7f14d80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:38.302 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.317+0000 7f14ef7fe640 1 -- 172.21.15.3:0/1261891800 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f14f006b250 msgr2=0x7f14d8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:38.302 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.317+0000 7f14ef7fe640 1 --2- 172.21.15.3:0/1261891800 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f14f006b250 0x7f14d8004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:38.302 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.317+0000 7f14ef7fe640 1 -- 172.21.15.3:0/1261891800 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f14d8004d40 msgr2=0x7f14d80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:38.302 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.317+0000 7f14ef7fe640 1 --2- 172.21.15.3:0/1261891800 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f14d8004d40 0x7f14d80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:38.302 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.317+0000 7f14ef7fe640 1 -- 172.21.15.3:0/1261891800 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14d8003520 con 0x7f14d80039b0 2024-05-09T06:36:38.302 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.318+0000 7f14ef7fe640 1 --2- 172.21.15.3:0/1261891800 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f14d80039b0 0x7f14d8003dd0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f14e0007fd0 tx=0x7f14d8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=248e0265bc474aa2 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:38.302 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.318+0000 7f14ee7fc640 1 -- 172.21.15.3:0/1261891800 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14e0007d50 con 0x7f14d80039b0 2024-05-09T06:36:38.302 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.318+0000 7f14ee7fc640 1 -- 172.21.15.3:0/1261891800 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14e0005ad0 con 0x7f14d80039b0 2024-05-09T06:36:38.302 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.318+0000 7f14ee7fc640 1 -- 172.21.15.3:0/1261891800 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14e0007070 con 0x7f14d80039b0 2024-05-09T06:36:38.302 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.318+0000 7f14f6112640 1 -- 172.21.15.3:0/1261891800 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f14d80039b0 msgr2=0x7f14d8003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:38.302 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.318+0000 7f14f6112640 1 --2- 172.21.15.3:0/1261891800 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f14d80039b0 0x7f14d8003dd0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f14e0007fd0 tx=0x7f14d8005830 comp rx=0 tx=0).stop 2024-05-09T06:36:38.302 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.318+0000 7f14effff640 1 -- 172.21.15.3:0/1261891800 reap_dead start 2024-05-09T06:36:38.302 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.318+0000 7f14f6112640 1 -- 172.21.15.3:0/1261891800 shutdown_connections 2024-05-09T06:36:38.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.318+0000 7f14f6112640 1 -- 172.21.15.3:0/1261891800 >> 172.21.15.3:0/1261891800 conn(0x7f14f00d07b0 msgr2=0x7f14f00d0bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:38.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.318+0000 7f14f6112640 1 -- 172.21.15.3:0/1261891800 shutdown_connections 2024-05-09T06:36:38.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.318+0000 7f14f6112640 1 -- 172.21.15.3:0/1261891800 wait complete. 2024-05-09T06:36:38.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.320+0000 7f14f6112640 1 Processor -- start 2024-05-09T06:36:38.305 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.320+0000 7f14f6112640 1 -- start start 2024-05-09T06:36:38.305 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.320+0000 7f14f6112640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f14d80039b0 0x7f14f0141720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:38.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.320+0000 7f14f6112640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f14f015b430 0x7f14f0160960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:38.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.321+0000 7f14f6112640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f14f015b8b0 0x7f14f0160ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:38.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.321+0000 7f14f6112640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f14f006da70 con 0x7f14f015b8b0 2024-05-09T06:36:38.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.321+0000 7f14f6112640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f14f006d8f0 con 0x7f14d80039b0 2024-05-09T06:36:38.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.321+0000 7f14f6112640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f14f0162ad0 con 0x7f14f015b430 2024-05-09T06:36:38.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.321+0000 7f14effff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f14f015b8b0 0x7f14f0160ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:38.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.321+0000 7f14effff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f14f015b8b0 0x7f14f0160ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:35712/0 (socket says 172.21.15.3:35712) 2024-05-09T06:36:38.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.321+0000 7f14effff640 1 -- 172.21.15.3:0/1675446556 learned_addr learned my addr 172.21.15.3:0/1675446556 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:36:38.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.321+0000 7f14ef7fe640 1 --2- 172.21.15.3:0/1675446556 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f14d80039b0 0x7f14f0141720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:38.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.321+0000 7f14eeffd640 1 --2- 172.21.15.3:0/1675446556 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f14f015b430 0x7f14f0160960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:38.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.321+0000 7f14effff640 1 -- 172.21.15.3:0/1675446556 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f14f015b430 msgr2=0x7f14f0160960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:38.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.321+0000 7f14effff640 1 --2- 172.21.15.3:0/1675446556 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f14f015b430 0x7f14f0160960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:38.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.321+0000 7f14effff640 1 -- 172.21.15.3:0/1675446556 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f14d80039b0 msgr2=0x7f14f0141720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:38.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.321+0000 7f14effff640 1 --2- 172.21.15.3:0/1675446556 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f14d80039b0 0x7f14f0141720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:38.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.321+0000 7f14effff640 1 -- 172.21.15.3:0/1675446556 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14f0141fc0 con 0x7f14f015b8b0 2024-05-09T06:36:38.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.321+0000 7f14effff640 1 --2- 172.21.15.3:0/1675446556 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f14f015b8b0 0x7f14f0160ea0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f14f00b3c60 tx=0x7f14e800f1b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:38.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.321+0000 7f14d7fff640 1 -- 172.21.15.3:0/1675446556 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14e800fd10 con 0x7f14f015b8b0 2024-05-09T06:36:38.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.321+0000 7f14d7fff640 1 -- 172.21.15.3:0/1675446556 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14e801d070 con 0x7f14f015b8b0 2024-05-09T06:36:38.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.321+0000 7f14d7fff640 1 -- 172.21.15.3:0/1675446556 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14e8015670 con 0x7f14f015b8b0 2024-05-09T06:36:38.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.321+0000 7f14f6112640 1 -- 172.21.15.3:0/1675446556 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14f0142260 con 0x7f14f015b8b0 2024-05-09T06:36:38.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.321+0000 7f14f6112640 1 -- 172.21.15.3:0/1675446556 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14f0142820 con 0x7f14f015b8b0 2024-05-09T06:36:38.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.322+0000 7f14f6112640 1 -- 172.21.15.3:0/1675446556 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14f00b3a80 con 0x7f14f015b8b0 2024-05-09T06:36:38.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.323+0000 7f14d7fff640 1 -- 172.21.15.3:0/1675446556 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f14e8015810 con 0x7f14f015b8b0 2024-05-09T06:36:38.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.323+0000 7f14d7fff640 1 --2- 172.21.15.3:0/1675446556 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f14dc06be60 0x7f14dc06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:38.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.323+0000 7f14d7fff640 1 -- 172.21.15.3:0/1675446556 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f14e808b710 con 0x7f14f015b8b0 2024-05-09T06:36:38.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.324+0000 7f14ef7fe640 1 --2- 172.21.15.3:0/1675446556 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f14dc06be60 0x7f14dc06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:38.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.324+0000 7f14ef7fe640 1 --2- 172.21.15.3:0/1675446556 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f14dc06be60 0x7f14dc06e320 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f14e0033000 tx=0x7f14e0009030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:38.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.328+0000 7f14d7fff640 1 -- 172.21.15.3:0/1675446556 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f14f00b3a80 con 0x7f14f015b8b0 2024-05-09T06:36:38.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.509+0000 7f14f6112640 1 -- 172.21.15.3:0/1675446556 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f14f00b3c90 con 0x7f14f015b8b0 2024-05-09T06:36:38.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.509+0000 7f14d7fff640 1 -- 172.21.15.3:0/1675446556 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7f14e8010040 con 0x7f14f015b8b0 2024-05-09T06:36:38.489 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 32 2024-05-09T06:36:38.491 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.511+0000 7f14f6112640 1 -- 172.21.15.3:0/1675446556 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f14dc06be60 msgr2=0x7f14dc06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:38.491 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.511+0000 7f14f6112640 1 --2- 172.21.15.3:0/1675446556 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f14dc06be60 0x7f14dc06e320 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f14e0033000 tx=0x7f14e0009030 comp rx=0 tx=0).stop 2024-05-09T06:36:38.491 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.511+0000 7f14f6112640 1 -- 172.21.15.3:0/1675446556 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f14f015b8b0 msgr2=0x7f14f0160ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:38.491 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.511+0000 7f14f6112640 1 --2- 172.21.15.3:0/1675446556 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f14f015b8b0 0x7f14f0160ea0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f14f00b3c60 tx=0x7f14e800f1b0 comp rx=0 tx=0).stop 2024-05-09T06:36:38.491 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.511+0000 7f14f6112640 1 -- 172.21.15.3:0/1675446556 shutdown_connections 2024-05-09T06:36:38.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.511+0000 7f14f6112640 1 --2- 172.21.15.3:0/1675446556 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f14dc06be60 0x7f14dc06e320 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:38.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.511+0000 7f14f6112640 1 --2- 172.21.15.3:0/1675446556 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f14f015b8b0 0x7f14f0160ea0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:38.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.511+0000 7f14f6112640 1 --2- 172.21.15.3:0/1675446556 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f14f015b430 0x7f14f0160960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:38.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.511+0000 7f14f6112640 1 --2- 172.21.15.3:0/1675446556 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f14d80039b0 0x7f14f0141720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:38.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.511+0000 7f14f6112640 1 -- 172.21.15.3:0/1675446556 >> 172.21.15.3:0/1675446556 conn(0x7f14f00d07b0 msgr2=0x7f14f006bec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:38.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.511+0000 7f14f6112640 1 -- 172.21.15.3:0/1675446556 shutdown_connections 2024-05-09T06:36:38.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.511+0000 7f14f6112640 1 -- 172.21.15.3:0/1675446556 wait complete. 2024-05-09T06:36:38.502 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-09T06:36:38.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.630+0000 7ff7f88f5640 1 Processor -- start 2024-05-09T06:36:38.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.630+0000 7ff7f88f5640 1 -- start start 2024-05-09T06:36:38.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.631+0000 7ff7f88f5640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff7f40b4e30 0x7ff7f406e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:38.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.631+0000 7ff7f88f5640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7ff7f406e790 0x7ff7f415c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:38.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.631+0000 7ff7f88f5640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff7f40b40f0 con 0x7ff7f415c0f0 2024-05-09T06:36:38.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.631+0000 7ff7f88f5640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7ff7f406ebc0 con 0x7ff7f40b4e30 2024-05-09T06:36:38.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.631+0000 7ff7f88f5640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7ff7f40b3050 con 0x7ff7f406e790 2024-05-09T06:36:38.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.631+0000 7ff7ebfff640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff7f40b4e30 0x7ff7f406e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:38.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.631+0000 7ff7ebfff640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff7f40b4e30 0x7ff7f406e250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:35714/0 (socket says 172.21.15.3:35714) 2024-05-09T06:36:38.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.631+0000 7ff7ebfff640 1 -- 172.21.15.3:0/2471447302 learned_addr learned my addr 172.21.15.3:0/2471447302 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:36:38.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.631+0000 7ff7f2d9d640 1 --2- 172.21.15.3:0/2471447302 >> v2:172.21.15.190:3300/0 conn(0x7ff7f406e790 0x7ff7f415c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:38.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.631+0000 7ff7ebfff640 1 -- 172.21.15.3:0/2471447302 >> v2:172.21.15.190:3300/0 conn(0x7ff7f406e790 msgr2=0x7ff7f415c4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:38.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.631+0000 7ff7ebfff640 1 --2- 172.21.15.3:0/2471447302 >> v2:172.21.15.190:3300/0 conn(0x7ff7f406e790 0x7ff7f415c4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:38.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.631+0000 7ff7ebfff640 1 -- 172.21.15.3:0/2471447302 >> v1:172.21.15.190:6789/0 conn(0x7ff7f415c0f0 legacy=0x7ff7f4160730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:38.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.631+0000 7ff7ebfff640 1 -- 172.21.15.3:0/2471447302 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7f40b3330 con 0x7ff7f40b4e30 2024-05-09T06:36:38.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.632+0000 7ff7ebfff640 1 --2- 172.21.15.3:0/2471447302 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff7f40b4e30 0x7ff7f406e250 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7ff7f4070bf0 tx=0x7ff7dc02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2782721d2c1a7f35 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:38.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.632+0000 7ff7f1d9b640 1 -- 172.21.15.3:0/2471447302 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7dc037070 con 0x7ff7f40b4e30 2024-05-09T06:36:38.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.632+0000 7ff7f1d9b640 1 -- 172.21.15.3:0/2471447302 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff7dc032440 con 0x7ff7f40b4e30 2024-05-09T06:36:38.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.632+0000 7ff7f1d9b640 1 -- 172.21.15.3:0/2471447302 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7dc032810 con 0x7ff7f40b4e30 2024-05-09T06:36:38.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.633+0000 7ff7f88f5640 1 -- 172.21.15.3:0/2471447302 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff7f40b4e30 msgr2=0x7ff7f406e250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:38.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.633+0000 7ff7f88f5640 1 --2- 172.21.15.3:0/2471447302 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff7f40b4e30 0x7ff7f406e250 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7ff7f4070bf0 tx=0x7ff7dc02a1a0 comp rx=0 tx=0).stop 2024-05-09T06:36:38.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.633+0000 7ff7f88f5640 1 -- 172.21.15.3:0/2471447302 shutdown_connections 2024-05-09T06:36:38.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.633+0000 7ff7f88f5640 1 --2- 172.21.15.3:0/2471447302 >> v2:172.21.15.190:3300/0 conn(0x7ff7f406e790 0x7ff7f415c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:38.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.633+0000 7ff7f88f5640 1 --2- 172.21.15.3:0/2471447302 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff7f40b4e30 0x7ff7f406e250 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:38.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.633+0000 7ff7f88f5640 1 -- 172.21.15.3:0/2471447302 >> 172.21.15.3:0/2471447302 conn(0x7ff7f40b1c20 msgr2=0x7ff7f40b2030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:38.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.633+0000 7ff7f88f5640 1 -- 172.21.15.3:0/2471447302 shutdown_connections 2024-05-09T06:36:38.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.633+0000 7ff7f88f5640 1 -- 172.21.15.3:0/2471447302 wait complete. 2024-05-09T06:36:38.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.634+0000 7ff7f88f5640 1 Processor -- start 2024-05-09T06:36:38.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.634+0000 7ff7f88f5640 1 -- start start 2024-05-09T06:36:38.617 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.635+0000 7ff7f88f5640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff7f406e790 0x7ff7f4140860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:38.617 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.635+0000 7ff7f88f5640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff7f40b4e30 0x7ff7f4154450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:38.617 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.635+0000 7ff7f88f5640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff7f415c0f0 0x7ff7f41596e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:38.617 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.635+0000 7ff7f88f5640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7ff7f40b3bd0 con 0x7ff7f406e790 2024-05-09T06:36:38.617 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.635+0000 7ff7f88f5640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7ff7f40b38f0 con 0x7ff7f40b4e30 2024-05-09T06:36:38.617 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.635+0000 7ff7f88f5640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7ff7f415ca10 con 0x7ff7f415c0f0 2024-05-09T06:36:38.617 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.635+0000 7ff7f259c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff7f406e790 0x7ff7f4140860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:38.617 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.635+0000 7ff7f259c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff7f406e790 0x7ff7f4140860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:35726/0 (socket says 172.21.15.3:35726) 2024-05-09T06:36:38.617 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.635+0000 7ff7f259c640 1 -- 172.21.15.3:0/3492921531 learned_addr learned my addr 172.21.15.3:0/3492921531 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:36:38.617 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.635+0000 7ff7ebfff640 1 --2- 172.21.15.3:0/3492921531 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff7f40b4e30 0x7ff7f4154450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:38.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.635+0000 7ff7f2d9d640 1 --2- 172.21.15.3:0/3492921531 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff7f415c0f0 0x7ff7f41596e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:38.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.635+0000 7ff7f259c640 1 -- 172.21.15.3:0/3492921531 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff7f415c0f0 msgr2=0x7ff7f41596e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:38.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.635+0000 7ff7f259c640 1 --2- 172.21.15.3:0/3492921531 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff7f415c0f0 0x7ff7f41596e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:38.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.635+0000 7ff7f259c640 1 -- 172.21.15.3:0/3492921531 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff7f40b4e30 msgr2=0x7ff7f4154450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:38.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.635+0000 7ff7f259c640 1 --2- 172.21.15.3:0/3492921531 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff7f40b4e30 0x7ff7f4154450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:38.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.635+0000 7ff7f259c640 1 -- 172.21.15.3:0/3492921531 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7f4154e50 con 0x7ff7f406e790 2024-05-09T06:36:38.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.635+0000 7ff7f2d9d640 1 --2- 172.21.15.3:0/3492921531 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff7f415c0f0 0x7ff7f41596e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:36:38.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.635+0000 7ff7f259c640 1 --2- 172.21.15.3:0/3492921531 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff7f406e790 0x7ff7f4140860 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7ff7e400b420 tx=0x7ff7e400b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:38.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.636+0000 7ff7eaffd640 1 -- 172.21.15.3:0/3492921531 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7e4011830 con 0x7ff7f406e790 2024-05-09T06:36:38.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.636+0000 7ff7eaffd640 1 -- 172.21.15.3:0/3492921531 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff7e4016070 con 0x7ff7f406e790 2024-05-09T06:36:38.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.636+0000 7ff7f88f5640 1 -- 172.21.15.3:0/3492921531 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7f4155130 con 0x7ff7f406e790 2024-05-09T06:36:38.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.636+0000 7ff7eaffd640 1 -- 172.21.15.3:0/3492921531 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7e4011c10 con 0x7ff7f406e790 2024-05-09T06:36:38.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.636+0000 7ff7f88f5640 1 -- 172.21.15.3:0/3492921531 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7f41556f0 con 0x7ff7f406e790 2024-05-09T06:36:38.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.637+0000 7ff7f88f5640 1 -- 172.21.15.3:0/3492921531 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7f4069550 con 0x7ff7f406e790 2024-05-09T06:36:38.619 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.637+0000 7ff7eaffd640 1 -- 172.21.15.3:0/3492921531 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff7e401b460 con 0x7ff7f406e790 2024-05-09T06:36:38.619 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.638+0000 7ff7eaffd640 1 --2- 172.21.15.3:0/3492921531 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7ff7d406be30 0x7ff7d406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:38.619 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.638+0000 7ff7eaffd640 1 -- 172.21.15.3:0/3492921531 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7ff7e408ab60 con 0x7ff7f406e790 2024-05-09T06:36:38.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.638+0000 7ff7ebfff640 1 --2- 172.21.15.3:0/3492921531 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7ff7d406be30 0x7ff7d406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:38.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.639+0000 7ff7ebfff640 1 --2- 172.21.15.3:0/3492921531 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7ff7d406be30 0x7ff7d406e2f0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7ff7dc034000 tx=0x7ff7dc02e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:38.622 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.642+0000 7ff7eaffd640 1 -- 172.21.15.3:0/3492921531 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7ff7e4059510 con 0x7ff7f406e790 2024-05-09T06:36:38.805 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.825+0000 7ff7f88f5640 1 -- 172.21.15.3:0/3492921531 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7ff7f415f820 con 0x7ff7f406e790 2024-05-09T06:36:38.805 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.825+0000 7ff7eaffd640 1 -- 172.21.15.3:0/3492921531 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7ff7e4058e10 con 0x7ff7f406e790 2024-05-09T06:36:38.805 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 64 2024-05-09T06:36:38.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.828+0000 7ff7f88f5640 1 -- 172.21.15.3:0/3492921531 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7ff7d406be30 msgr2=0x7ff7d406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:38.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.828+0000 7ff7f88f5640 1 --2- 172.21.15.3:0/3492921531 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7ff7d406be30 0x7ff7d406e2f0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7ff7dc034000 tx=0x7ff7dc02e000 comp rx=0 tx=0).stop 2024-05-09T06:36:38.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.828+0000 7ff7f88f5640 1 -- 172.21.15.3:0/3492921531 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff7f406e790 msgr2=0x7ff7f4140860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:38.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.828+0000 7ff7f88f5640 1 --2- 172.21.15.3:0/3492921531 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff7f406e790 0x7ff7f4140860 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7ff7e400b420 tx=0x7ff7e400b8e0 comp rx=0 tx=0).stop 2024-05-09T06:36:38.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.828+0000 7ff7f88f5640 1 -- 172.21.15.3:0/3492921531 shutdown_connections 2024-05-09T06:36:38.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.828+0000 7ff7f88f5640 1 --2- 172.21.15.3:0/3492921531 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7ff7d406be30 0x7ff7d406e2f0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:38.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.828+0000 7ff7f88f5640 1 --2- 172.21.15.3:0/3492921531 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff7f415c0f0 0x7ff7f41596e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:38.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.828+0000 7ff7f88f5640 1 --2- 172.21.15.3:0/3492921531 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff7f40b4e30 0x7ff7f4154450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:38.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.828+0000 7ff7f88f5640 1 --2- 172.21.15.3:0/3492921531 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff7f406e790 0x7ff7f4140860 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:38.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.828+0000 7ff7f88f5640 1 -- 172.21.15.3:0/3492921531 >> 172.21.15.3:0/3492921531 conn(0x7ff7f40b1c20 msgr2=0x7ff7f415a540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:38.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.828+0000 7ff7f88f5640 1 -- 172.21.15.3:0/3492921531 shutdown_connections 2024-05-09T06:36:38.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.828+0000 7ff7f88f5640 1 -- 172.21.15.3:0/3492921531 wait complete. 2024-05-09T06:36:38.820 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-09T06:36:38.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.948+0000 7fd5ec0a5640 1 Processor -- start 2024-05-09T06:36:38.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.948+0000 7fd5ec0a5640 1 -- start start 2024-05-09T06:36:38.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.949+0000 7fd5ec0a5640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fd5e40b4e30 0x7fd5e406e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:38.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.949+0000 7fd5ec0a5640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fd5e406e790 0x7fd5e415c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:38.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.949+0000 7fd5ec0a5640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd5e40b40f0 con 0x7fd5e415c0f0 2024-05-09T06:36:38.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.949+0000 7fd5ec0a5640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fd5e406ebc0 con 0x7fd5e40b4e30 2024-05-09T06:36:38.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.949+0000 7fd5ec0a5640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7fd5e40b3050 con 0x7fd5e406e790 2024-05-09T06:36:38.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.949+0000 7fd5e9e41640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fd5e415c0f0 0x7fd5e4160730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:59016/0 (socket says 172.21.15.3:59016) 2024-05-09T06:36:38.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.949+0000 7fd5e9e41640 1 -- 172.21.15.3:0/2178809685 learned_addr learned my addr 172.21.15.3:0/2178809685 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-05-09T06:36:38.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.949+0000 7fd5e9640640 1 --2- 172.21.15.3:0/2178809685 >> v2:172.21.15.3:3300/0 conn(0x7fd5e40b4e30 0x7fd5e406e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:38.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.949+0000 7fd5ea642640 1 --2- 172.21.15.3:0/2178809685 >> v2:172.21.15.190:3300/0 conn(0x7fd5e406e790 0x7fd5e415c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:38.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.949+0000 7fd5e8e3f640 1 -- 172.21.15.3:0/2178809685 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4047339543 0 0) 0x7fd5e40b40f0 con 0x7fd5e415c0f0 2024-05-09T06:36:38.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.949+0000 7fd5e8e3f640 1 -- 172.21.15.3:0/2178809685 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd5d0003520 con 0x7fd5e415c0f0 2024-05-09T06:36:38.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.949+0000 7fd5e9640640 1 -- 172.21.15.3:0/2178809685 >> v2:172.21.15.190:3300/0 conn(0x7fd5e406e790 msgr2=0x7fd5e415c4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:38.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.949+0000 7fd5e9640640 1 --2- 172.21.15.3:0/2178809685 >> v2:172.21.15.190:3300/0 conn(0x7fd5e406e790 0x7fd5e415c4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:38.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.949+0000 7fd5e9640640 1 -- 172.21.15.3:0/2178809685 >> v1:172.21.15.3:6789/0 conn(0x7fd5e415c0f0 legacy=0x7fd5e4160730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:38.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.949+0000 7fd5e9640640 1 -- 172.21.15.3:0/2178809685 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5e40b3330 con 0x7fd5e40b4e30 2024-05-09T06:36:38.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.950+0000 7fd5e9640640 1 --2- 172.21.15.3:0/2178809685 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd5e40b4e30 0x7fd5e406e250 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fd5e4070bf0 tx=0x7fd5d402a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc896b9ad1cc3a63 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:38.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.950+0000 7fd5e8e3f640 1 -- 172.21.15.3:0/2178809685 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5d4037070 con 0x7fd5e40b4e30 2024-05-09T06:36:38.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.950+0000 7fd5e8e3f640 1 -- 172.21.15.3:0/2178809685 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd5d4032440 con 0x7fd5e40b4e30 2024-05-09T06:36:38.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.950+0000 7fd5e8e3f640 1 -- 172.21.15.3:0/2178809685 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5d402ae60 con 0x7fd5e40b4e30 2024-05-09T06:36:38.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.951+0000 7fd5ec0a5640 1 -- 172.21.15.3:0/2178809685 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd5e40b4e30 msgr2=0x7fd5e406e250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:38.932 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.951+0000 7fd5ec0a5640 1 --2- 172.21.15.3:0/2178809685 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd5e40b4e30 0x7fd5e406e250 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fd5e4070bf0 tx=0x7fd5d402a110 comp rx=0 tx=0).stop 2024-05-09T06:36:38.932 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.951+0000 7fd5ec0a5640 1 -- 172.21.15.3:0/2178809685 shutdown_connections 2024-05-09T06:36:38.932 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.951+0000 7fd5ec0a5640 1 --2- 172.21.15.3:0/2178809685 >> v2:172.21.15.190:3300/0 conn(0x7fd5e406e790 0x7fd5e415c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:38.932 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.951+0000 7fd5ec0a5640 1 --2- 172.21.15.3:0/2178809685 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd5e40b4e30 0x7fd5e406e250 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:38.932 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.951+0000 7fd5ec0a5640 1 -- 172.21.15.3:0/2178809685 >> 172.21.15.3:0/2178809685 conn(0x7fd5e40b1c20 msgr2=0x7fd5e40b2030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:38.932 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.951+0000 7fd5ec0a5640 1 -- 172.21.15.3:0/2178809685 shutdown_connections 2024-05-09T06:36:38.932 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.951+0000 7fd5ec0a5640 1 -- 172.21.15.3:0/2178809685 wait complete. 2024-05-09T06:36:38.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.952+0000 7fd5ec0a5640 1 Processor -- start 2024-05-09T06:36:38.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.952+0000 7fd5ec0a5640 1 -- start start 2024-05-09T06:36:38.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.953+0000 7fd5ec0a5640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd5e406e790 0x7fd5e41f84a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:38.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.953+0000 7fd5ec0a5640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd5e40b4e30 0x7fd5e415e510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:38.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.953+0000 7fd5ec0a5640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd5e415c0f0 0x7fd5e41477f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:38.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.953+0000 7fd5ec0a5640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fd5e40b3bd0 con 0x7fd5e415c0f0 2024-05-09T06:36:38.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.953+0000 7fd5ec0a5640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fd5e40b38f0 con 0x7fd5e406e790 2024-05-09T06:36:38.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.953+0000 7fd5ec0a5640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fd5e415ca10 con 0x7fd5e40b4e30 2024-05-09T06:36:38.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.953+0000 7fd5ea642640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd5e415c0f0 0x7fd5e41477f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:38.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.953+0000 7fd5ea642640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd5e415c0f0 0x7fd5e41477f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:35746/0 (socket says 172.21.15.3:35746) 2024-05-09T06:36:38.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.953+0000 7fd5ea642640 1 -- 172.21.15.3:0/3280764910 learned_addr learned my addr 172.21.15.3:0/3280764910 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:36:38.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.953+0000 7fd5e9e41640 1 --2- 172.21.15.3:0/3280764910 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd5e406e790 0x7fd5e41f84a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:38.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.953+0000 7fd5e9640640 1 --2- 172.21.15.3:0/3280764910 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd5e40b4e30 0x7fd5e415e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:38.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.953+0000 7fd5ea642640 1 -- 172.21.15.3:0/3280764910 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd5e40b4e30 msgr2=0x7fd5e415e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:38.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.953+0000 7fd5ea642640 1 --2- 172.21.15.3:0/3280764910 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd5e40b4e30 0x7fd5e415e510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:38.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.953+0000 7fd5ea642640 1 -- 172.21.15.3:0/3280764910 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd5e406e790 msgr2=0x7fd5e41f84a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:38.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.953+0000 7fd5ea642640 1 --2- 172.21.15.3:0/3280764910 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd5e406e790 0x7fd5e41f84a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:38.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.953+0000 7fd5ea642640 1 -- 172.21.15.3:0/3280764910 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5e414f800 con 0x7fd5e415c0f0 2024-05-09T06:36:38.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.953+0000 7fd5e9640640 1 --2- 172.21.15.3:0/3280764910 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd5e40b4e30 0x7fd5e415e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:36:38.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.953+0000 7fd5ea642640 1 --2- 172.21.15.3:0/3280764910 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd5e415c0f0 0x7fd5e41477f0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fd5dc007e40 tx=0x7fd5dc00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:38.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.954+0000 7fd5da7fc640 1 -- 172.21.15.3:0/3280764910 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5dc014620 con 0x7fd5e415c0f0 2024-05-09T06:36:38.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.954+0000 7fd5da7fc640 1 -- 172.21.15.3:0/3280764910 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd5dc019070 con 0x7fd5e415c0f0 2024-05-09T06:36:38.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.954+0000 7fd5da7fc640 1 -- 172.21.15.3:0/3280764910 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5dc014a20 con 0x7fd5e415c0f0 2024-05-09T06:36:38.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.954+0000 7fd5ec0a5640 1 -- 172.21.15.3:0/3280764910 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5e414fae0 con 0x7fd5e415c0f0 2024-05-09T06:36:38.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.954+0000 7fd5ec0a5640 1 -- 172.21.15.3:0/3280764910 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5e41500a0 con 0x7fd5e415c0f0 2024-05-09T06:36:38.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.954+0000 7fd5ec0a5640 1 -- 172.21.15.3:0/3280764910 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5e4069550 con 0x7fd5e415c0f0 2024-05-09T06:36:38.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.955+0000 7fd5da7fc640 1 -- 172.21.15.3:0/3280764910 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd5dc014bc0 con 0x7fd5e415c0f0 2024-05-09T06:36:38.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.955+0000 7fd5da7fc640 1 --2- 172.21.15.3:0/3280764910 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fd5c806be30 0x7fd5c806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:38.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.956+0000 7fd5da7fc640 1 -- 172.21.15.3:0/3280764910 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fd5dc08b770 con 0x7fd5e415c0f0 2024-05-09T06:36:38.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.956+0000 7fd5e9e41640 1 --2- 172.21.15.3:0/3280764910 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fd5c806be30 0x7fd5c806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:38.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.956+0000 7fd5e9e41640 1 --2- 172.21.15.3:0/3280764910 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fd5c806be30 0x7fd5c806e2f0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fd5e0005880 tx=0x7fd5e0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:38.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:38.960+0000 7fd5da7fc640 1 -- 172.21.15.3:0/3280764910 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fd5dc05a120 con 0x7fd5e415c0f0 2024-05-09T06:36:39.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:39.155+0000 7fd5ec0a5640 1 -- 172.21.15.3:0/3280764910 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fd5e406efe0 con 0x7fd5e415c0f0 2024-05-09T06:36:39.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:39.156+0000 7fd5da7fc640 1 -- 172.21.15.3:0/3280764910 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v20) ==== 80+0+0 (secure 0 0 0) 0x7fd5dc059a20 con 0x7fd5e415c0f0 2024-05-09T06:36:39.136 INFO:teuthology.orchestra.run.smithi003.stderr:listed 0 entries 2024-05-09T06:36:39.138 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:39.158+0000 7fd5ec0a5640 1 -- 172.21.15.3:0/3280764910 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fd5c806be30 msgr2=0x7fd5c806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:39.138 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:39.158+0000 7fd5ec0a5640 1 --2- 172.21.15.3:0/3280764910 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fd5c806be30 0x7fd5c806e2f0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fd5e0005880 tx=0x7fd5e0009040 comp rx=0 tx=0).stop 2024-05-09T06:36:39.138 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:39.158+0000 7fd5ec0a5640 1 -- 172.21.15.3:0/3280764910 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd5e415c0f0 msgr2=0x7fd5e41477f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:39.138 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:39.158+0000 7fd5ec0a5640 1 --2- 172.21.15.3:0/3280764910 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd5e415c0f0 0x7fd5e41477f0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fd5dc007e40 tx=0x7fd5dc00e4b0 comp rx=0 tx=0).stop 2024-05-09T06:36:39.138 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:39.158+0000 7fd5ec0a5640 1 -- 172.21.15.3:0/3280764910 shutdown_connections 2024-05-09T06:36:39.138 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:39.158+0000 7fd5ec0a5640 1 --2- 172.21.15.3:0/3280764910 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fd5c806be30 0x7fd5c806e2f0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:39.138 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:39.158+0000 7fd5ec0a5640 1 --2- 172.21.15.3:0/3280764910 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd5e415c0f0 0x7fd5e41477f0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:39.138 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:39.158+0000 7fd5ec0a5640 1 --2- 172.21.15.3:0/3280764910 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd5e40b4e30 0x7fd5e415e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:39.138 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:39.158+0000 7fd5ec0a5640 1 --2- 172.21.15.3:0/3280764910 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd5e406e790 0x7fd5e41f84a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:39.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:39.158+0000 7fd5ec0a5640 1 -- 172.21.15.3:0/3280764910 >> 172.21.15.3:0/3280764910 conn(0x7fd5e40b1c20 msgr2=0x7fd5e40b0c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:39.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:39.158+0000 7fd5ec0a5640 1 -- 172.21.15.3:0/3280764910 shutdown_connections 2024-05-09T06:36:39.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:39.158+0000 7fd5ec0a5640 1 -- 172.21.15.3:0/3280764910 wait complete. 2024-05-09T06:36:39.149 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-05-09T06:36:39.149 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:36:39.149 DEBUG:teuthology.orchestra.run.smithi003:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2024-05-09T06:36:39.177 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi003.front.sepia.ceph.com 2024-05-09T06:36:39.177 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:36:39.177 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-09T06:36:39.244 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-05-09T06:36:39.244 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T06:36:39.264 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-05-09T06:36:39.264 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T06:36:39.324 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-05-09T06:36:39.324 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T06:36:39.383 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T06:36:39.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:39.509+0000 7f72b7a4d640 1 Processor -- start 2024-05-09T06:36:39.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:39.509+0000 7f72b7a4d640 1 -- start start 2024-05-09T06:36:39.490 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:39.510+0000 7f72b7a4d640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f72b015e420 0x7f72b006b250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:39.490 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:39.510+0000 7f72b7a4d640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f72b006b790 0x7f72b00b4640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:39.490 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:39.510+0000 7f72b7a4d640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f72b00b5010 con 0x7f72b00b4b80 2024-05-09T06:36:39.490 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:39.510+0000 7f72b7a4d640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f72b006bbf0 con 0x7f72b015e420 2024-05-09T06:36:39.490 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:39.510+0000 7f72b7a4d640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f72b015e800 con 0x7f72b006b790 2024-05-09T06:36:39.490 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:39.510+0000 7f72b57e9640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f72b015e420 0x7f72b006b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:39.491 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:39.510+0000 7f72b57e9640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f72b015e420 0x7f72b006b250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:35750/0 (socket says 172.21.15.3:35750) 2024-05-09T06:36:39.491 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:39.510+0000 7f72b57e9640 1 -- 172.21.15.3:0/3194965452 learned_addr learned my addr 172.21.15.3:0/3194965452 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:36:39.491 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:39.510+0000 7f72b4fe8640 1 --2- 172.21.15.3:0/3194965452 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f72b006b790 0x7f72b00b4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:39.491 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:39.510+0000 7f72b57e9640 1 -- 172.21.15.3:0/3194965452 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f72b006b790 msgr2=0x7f72b00b4640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:39.491 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:39.510+0000 7f72b57e9640 1 --2- 172.21.15.3:0/3194965452 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f72b006b790 0x7f72b00b4640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:39.491 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:39.510+0000 7f72b57e9640 1 -- 172.21.15.3:0/3194965452 >> v1:172.21.15.190:6789/0 conn(0x7f72b00b4b80 legacy=0x7f72b006fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:39.491 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:39.510+0000 7f72b57e9640 1 -- 172.21.15.3:0/3194965452 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72b00ae600 con 0x7f72b015e420 2024-05-09T06:36:39.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:39.511+0000 7f72b57e9640 1 --2- 172.21.15.3:0/3194965452 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f72b015e420 0x7f72b006b250 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f72ac009e50 tx=0x7f72ac02a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52c93a02e6dba962 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:39.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:39.511+0000 7f72a7fff640 1 -- 172.21.15.3:0/3194965452 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72ac036070 con 0x7f72b015e420 2024-05-09T06:36:39.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:39.511+0000 7f72a7fff640 1 -- 172.21.15.3:0/3194965452 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f72ac031600 con 0x7f72b015e420 2024-05-09T06:36:39.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:39.511+0000 7f72a7fff640 1 -- 172.21.15.3:0/3194965452 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72ac0319d0 con 0x7f72b015e420 2024-05-09T06:36:39.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:39.512+0000 7f72b7a4d640 1 -- 172.21.15.3:0/3194965452 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f72b015e420 msgr2=0x7f72b006b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:39.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:39.512+0000 7f72b7a4d640 1 --2- 172.21.15.3:0/3194965452 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f72b015e420 0x7f72b006b250 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f72ac009e50 tx=0x7f72ac02a680 comp rx=0 tx=0).stop 2024-05-09T06:36:39.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:39.512+0000 7f72b7a4d640 1 -- 172.21.15.3:0/3194965452 shutdown_connections 2024-05-09T06:36:39.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:39.512+0000 7f72b7a4d640 1 --2- 172.21.15.3:0/3194965452 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f72b006b790 0x7f72b00b4640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:39.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:39.512+0000 7f72b7a4d640 1 --2- 172.21.15.3:0/3194965452 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f72b015e420 0x7f72b006b250 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:39.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:39.512+0000 7f72b7a4d640 1 -- 172.21.15.3:0/3194965452 >> 172.21.15.3:0/3194965452 conn(0x7f72b006e3d0 msgr2=0x7f72b006e7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:39.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:39.512+0000 7f72b7a4d640 1 -- 172.21.15.3:0/3194965452 shutdown_connections 2024-05-09T06:36:39.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:39.512+0000 7f72b7a4d640 1 -- 172.21.15.3:0/3194965452 wait complete. 2024-05-09T06:36:39.495 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:39.513+0000 7f72b7a4d640 1 Processor -- start 2024-05-09T06:36:39.495 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:39.513+0000 7f72b7a4d640 1 -- start start 2024-05-09T06:36:39.495 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:39.514+0000 7f72b7a4d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f72b006b790 0x7f72b014b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:39.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:39.514+0000 7f72b7a4d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f72b00b4b80 0x7f72b0156f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:39.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:39.514+0000 7f72b7a4d640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f72b015e420 0x7f72b0157440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:39.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:39.514+0000 7f72b7a4d640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f72b00b5850 con 0x7f72b006b790 2024-05-09T06:36:39.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:39.514+0000 7f72b7a4d640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f72b00b5570 con 0x7f72b00b4b80 2024-05-09T06:36:39.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:39.514+0000 7f72b7a4d640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f72b00b5b30 con 0x7f72b015e420 2024-05-09T06:36:39.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:39.514+0000 7f72b57e9640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f72b006b790 0x7f72b014b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:39.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:39.514+0000 7f72b57e9640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f72b006b790 0x7f72b014b8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:35758/0 (socket says 172.21.15.3:35758) 2024-05-09T06:36:39.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:39.514+0000 7f72b57e9640 1 -- 172.21.15.3:0/2538294448 learned_addr learned my addr 172.21.15.3:0/2538294448 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:36:39.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:39.514+0000 7f72b4fe8640 1 --2- 172.21.15.3:0/2538294448 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f72b00b4b80 0x7f72b0156f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:39.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:39.514+0000 7f72b5fea640 1 --2- 172.21.15.3:0/2538294448 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f72b015e420 0x7f72b0157440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:39.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:39.514+0000 7f72b57e9640 1 -- 172.21.15.3:0/2538294448 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f72b015e420 msgr2=0x7f72b0157440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:39.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:39.514+0000 7f72b57e9640 1 --2- 172.21.15.3:0/2538294448 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f72b015e420 0x7f72b0157440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:39.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:39.514+0000 7f72b57e9640 1 -- 172.21.15.3:0/2538294448 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f72b00b4b80 msgr2=0x7f72b0156f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:39.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:39.514+0000 7f72b57e9640 1 --2- 172.21.15.3:0/2538294448 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f72b00b4b80 0x7f72b0156f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:39.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:39.514+0000 7f72b57e9640 1 -- 172.21.15.3:0/2538294448 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72b014be10 con 0x7f72b006b790 2024-05-09T06:36:39.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:39.515+0000 7f72b57e9640 1 --2- 172.21.15.3:0/2538294448 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f72b006b790 0x7f72b014b8d0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f72ac000f80 tx=0x7f72ac004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:39.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:39.515+0000 7f72a5ffb640 1 -- 172.21.15.3:0/2538294448 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72ac036050 con 0x7f72b006b790 2024-05-09T06:36:39.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:39.515+0000 7f72a5ffb640 1 -- 172.21.15.3:0/2538294448 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f72ac042070 con 0x7f72b006b790 2024-05-09T06:36:39.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:39.515+0000 7f72b7a4d640 1 -- 172.21.15.3:0/2538294448 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72b0161410 con 0x7f72b006b790 2024-05-09T06:36:39.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:39.515+0000 7f72a5ffb640 1 -- 172.21.15.3:0/2538294448 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72ac009510 con 0x7f72b006b790 2024-05-09T06:36:39.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:39.515+0000 7f72b7a4d640 1 -- 172.21.15.3:0/2538294448 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72b00cdf50 con 0x7f72b006b790 2024-05-09T06:36:39.498 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:39.516+0000 7f72b7a4d640 1 -- 172.21.15.3:0/2538294448 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f72b004f660 con 0x7f72b006b790 2024-05-09T06:36:39.498 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:39.517+0000 7f72a5ffb640 1 -- 172.21.15.3:0/2538294448 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f72ac007c70 con 0x7f72b006b790 2024-05-09T06:36:39.498 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:39.517+0000 7f72a5ffb640 1 --2- 172.21.15.3:0/2538294448 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f729406be60 0x7f729406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:39.498 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:39.517+0000 7f72a5ffb640 1 -- 172.21.15.3:0/2538294448 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f72ac0abcf0 con 0x7f72b006b790 2024-05-09T06:36:39.498 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:39.518+0000 7f72b4fe8640 1 --2- 172.21.15.3:0/2538294448 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f729406be60 0x7f729406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:39.498 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:39.518+0000 7f72b4fe8640 1 --2- 172.21.15.3:0/2538294448 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f729406be60 0x7f729406e320 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f72a0006fd0 tx=0x7f72a0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:39.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:39.522+0000 7f72a5ffb640 1 -- 172.21.15.3:0/2538294448 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f72ac0b0070 con 0x7f72b006b790 2024-05-09T06:36:39.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:39.705+0000 7f72b7a4d640 1 -- 172.21.15.3:0/2538294448 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f72b00b4190 con 0x7f72b006b790 2024-05-09T06:36:39.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:39.706+0000 7f72a5ffb640 1 -- 172.21.15.3:0/2538294448 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16521 (secure 0 0 0) 0x7f72ac045360 con 0x7f72b006b790 2024-05-09T06:36:39.686 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-09T06:36:39.686 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":20,"fsid":"16ed737d-2665-431a-8ac6-ef242be3975a","created":"2024-05-09T06:35:42.541368+0000","modified":"2024-05-09T06:36:08.856391+0000","last_up_change":"2024-05-09T06:35:53.778274+0000","last_in_change":"2024-05-09T06:35:44.780586+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T06:35:54.819999+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T06:36:04.931314+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T06:36:06.200433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fae3a8db-4826-4f3c-b33a-12c09bb5df6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":1897849827},{"type":"v1","addr":"172.21.15.3:6817","nonce":1897849827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":1897849827},{"type":"v1","addr":"172.21.15.3:6819","nonce":1897849827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":1897849827},{"type":"v1","addr":"172.21.15.3:6823","nonce":1897849827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":1897849827},{"type":"v1","addr":"172.21.15.3:6821","nonce":1897849827}]},"public_addr":"172.21.15.3:6817/1897849827","cluster_addr":"172.21.15.3:6819/1897849827","heartbeat_back_addr":"172.21.15.3:6823/1897849827","heartbeat_front_addr":"172.21.15.3:6821/1897849827","state":["exists","up"]},{"osd":1,"uuid":"646e882d-4ce4-4720-8c84-f71df563a35e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":926805498},{"type":"v1","addr":"172.21.15.3:6801","nonce":926805498}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":926805498},{"type":"v1","addr":"172.21.15.3:6803","nonce":926805498}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":926805498},{"type":"v1","addr":"172.21.15.3:6807","nonce":926805498}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":926805498},{"type":"v1","addr":"172.21.15.3:6805","nonce":926805498}]},"public_addr":"172.21.15.3:6801/926805498","cluster_addr":"172.21.15.3:6803/926805498","heartbeat_back_addr":"172.21.15.3:6807/926805498","heartbeat_front_addr":"172.21.15.3:6805/926805498","state":["exists","up"]},{"osd":2,"uuid":"1abbadb5-a570-4359-9643-acff3ede35a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":801295451},{"type":"v1","addr":"172.21.15.3:6809","nonce":801295451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":801295451},{"type":"v1","addr":"172.21.15.3:6811","nonce":801295451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":801295451},{"type":"v1","addr":"172.21.15.3:6815","nonce":801295451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":801295451},{"type":"v1","addr":"172.21.15.3:6813","nonce":801295451}]},"public_addr":"172.21.15.3:6809/801295451","cluster_addr":"172.21.15.3:6811/801295451","heartbeat_back_addr":"172.21.15.3:6815/801295451","heartbeat_front_addr":"172.21.15.3:6813/801295451","state":["exists","up"]},{"osd":3,"uuid":"62ac4dc1-2371-4dd2-acd3-e2dbb11aedf5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":1903389857},{"type":"v1","addr":"172.21.15.3:6825","nonce":1903389857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":1903389857},{"type":"v1","addr":"172.21.15.3:6827","nonce":1903389857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":1903389857},{"type":"v1","addr":"172.21.15.3:6831","nonce":1903389857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":1903389857},{"type":"v1","addr":"172.21.15.3:6829","nonce":1903389857}]},"public_addr":"172.21.15.3:6825/1903389857","cluster_addr":"172.21.15.3:6827/1903389857","heartbeat_back_addr":"172.21.15.3:6831/1903389857","heartbeat_front_addr":"172.21.15.3:6829/1903389857","state":["exists","up"]},{"osd":4,"uuid":"ae808398-8326-4d7c-9319-a2a0eafffb59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":2143624576},{"type":"v1","addr":"172.21.15.190:6801","nonce":2143624576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":2143624576},{"type":"v1","addr":"172.21.15.190:6803","nonce":2143624576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":2143624576},{"type":"v1","addr":"172.21.15.190:6807","nonce":2143624576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":2143624576},{"type":"v1","addr":"172.21.15.190:6805","nonce":2143624576}]},"public_addr":"172.21.15.190:6801/2143624576","cluster_addr":"172.21.15.190:6803/2143624576","heartbeat_back_addr":"172.21.15.190:6807/2143624576","heartbeat_front_addr":"172.21.15.190:6805/2143624576","state":["exists","up"]},{"osd":5,"uuid":"918adf50-c440-48ac-9a8d-4837562d7275","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":1766923126},{"type":"v1","addr":"172.21.15.190:6814","nonce":1766923126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":1766923126},{"type":"v1","addr":"172.21.15.190:6818","nonce":1766923126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":1766923126},{"type":"v1","addr":"172.21.15.190:6823","nonce":1766923126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":1766923126},{"type":"v1","addr":"172.21.15.190:6821","nonce":1766923126}]},"public_addr":"172.21.15.190:6814/1766923126","cluster_addr":"172.21.15.190:6818/1766923126","heartbeat_back_addr":"172.21.15.190:6823/1766923126","heartbeat_front_addr":"172.21.15.190:6821/1766923126","state":["exists","up"]},{"osd":6,"uuid":"ac2882bc-2a7c-463c-8f6c-76ff49828adc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":342414248},{"type":"v1","addr":"172.21.15.190:6825","nonce":342414248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":342414248},{"type":"v1","addr":"172.21.15.190:6827","nonce":342414248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":342414248},{"type":"v1","addr":"172.21.15.190:6831","nonce":342414248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":342414248},{"type":"v1","addr":"172.21.15.190:6829","nonce":342414248}]},"public_addr":"172.21.15.190:6825/342414248","cluster_addr":"172.21.15.190:6827/342414248","heartbeat_back_addr":"172.21.15.190:6831/342414248","heartbeat_front_addr":"172.21.15.190:6829/342414248","state":["exists","up"]},{"osd":7,"uuid":"dd4e767a-4730-4c70-8bd5-3f088140a5a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":537849891},{"type":"v1","addr":"172.21.15.190:6809","nonce":537849891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":537849891},{"type":"v1","addr":"172.21.15.190:6811","nonce":537849891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6817","nonce":537849891},{"type":"v1","addr":"172.21.15.190:6819","nonce":537849891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6813","nonce":537849891},{"type":"v1","addr":"172.21.15.190:6815","nonce":537849891}]},"public_addr":"172.21.15.190:6809/537849891","cluster_addr":"172.21.15.190:6811/537849891","heartbeat_back_addr":"172.21.15.190:6819/537849891","heartbeat_front_addr":"172.21.15.190:6815/537849891","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.889784+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.974002+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.973549+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.955396+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.928471+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.902418+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.958696+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.926474+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T06:36:39.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:39.708+0000 7f72b7a4d640 1 -- 172.21.15.3:0/2538294448 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f729406be60 msgr2=0x7f729406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:39.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:39.708+0000 7f72b7a4d640 1 --2- 172.21.15.3:0/2538294448 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f729406be60 0x7f729406e320 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f72a0006fd0 tx=0x7f72a0008040 comp rx=0 tx=0).stop 2024-05-09T06:36:39.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:39.708+0000 7f72b7a4d640 1 -- 172.21.15.3:0/2538294448 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f72b006b790 msgr2=0x7f72b014b8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:39.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:39.708+0000 7f72b7a4d640 1 --2- 172.21.15.3:0/2538294448 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f72b006b790 0x7f72b014b8d0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f72ac000f80 tx=0x7f72ac004a40 comp rx=0 tx=0).stop 2024-05-09T06:36:39.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:39.708+0000 7f72b7a4d640 1 -- 172.21.15.3:0/2538294448 shutdown_connections 2024-05-09T06:36:39.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:39.708+0000 7f72b7a4d640 1 --2- 172.21.15.3:0/2538294448 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f729406be60 0x7f729406e320 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:39.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:39.708+0000 7f72b7a4d640 1 --2- 172.21.15.3:0/2538294448 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f72b015e420 0x7f72b0157440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:39.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:39.708+0000 7f72b7a4d640 1 --2- 172.21.15.3:0/2538294448 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f72b00b4b80 0x7f72b0156f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:39.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:39.708+0000 7f72b7a4d640 1 --2- 172.21.15.3:0/2538294448 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f72b006b790 0x7f72b014b8d0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:39.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:39.708+0000 7f72b7a4d640 1 -- 172.21.15.3:0/2538294448 >> 172.21.15.3:0/2538294448 conn(0x7f72b006e3d0 msgr2=0x7f72b00b2160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:39.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:39.708+0000 7f72b7a4d640 1 -- 172.21.15.3:0/2538294448 shutdown_connections 2024-05-09T06:36:39.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:39.708+0000 7f72b7a4d640 1 -- 172.21.15.3:0/2538294448 wait complete. 2024-05-09T06:36:39.699 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-09T06:35:54.819999+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-09T06:36:04.931314+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-09T06:36:06.200433+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-09T06:36:39.700 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-09T06:36:39.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:39.827+0000 7fb2baff4640 1 Processor -- start 2024-05-09T06:36:39.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:39.828+0000 7fb2baff4640 1 -- start start 2024-05-09T06:36:39.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:39.828+0000 7fb2baff4640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fb2b415e5c0 0x7fb2b40d2800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:39.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:39.828+0000 7fb2baff4640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb2b40d4c40 0x7fb2b40d0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:39.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:39.828+0000 7fb2baff4640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb2b415e9a0 con 0x7fb2b40d1130 2024-05-09T06:36:39.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:39.828+0000 7fb2baff4640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fb2b40d31c0 con 0x7fb2b415e5c0 2024-05-09T06:36:39.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:39.828+0000 7fb2baff4640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fb2b40d50a0 con 0x7fb2b40d4c40 2024-05-09T06:36:39.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:39.828+0000 7fb2b8d90640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fb2b415e5c0 0x7fb2b40d2800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:39.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:39.829+0000 7fb2b8d90640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fb2b415e5c0 0x7fb2b40d2800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:35766/0 (socket says 172.21.15.3:35766) 2024-05-09T06:36:39.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:39.829+0000 7fb2b9591640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fb2b40d1130 0x7fb2b40ced30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:59020/0 (socket says 172.21.15.3:59020) 2024-05-09T06:36:39.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:39.829+0000 7fb2b8d90640 1 -- 172.21.15.3:0/2864686983 learned_addr learned my addr 172.21.15.3:0/2864686983 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:36:39.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:39.829+0000 7fb2b3fff640 1 --2- 172.21.15.3:0/2864686983 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb2b40d4c40 0x7fb2b40d0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:39.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:39.829+0000 7fb2b8d90640 1 -- 172.21.15.3:0/2864686983 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb2b40d4c40 msgr2=0x7fb2b40d0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:39.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:39.829+0000 7fb2b8d90640 1 --2- 172.21.15.3:0/2864686983 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb2b40d4c40 0x7fb2b40d0bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:39.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:39.829+0000 7fb2b8d90640 1 -- 172.21.15.3:0/2864686983 >> v1:172.21.15.3:6789/0 conn(0x7fb2b40d1130 legacy=0x7fb2b40ced30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:39.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:39.829+0000 7fb2b8d90640 1 -- 172.21.15.3:0/2864686983 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2b40d2ee0 con 0x7fb2b415e5c0 2024-05-09T06:36:39.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:39.830+0000 7fb2b8d90640 1 --2- 172.21.15.3:0/2864686983 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb2b415e5c0 0x7fb2b40d2800 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fb2a4009870 tx=0x7fb2a402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18407b4281b70340 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:39.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:39.830+0000 7fb2b37fe640 1 -- 172.21.15.3:0/2864686983 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2a4037070 con 0x7fb2b415e5c0 2024-05-09T06:36:39.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:39.830+0000 7fb2b37fe640 1 -- 172.21.15.3:0/2864686983 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb2a4032440 con 0x7fb2b415e5c0 2024-05-09T06:36:39.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:39.830+0000 7fb2b37fe640 1 -- 172.21.15.3:0/2864686983 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2a4032810 con 0x7fb2b415e5c0 2024-05-09T06:36:39.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:39.830+0000 7fb2baff4640 1 -- 172.21.15.3:0/2864686983 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb2b415e5c0 msgr2=0x7fb2b40d2800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:39.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:39.830+0000 7fb2baff4640 1 --2- 172.21.15.3:0/2864686983 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb2b415e5c0 0x7fb2b40d2800 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fb2a4009870 tx=0x7fb2a402a270 comp rx=0 tx=0).stop 2024-05-09T06:36:39.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:39.830+0000 7fb2baff4640 1 -- 172.21.15.3:0/2864686983 shutdown_connections 2024-05-09T06:36:39.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:39.830+0000 7fb2baff4640 1 --2- 172.21.15.3:0/2864686983 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb2b40d4c40 0x7fb2b40d0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:39.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:39.830+0000 7fb2baff4640 1 --2- 172.21.15.3:0/2864686983 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb2b415e5c0 0x7fb2b40d2800 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:39.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:39.830+0000 7fb2baff4640 1 -- 172.21.15.3:0/2864686983 >> 172.21.15.3:0/2864686983 conn(0x7fb2b409d500 msgr2=0x7fb2b40d4220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:39.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:39.830+0000 7fb2baff4640 1 -- 172.21.15.3:0/2864686983 shutdown_connections 2024-05-09T06:36:39.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:39.830+0000 7fb2baff4640 1 -- 172.21.15.3:0/2864686983 wait complete. 2024-05-09T06:36:39.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:39.832+0000 7fb2baff4640 1 Processor -- start 2024-05-09T06:36:39.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:39.832+0000 7fb2baff4640 1 -- start start 2024-05-09T06:36:39.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:39.832+0000 7fb2baff4640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb2b40d1130 0x7fb2b40cbdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:39.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:39.832+0000 7fb2baff4640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb2b40d4c40 0x7fb2b40cde40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:39.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:39.832+0000 7fb2baff4640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb2b415e5c0 0x7fb2b415acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:39.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:39.832+0000 7fb2baff4640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fb2b40b42c0 con 0x7fb2b415e5c0 2024-05-09T06:36:39.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:39.832+0000 7fb2baff4640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fb2b40b3fe0 con 0x7fb2b40d1130 2024-05-09T06:36:39.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:39.832+0000 7fb2baff4640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fb2b40b57f0 con 0x7fb2b40d4c40 2024-05-09T06:36:39.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:39.833+0000 7fb2b9591640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb2b415e5c0 0x7fb2b415acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:39.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:39.833+0000 7fb2b9591640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb2b415e5c0 0x7fb2b415acb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:35772/0 (socket says 172.21.15.3:35772) 2024-05-09T06:36:39.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:39.833+0000 7fb2b9591640 1 -- 172.21.15.3:0/3721138397 learned_addr learned my addr 172.21.15.3:0/3721138397 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:36:39.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:39.833+0000 7fb2b3fff640 1 --2- 172.21.15.3:0/3721138397 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb2b40d4c40 0x7fb2b40cde40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:39.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:39.833+0000 7fb2b8d90640 1 --2- 172.21.15.3:0/3721138397 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb2b40d1130 0x7fb2b40cbdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:39.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:39.833+0000 7fb2b9591640 1 -- 172.21.15.3:0/3721138397 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb2b40d4c40 msgr2=0x7fb2b40cde40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:39.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:39.833+0000 7fb2b9591640 1 --2- 172.21.15.3:0/3721138397 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb2b40d4c40 0x7fb2b40cde40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:39.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:39.833+0000 7fb2b9591640 1 -- 172.21.15.3:0/3721138397 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb2b40d1130 msgr2=0x7fb2b40cbdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:39.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:39.833+0000 7fb2b9591640 1 --2- 172.21.15.3:0/3721138397 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb2b40d1130 0x7fb2b40cbdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:39.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:39.833+0000 7fb2b9591640 1 -- 172.21.15.3:0/3721138397 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2b40cb2f0 con 0x7fb2b415e5c0 2024-05-09T06:36:39.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:39.833+0000 7fb2b9591640 1 --2- 172.21.15.3:0/3721138397 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb2b415e5c0 0x7fb2b415acb0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fb2ac00c400 tx=0x7fb2ac00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:39.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:39.833+0000 7fb2b17fa640 1 -- 172.21.15.3:0/3721138397 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2ac0074e0 con 0x7fb2b415e5c0 2024-05-09T06:36:39.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:39.833+0000 7fb2b17fa640 1 -- 172.21.15.3:0/3721138397 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb2ac007680 con 0x7fb2b415e5c0 2024-05-09T06:36:39.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:39.833+0000 7fb2baff4640 1 -- 172.21.15.3:0/3721138397 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2b4158260 con 0x7fb2b415e5c0 2024-05-09T06:36:39.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:39.833+0000 7fb2b17fa640 1 -- 172.21.15.3:0/3721138397 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2ac0049e0 con 0x7fb2b415e5c0 2024-05-09T06:36:39.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:39.833+0000 7fb2baff4640 1 -- 172.21.15.3:0/3721138397 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2b40cb5b0 con 0x7fb2b415e5c0 2024-05-09T06:36:39.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:39.834+0000 7fb2baff4640 1 -- 172.21.15.3:0/3721138397 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2b40734b0 con 0x7fb2b415e5c0 2024-05-09T06:36:39.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:39.835+0000 7fb2b17fa640 1 -- 172.21.15.3:0/3721138397 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb2ac01a020 con 0x7fb2b415e5c0 2024-05-09T06:36:39.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:39.835+0000 7fb2b17fa640 1 --2- 172.21.15.3:0/3721138397 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fb29406be60 0x7fb29406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:39.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:39.835+0000 7fb2b17fa640 1 -- 172.21.15.3:0/3721138397 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fb2ac08a540 con 0x7fb2b415e5c0 2024-05-09T06:36:39.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:39.836+0000 7fb2b8d90640 1 --2- 172.21.15.3:0/3721138397 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fb29406be60 0x7fb29406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:39.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:39.836+0000 7fb2b8d90640 1 --2- 172.21.15.3:0/3721138397 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fb29406be60 0x7fb29406e320 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fb2a4034000 tx=0x7fb2a402e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:39.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:39.840+0000 7fb2b17fa640 1 -- 172.21.15.3:0/3721138397 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fb2b40734b0 con 0x7fb2b415e5c0 2024-05-09T06:36:40.004 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.024+0000 7fb2baff4640 1 -- 172.21.15.3:0/3721138397 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fb2b4069550 con 0x7fb2b415e5c0 2024-05-09T06:36:40.004 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.024+0000 7fb2b17fa640 1 -- 172.21.15.3:0/3721138397 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7fb2ac058800 con 0x7fb2b415e5c0 2024-05-09T06:36:40.004 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 1 2024-05-09T06:36:40.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.026+0000 7fb2baff4640 1 -- 172.21.15.3:0/3721138397 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fb29406be60 msgr2=0x7fb29406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:40.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.026+0000 7fb2baff4640 1 --2- 172.21.15.3:0/3721138397 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fb29406be60 0x7fb29406e320 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fb2a4034000 tx=0x7fb2a402e000 comp rx=0 tx=0).stop 2024-05-09T06:36:40.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.026+0000 7fb2baff4640 1 -- 172.21.15.3:0/3721138397 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb2b415e5c0 msgr2=0x7fb2b415acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:40.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.026+0000 7fb2baff4640 1 --2- 172.21.15.3:0/3721138397 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb2b415e5c0 0x7fb2b415acb0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fb2ac00c400 tx=0x7fb2ac00c8c0 comp rx=0 tx=0).stop 2024-05-09T06:36:40.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.026+0000 7fb2baff4640 1 -- 172.21.15.3:0/3721138397 shutdown_connections 2024-05-09T06:36:40.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.026+0000 7fb2baff4640 1 --2- 172.21.15.3:0/3721138397 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fb29406be60 0x7fb29406e320 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:40.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.026+0000 7fb2baff4640 1 --2- 172.21.15.3:0/3721138397 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb2b415e5c0 0x7fb2b415acb0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:40.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.026+0000 7fb2baff4640 1 --2- 172.21.15.3:0/3721138397 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb2b40d4c40 0x7fb2b40cde40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:40.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.026+0000 7fb2baff4640 1 --2- 172.21.15.3:0/3721138397 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb2b40d1130 0x7fb2b40cbdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:40.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.026+0000 7fb2baff4640 1 -- 172.21.15.3:0/3721138397 >> 172.21.15.3:0/3721138397 conn(0x7fb2b409d500 msgr2=0x7fb2b40d42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:40.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.026+0000 7fb2baff4640 1 -- 172.21.15.3:0/3721138397 shutdown_connections 2024-05-09T06:36:40.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.026+0000 7fb2baff4640 1 -- 172.21.15.3:0/3721138397 wait complete. 2024-05-09T06:36:40.017 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-09T06:36:40.125 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.145+0000 7f1bacf70640 1 Processor -- start 2024-05-09T06:36:40.125 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.145+0000 7f1bacf70640 1 -- start start 2024-05-09T06:36:40.126 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.146+0000 7f1bacf70640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f1ba8072900 0x7f1ba80cf8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:40.126 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.146+0000 7f1bacf70640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f1ba80b4130 0x7f1ba80b45e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:40.127 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.146+0000 7f1bacf70640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1ba8071860 con 0x7f1ba8144a10 2024-05-09T06:36:40.127 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.146+0000 7f1bacf70640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f1ba8072470 con 0x7f1ba80b4130 2024-05-09T06:36:40.127 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.146+0000 7f1bacf70640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f1ba8071b40 con 0x7f1ba8072900 2024-05-09T06:36:40.127 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.146+0000 7f1ba659c640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f1ba8144a10 0x7f1ba8071080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:59036/0 (socket says 172.21.15.3:59036) 2024-05-09T06:36:40.127 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.146+0000 7f1ba659c640 1 -- 172.21.15.3:0/1075964949 learned_addr learned my addr 172.21.15.3:0/1075964949 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-05-09T06:36:40.127 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.146+0000 7f1ba6d9d640 1 --2- 172.21.15.3:0/1075964949 >> v2:172.21.15.3:3300/0 conn(0x7f1ba80b4130 0x7f1ba80b45e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:40.127 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.146+0000 7f1ba5d9b640 1 --2- 172.21.15.3:0/1075964949 >> v2:172.21.15.190:3300/0 conn(0x7f1ba8072900 0x7f1ba80cf8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:40.127 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.146+0000 7f1ba559a640 1 -- 172.21.15.3:0/1075964949 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 437147559 0 0) 0x7f1ba8071860 con 0x7f1ba8144a10 2024-05-09T06:36:40.127 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.146+0000 7f1ba559a640 1 -- 172.21.15.3:0/1075964949 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1b90003520 con 0x7f1ba8144a10 2024-05-09T06:36:40.127 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.146+0000 7f1ba6d9d640 1 -- 172.21.15.3:0/1075964949 >> v2:172.21.15.190:3300/0 conn(0x7f1ba8072900 msgr2=0x7f1ba80cf8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:40.127 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.146+0000 7f1ba6d9d640 1 --2- 172.21.15.3:0/1075964949 >> v2:172.21.15.190:3300/0 conn(0x7f1ba8072900 0x7f1ba80cf8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:40.127 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.146+0000 7f1ba6d9d640 1 -- 172.21.15.3:0/1075964949 >> v1:172.21.15.3:6789/0 conn(0x7f1ba8144a10 legacy=0x7f1ba8071080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:40.127 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.146+0000 7f1ba6d9d640 1 -- 172.21.15.3:0/1075964949 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ba8071e20 con 0x7f1ba80b4130 2024-05-09T06:36:40.129 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.147+0000 7f1ba6d9d640 1 --2- 172.21.15.3:0/1075964949 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1ba80b4130 0x7f1ba80b45e0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f1ba80736f0 tx=0x7f1ba002a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d7a88923950deed server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:40.129 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.147+0000 7f1ba559a640 1 -- 172.21.15.3:0/1075964949 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ba0007740 con 0x7f1ba80b4130 2024-05-09T06:36:40.129 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.147+0000 7f1ba559a640 1 -- 172.21.15.3:0/1075964949 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ba00078e0 con 0x7f1ba80b4130 2024-05-09T06:36:40.129 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.147+0000 7f1ba559a640 1 -- 172.21.15.3:0/1075964949 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ba0007cb0 con 0x7f1ba80b4130 2024-05-09T06:36:40.129 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.148+0000 7f1bacf70640 1 -- 172.21.15.3:0/1075964949 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1ba80b4130 msgr2=0x7f1ba80b45e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:40.129 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.148+0000 7f1bacf70640 1 --2- 172.21.15.3:0/1075964949 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1ba80b4130 0x7f1ba80b45e0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f1ba80736f0 tx=0x7f1ba002a5f0 comp rx=0 tx=0).stop 2024-05-09T06:36:40.129 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.148+0000 7f1bacf70640 1 -- 172.21.15.3:0/1075964949 shutdown_connections 2024-05-09T06:36:40.129 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.148+0000 7f1bacf70640 1 --2- 172.21.15.3:0/1075964949 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1ba80b4130 0x7f1ba80b45e0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:40.129 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.148+0000 7f1bacf70640 1 --2- 172.21.15.3:0/1075964949 >> v2:172.21.15.190:3300/0 conn(0x7f1ba8072900 0x7f1ba80cf8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:40.129 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.148+0000 7f1bacf70640 1 -- 172.21.15.3:0/1075964949 >> 172.21.15.3:0/1075964949 conn(0x7f1ba80b1c20 msgr2=0x7f1ba80b2030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:40.129 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.148+0000 7f1bacf70640 1 -- 172.21.15.3:0/1075964949 shutdown_connections 2024-05-09T06:36:40.129 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.148+0000 7f1bacf70640 1 -- 172.21.15.3:0/1075964949 wait complete. 2024-05-09T06:36:40.131 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.149+0000 7f1bacf70640 1 Processor -- start 2024-05-09T06:36:40.132 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.149+0000 7f1bacf70640 1 -- start start 2024-05-09T06:36:40.132 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.150+0000 7f1bacf70640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1ba8072900 0x7f1ba8137960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:40.132 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.150+0000 7f1bacf70640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1ba80b4130 0x7f1ba814a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:40.132 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.150+0000 7f1bacf70640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1ba8144a10 0x7f1ba81f74f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:40.132 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.150+0000 7f1bacf70640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f1ba815b750 con 0x7f1ba8144a10 2024-05-09T06:36:40.132 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.150+0000 7f1bacf70640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f1ba815b5d0 con 0x7f1ba8072900 2024-05-09T06:36:40.132 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.150+0000 7f1bacf70640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f1ba815b8d0 con 0x7f1ba80b4130 2024-05-09T06:36:40.132 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.150+0000 7f1ba6d9d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1ba8144a10 0x7f1ba81f74f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:40.132 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.150+0000 7f1ba6d9d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1ba8144a10 0x7f1ba81f74f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:35788/0 (socket says 172.21.15.3:35788) 2024-05-09T06:36:40.132 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.150+0000 7f1ba6d9d640 1 -- 172.21.15.3:0/3491008835 learned_addr learned my addr 172.21.15.3:0/3491008835 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:36:40.132 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.151+0000 7f1ba5d9b640 1 --2- 172.21.15.3:0/3491008835 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1ba80b4130 0x7f1ba814a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:40.132 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.151+0000 7f1ba659c640 1 --2- 172.21.15.3:0/3491008835 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1ba8072900 0x7f1ba8137960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:40.132 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.151+0000 7f1ba6d9d640 1 -- 172.21.15.3:0/3491008835 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1ba80b4130 msgr2=0x7f1ba814a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:40.132 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.151+0000 7f1ba6d9d640 1 --2- 172.21.15.3:0/3491008835 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1ba80b4130 0x7f1ba814a3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:40.133 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.151+0000 7f1ba6d9d640 1 -- 172.21.15.3:0/3491008835 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1ba8072900 msgr2=0x7f1ba8137960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:40.133 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.151+0000 7f1ba6d9d640 1 --2- 172.21.15.3:0/3491008835 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1ba8072900 0x7f1ba8137960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:40.133 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.151+0000 7f1ba6d9d640 1 -- 172.21.15.3:0/3491008835 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ba813dd70 con 0x7f1ba8144a10 2024-05-09T06:36:40.133 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.151+0000 7f1ba5d9b640 1 --2- 172.21.15.3:0/3491008835 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1ba80b4130 0x7f1ba814a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:36:40.133 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.151+0000 7f1ba6d9d640 1 --2- 172.21.15.3:0/3491008835 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1ba8144a10 0x7f1ba81f74f0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f1ba0009e50 tx=0x7f1ba0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:40.133 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.151+0000 7f1b96ffd640 1 -- 172.21.15.3:0/3491008835 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ba0040070 con 0x7f1ba8144a10 2024-05-09T06:36:40.133 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.151+0000 7f1b96ffd640 1 -- 172.21.15.3:0/3491008835 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ba0031bf0 con 0x7f1ba8144a10 2024-05-09T06:36:40.133 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.151+0000 7f1bacf70640 1 -- 172.21.15.3:0/3491008835 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ba813e050 con 0x7f1ba8144a10 2024-05-09T06:36:40.133 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.151+0000 7f1b96ffd640 1 -- 172.21.15.3:0/3491008835 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ba0009450 con 0x7f1ba8144a10 2024-05-09T06:36:40.133 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.151+0000 7f1bacf70640 1 -- 172.21.15.3:0/3491008835 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ba814a8f0 con 0x7f1ba8144a10 2024-05-09T06:36:40.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.152+0000 7f1bacf70640 1 -- 172.21.15.3:0/3491008835 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ba8069550 con 0x7f1ba8144a10 2024-05-09T06:36:40.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.153+0000 7f1b96ffd640 1 -- 172.21.15.3:0/3491008835 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1ba0039040 con 0x7f1ba8144a10 2024-05-09T06:36:40.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.153+0000 7f1b96ffd640 1 --2- 172.21.15.3:0/3491008835 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f1b8806be60 0x7f1b8806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:40.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.153+0000 7f1b96ffd640 1 -- 172.21.15.3:0/3491008835 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f1ba00ac6c0 con 0x7f1ba8144a10 2024-05-09T06:36:40.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.153+0000 7f1ba659c640 1 --2- 172.21.15.3:0/3491008835 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f1b8806be60 0x7f1b8806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:40.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.154+0000 7f1ba659c640 1 --2- 172.21.15.3:0/3491008835 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f1b8806be60 0x7f1b8806e320 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f1b98005880 tx=0x7f1b98009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:40.137 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.157+0000 7f1b96ffd640 1 -- 172.21.15.3:0/3491008835 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f1ba004a060 con 0x7f1ba8144a10 2024-05-09T06:36:40.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.340+0000 7f1bacf70640 1 -- 172.21.15.3:0/3491008835 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f1ba80b3830 con 0x7f1ba8144a10 2024-05-09T06:36:40.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.340+0000 7f1b96ffd640 1 -- 172.21.15.3:0/3491008835 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7f1ba007b100 con 0x7f1ba8144a10 2024-05-09T06:36:40.320 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 32 2024-05-09T06:36:40.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.342+0000 7f1bacf70640 1 -- 172.21.15.3:0/3491008835 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f1b8806be60 msgr2=0x7f1b8806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:40.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.342+0000 7f1bacf70640 1 --2- 172.21.15.3:0/3491008835 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f1b8806be60 0x7f1b8806e320 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f1b98005880 tx=0x7f1b98009040 comp rx=0 tx=0).stop 2024-05-09T06:36:40.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.342+0000 7f1bacf70640 1 -- 172.21.15.3:0/3491008835 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1ba8144a10 msgr2=0x7f1ba81f74f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:40.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.342+0000 7f1bacf70640 1 --2- 172.21.15.3:0/3491008835 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1ba8144a10 0x7f1ba81f74f0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f1ba0009e50 tx=0x7f1ba0004a40 comp rx=0 tx=0).stop 2024-05-09T06:36:40.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.342+0000 7f1bacf70640 1 -- 172.21.15.3:0/3491008835 shutdown_connections 2024-05-09T06:36:40.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.342+0000 7f1bacf70640 1 --2- 172.21.15.3:0/3491008835 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f1b8806be60 0x7f1b8806e320 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:40.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.342+0000 7f1bacf70640 1 --2- 172.21.15.3:0/3491008835 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1ba8144a10 0x7f1ba81f74f0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:40.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.342+0000 7f1bacf70640 1 --2- 172.21.15.3:0/3491008835 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1ba80b4130 0x7f1ba814a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:40.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.342+0000 7f1bacf70640 1 --2- 172.21.15.3:0/3491008835 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1ba8072900 0x7f1ba8137960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:40.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.342+0000 7f1bacf70640 1 -- 172.21.15.3:0/3491008835 >> 172.21.15.3:0/3491008835 conn(0x7f1ba80b1c20 msgr2=0x7f1ba81385d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:40.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.342+0000 7f1bacf70640 1 -- 172.21.15.3:0/3491008835 shutdown_connections 2024-05-09T06:36:40.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.342+0000 7f1bacf70640 1 -- 172.21.15.3:0/3491008835 wait complete. 2024-05-09T06:36:40.333 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-09T06:36:40.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.460+0000 7f5a2fef5640 1 Processor -- start 2024-05-09T06:36:40.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.461+0000 7f5a2fef5640 1 -- start start 2024-05-09T06:36:40.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.461+0000 7f5a2fef5640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f5a280d2800 0x7f5a280d4c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:40.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.461+0000 7f5a2fef5640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5a2815e9a0 con 0x7f5a2815e5c0 2024-05-09T06:36:40.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.461+0000 7f5a2fef5640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5a280d2c50 con 0x7f5a280d1300 2024-05-09T06:36:40.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.461+0000 7f5a2fef5640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f5a280d31c0 con 0x7f5a280d2800 2024-05-09T06:36:40.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.462+0000 7f5a2dc91640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f5a2815e5c0 0x7f5a280d0bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:59048/0 (socket says 172.21.15.3:59048) 2024-05-09T06:36:40.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.462+0000 7f5a2dc91640 1 -- 172.21.15.3:0/3517545328 learned_addr learned my addr 172.21.15.3:0/3517545328 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-05-09T06:36:40.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.462+0000 7f5a2e492640 1 --2- 172.21.15.3:0/3517545328 >> v2:172.21.15.3:3300/0 conn(0x7f5a280d2800 0x7f5a280d4c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:40.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.462+0000 7f5a2cc8f640 1 -- 172.21.15.3:0/3517545328 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1940576918 0 0) 0x7f5a2815e9a0 con 0x7f5a2815e5c0 2024-05-09T06:36:40.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.462+0000 7f5a2cc8f640 1 -- 172.21.15.3:0/3517545328 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5a10003520 con 0x7f5a2815e5c0 2024-05-09T06:36:40.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.462+0000 7f5a2e492640 1 -- 172.21.15.3:0/3517545328 >> v1:172.21.15.190:6789/0 conn(0x7f5a280d1300 legacy=0x7f5a280b47e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:40.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.462+0000 7f5a2e492640 1 -- 172.21.15.3:0/3517545328 >> v1:172.21.15.3:6789/0 conn(0x7f5a2815e5c0 legacy=0x7f5a280d0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:40.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.462+0000 7f5a2e492640 1 -- 172.21.15.3:0/3517545328 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a280d2ee0 con 0x7f5a280d2800 2024-05-09T06:36:40.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.463+0000 7f5a2e492640 1 --2- 172.21.15.3:0/3517545328 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5a280d2800 0x7f5a280d4c40 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f5a18009d90 tx=0x7f5a1802a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13e2662699b790ad server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:40.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.463+0000 7f5a2cc8f640 1 -- 172.21.15.3:0/3517545328 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a18007740 con 0x7f5a280d2800 2024-05-09T06:36:40.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.463+0000 7f5a2cc8f640 1 -- 172.21.15.3:0/3517545328 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a180078e0 con 0x7f5a280d2800 2024-05-09T06:36:40.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.463+0000 7f5a2cc8f640 1 -- 172.21.15.3:0/3517545328 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a18007cb0 con 0x7f5a280d2800 2024-05-09T06:36:40.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.463+0000 7f5a2fef5640 1 -- 172.21.15.3:0/3517545328 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5a280d2800 msgr2=0x7f5a280d4c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:40.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.463+0000 7f5a2fef5640 1 --2- 172.21.15.3:0/3517545328 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5a280d2800 0x7f5a280d4c40 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f5a18009d90 tx=0x7f5a1802a6f0 comp rx=0 tx=0).stop 2024-05-09T06:36:40.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.463+0000 7f5a2fef5640 1 -- 172.21.15.3:0/3517545328 shutdown_connections 2024-05-09T06:36:40.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.463+0000 7f5a2fef5640 1 --2- 172.21.15.3:0/3517545328 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5a280d2800 0x7f5a280d4c40 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:40.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.463+0000 7f5a2fef5640 1 -- 172.21.15.3:0/3517545328 >> 172.21.15.3:0/3517545328 conn(0x7f5a2809d500 msgr2=0x7f5a280d4220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:40.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.463+0000 7f5a2fef5640 1 -- 172.21.15.3:0/3517545328 shutdown_connections 2024-05-09T06:36:40.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.464+0000 7f5a2fef5640 1 -- 172.21.15.3:0/3517545328 wait complete. 2024-05-09T06:36:40.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.465+0000 7f5a2fef5640 1 Processor -- start 2024-05-09T06:36:40.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.465+0000 7f5a2fef5640 1 -- start start 2024-05-09T06:36:40.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.465+0000 7f5a2fef5640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5a280d1300 0x7f5a28153740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:40.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.465+0000 7f5a2fef5640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5a280d2800 0x7f5a280b8cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:40.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.465+0000 7f5a2fef5640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5a2815e5c0 0x7f5a281f3c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:40.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.465+0000 7f5a2fef5640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f5a280b42c0 con 0x7f5a280d1300 2024-05-09T06:36:40.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.465+0000 7f5a2fef5640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f5a280b3fe0 con 0x7f5a2815e5c0 2024-05-09T06:36:40.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.465+0000 7f5a2fef5640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f5a280b4ef0 con 0x7f5a280d2800 2024-05-09T06:36:40.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.466+0000 7f5a2dc91640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5a280d1300 0x7f5a28153740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:40.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.466+0000 7f5a2dc91640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5a280d1300 0x7f5a28153740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:35814/0 (socket says 172.21.15.3:35814) 2024-05-09T06:36:40.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.466+0000 7f5a2dc91640 1 -- 172.21.15.3:0/2732331378 learned_addr learned my addr 172.21.15.3:0/2732331378 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:36:40.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.466+0000 7f5a2e492640 1 --2- 172.21.15.3:0/2732331378 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5a2815e5c0 0x7f5a281f3c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:40.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.466+0000 7f5a2d490640 1 --2- 172.21.15.3:0/2732331378 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5a280d2800 0x7f5a280b8cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:40.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.466+0000 7f5a2dc91640 1 -- 172.21.15.3:0/2732331378 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5a280d2800 msgr2=0x7f5a280b8cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:40.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.466+0000 7f5a2dc91640 1 --2- 172.21.15.3:0/2732331378 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5a280d2800 0x7f5a280b8cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:40.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.466+0000 7f5a2dc91640 1 -- 172.21.15.3:0/2732331378 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5a2815e5c0 msgr2=0x7f5a281f3c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:40.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.466+0000 7f5a2dc91640 1 --2- 172.21.15.3:0/2732331378 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5a2815e5c0 0x7f5a281f3c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:40.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.466+0000 7f5a2dc91640 1 -- 172.21.15.3:0/2732331378 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a280b96d0 con 0x7f5a280d1300 2024-05-09T06:36:40.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.466+0000 7f5a2dc91640 1 --2- 172.21.15.3:0/2732331378 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5a280d1300 0x7f5a28153740 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f5a2000c410 tx=0x7f5a2000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:40.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.466+0000 7f5a267fc640 1 -- 172.21.15.3:0/2732331378 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a20017070 con 0x7f5a280d1300 2024-05-09T06:36:40.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.466+0000 7f5a267fc640 1 -- 172.21.15.3:0/2732331378 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a20005ce0 con 0x7f5a280d1300 2024-05-09T06:36:40.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.466+0000 7f5a2fef5640 1 -- 172.21.15.3:0/2732331378 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a280b99b0 con 0x7f5a280d1300 2024-05-09T06:36:40.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.466+0000 7f5a267fc640 1 -- 172.21.15.3:0/2732331378 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a200072f0 con 0x7f5a280d1300 2024-05-09T06:36:40.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.466+0000 7f5a2fef5640 1 -- 172.21.15.3:0/2732331378 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a2815eff0 con 0x7f5a280d1300 2024-05-09T06:36:40.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.467+0000 7f5a2fef5640 1 -- 172.21.15.3:0/2732331378 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a28069550 con 0x7f5a280d1300 2024-05-09T06:36:40.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.468+0000 7f5a267fc640 1 -- 172.21.15.3:0/2732331378 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5a2001a070 con 0x7f5a280d1300 2024-05-09T06:36:40.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.468+0000 7f5a267fc640 1 --2- 172.21.15.3:0/2732331378 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f5a1406be30 0x7f5a1406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:40.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.468+0000 7f5a267fc640 1 -- 172.21.15.3:0/2732331378 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f5a2008a2e0 con 0x7f5a280d1300 2024-05-09T06:36:40.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.469+0000 7f5a2d490640 1 --2- 172.21.15.3:0/2732331378 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f5a1406be30 0x7f5a1406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:40.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.469+0000 7f5a2d490640 1 --2- 172.21.15.3:0/2732331378 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f5a1406be30 0x7f5a1406e2f0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f5a1c007fd0 tx=0x7f5a1c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:40.452 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.473+0000 7f5a267fc640 1 -- 172.21.15.3:0/2732331378 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f5a20058d40 con 0x7f5a280d1300 2024-05-09T06:36:40.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.655+0000 7f5a2fef5640 1 -- 172.21.15.3:0/2732331378 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f5a280b9b80 con 0x7f5a280d1300 2024-05-09T06:36:40.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.655+0000 7f5a267fc640 1 -- 172.21.15.3:0/2732331378 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7f5a20058640 con 0x7f5a280d1300 2024-05-09T06:36:40.635 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 64 2024-05-09T06:36:40.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.657+0000 7f5a2fef5640 1 -- 172.21.15.3:0/2732331378 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f5a1406be30 msgr2=0x7f5a1406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:40.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.657+0000 7f5a2fef5640 1 --2- 172.21.15.3:0/2732331378 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f5a1406be30 0x7f5a1406e2f0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f5a1c007fd0 tx=0x7f5a1c009040 comp rx=0 tx=0).stop 2024-05-09T06:36:40.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.657+0000 7f5a2fef5640 1 -- 172.21.15.3:0/2732331378 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5a280d1300 msgr2=0x7f5a28153740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:40.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.657+0000 7f5a2fef5640 1 --2- 172.21.15.3:0/2732331378 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5a280d1300 0x7f5a28153740 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f5a2000c410 tx=0x7f5a2000c8d0 comp rx=0 tx=0).stop 2024-05-09T06:36:40.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.657+0000 7f5a2fef5640 1 -- 172.21.15.3:0/2732331378 shutdown_connections 2024-05-09T06:36:40.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.657+0000 7f5a2fef5640 1 --2- 172.21.15.3:0/2732331378 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f5a1406be30 0x7f5a1406e2f0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:40.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.657+0000 7f5a2fef5640 1 --2- 172.21.15.3:0/2732331378 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5a2815e5c0 0x7f5a281f3c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:40.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.657+0000 7f5a2fef5640 1 --2- 172.21.15.3:0/2732331378 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5a280d2800 0x7f5a280b8cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:40.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.657+0000 7f5a2fef5640 1 --2- 172.21.15.3:0/2732331378 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5a280d1300 0x7f5a28153740 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:40.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.657+0000 7f5a2fef5640 1 -- 172.21.15.3:0/2732331378 >> 172.21.15.3:0/2732331378 conn(0x7f5a2809d500 msgr2=0x7f5a280b6110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:40.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.657+0000 7f5a2fef5640 1 -- 172.21.15.3:0/2732331378 shutdown_connections 2024-05-09T06:36:40.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.657+0000 7f5a2fef5640 1 -- 172.21.15.3:0/2732331378 wait complete. 2024-05-09T06:36:40.648 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-09T06:36:40.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.773+0000 7f7b60df9640 1 Processor -- start 2024-05-09T06:36:40.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.773+0000 7f7b60df9640 1 -- start start 2024-05-09T06:36:40.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.774+0000 7f7b60df9640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f7b5c06ed10 0x7f7b5c0ce8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:40.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.774+0000 7f7b60df9640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7b5c0cede0 0x7f7b5c072660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:40.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.774+0000 7f7b60df9640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7b5c0cf250 con 0x7f7b5c145a50 2024-05-09T06:36:40.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.774+0000 7f7b60df9640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f7b5c145e30 con 0x7f7b5c06ed10 2024-05-09T06:36:40.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.774+0000 7f7b60df9640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f7b5c160bc0 con 0x7f7b5c0cede0 2024-05-09T06:36:40.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.774+0000 7f7b51d9b640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f7b5c06ed10 0x7f7b5c0ce8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:40.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.774+0000 7f7b5ad9d640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7b5c0cede0 0x7f7b5c072660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:40.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.774+0000 7f7b5a59c640 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7f7b5c145a50 0x7f7b5c06e600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.3:58182/0 (socket says 172.21.15.3:58182) 2024-05-09T06:36:40.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.774+0000 7f7b5a59c640 1 -- 172.21.15.3:0/1645773186 learned_addr learned my addr 172.21.15.3:0/1645773186 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-05-09T06:36:40.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.774+0000 7f7b5ad9d640 1 -- 172.21.15.3:0/1645773186 >> v2:172.21.15.190:3300/0 conn(0x7f7b5c06ed10 msgr2=0x7f7b5c0ce8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:40.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.774+0000 7f7b5ad9d640 1 --2- 172.21.15.3:0/1645773186 >> v2:172.21.15.190:3300/0 conn(0x7f7b5c06ed10 0x7f7b5c0ce8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:40.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.774+0000 7f7b5ad9d640 1 -- 172.21.15.3:0/1645773186 >> v1:172.21.15.190:6789/0 conn(0x7f7b5c145a50 legacy=0x7f7b5c06e600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:40.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.774+0000 7f7b5ad9d640 1 -- 172.21.15.3:0/1645773186 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b5c15fe00 con 0x7f7b5c0cede0 2024-05-09T06:36:40.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.774+0000 7f7b51d9b640 1 --2- 172.21.15.3:0/1645773186 >> v2:172.21.15.190:3300/0 conn(0x7f7b5c06ed10 0x7f7b5c0ce8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T06:36:40.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.775+0000 7f7b5ad9d640 1 --2- 172.21.15.3:0/1645773186 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7b5c0cede0 0x7f7b5c072660 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f7b54009d90 tx=0x7f7b5402a5d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ed09897e66213f58 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:40.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.775+0000 7f7b59d9b640 1 -- 172.21.15.3:0/1645773186 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b54031600 con 0x7f7b5c0cede0 2024-05-09T06:36:40.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.775+0000 7f7b59d9b640 1 -- 172.21.15.3:0/1645773186 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b540317a0 con 0x7f7b5c0cede0 2024-05-09T06:36:40.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.775+0000 7f7b59d9b640 1 -- 172.21.15.3:0/1645773186 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b54031b70 con 0x7f7b5c0cede0 2024-05-09T06:36:40.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.776+0000 7f7b60df9640 1 -- 172.21.15.3:0/1645773186 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7b5c0cede0 msgr2=0x7f7b5c072660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:40.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.776+0000 7f7b60df9640 1 --2- 172.21.15.3:0/1645773186 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7b5c0cede0 0x7f7b5c072660 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f7b54009d90 tx=0x7f7b5402a5d0 comp rx=0 tx=0).stop 2024-05-09T06:36:40.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.776+0000 7f7b60df9640 1 -- 172.21.15.3:0/1645773186 shutdown_connections 2024-05-09T06:36:40.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.776+0000 7f7b60df9640 1 --2- 172.21.15.3:0/1645773186 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7b5c0cede0 0x7f7b5c072660 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:40.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.776+0000 7f7b60df9640 1 --2- 172.21.15.3:0/1645773186 >> v2:172.21.15.190:3300/0 conn(0x7f7b5c06ed10 0x7f7b5c0ce8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:40.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.776+0000 7f7b60df9640 1 -- 172.21.15.3:0/1645773186 >> 172.21.15.3:0/1645773186 conn(0x7f7b5c0b6530 msgr2=0x7f7b5c0b6910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:40.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.776+0000 7f7b60df9640 1 -- 172.21.15.3:0/1645773186 shutdown_connections 2024-05-09T06:36:40.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.776+0000 7f7b60df9640 1 -- 172.21.15.3:0/1645773186 wait complete. 2024-05-09T06:36:40.759 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.777+0000 7f7b60df9640 1 Processor -- start 2024-05-09T06:36:40.759 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.777+0000 7f7b60df9640 1 -- start start 2024-05-09T06:36:40.759 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.778+0000 7f7b60df9640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7b5c06ed10 0x7f7b5c159710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:40.759 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.778+0000 7f7b60df9640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7b5c0cede0 0x7f7b5c1f2c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:40.759 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.778+0000 7f7b60df9640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7b5c145a50 0x7f7b5c13a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:40.759 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.778+0000 7f7b60df9640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f7b5c1606a0 con 0x7f7b5c145a50 2024-05-09T06:36:40.760 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.778+0000 7f7b60df9640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f7b5c1603c0 con 0x7f7b5c0cede0 2024-05-09T06:36:40.760 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.778+0000 7f7b60df9640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f7b5c0cf4f0 con 0x7f7b5c06ed10 2024-05-09T06:36:40.760 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.778+0000 7f7b5ad9d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7b5c145a50 0x7f7b5c13a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:40.760 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.778+0000 7f7b5ad9d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7b5c145a50 0x7f7b5c13a580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46478/0 (socket says 172.21.15.3:46478) 2024-05-09T06:36:40.760 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.778+0000 7f7b5ad9d640 1 -- 172.21.15.3:0/2530249194 learned_addr learned my addr 172.21.15.3:0/2530249194 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:36:40.760 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.779+0000 7f7b5a59c640 1 --2- 172.21.15.3:0/2530249194 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7b5c06ed10 0x7f7b5c159710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:40.760 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.779+0000 7f7b51d9b640 1 --2- 172.21.15.3:0/2530249194 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7b5c0cede0 0x7f7b5c1f2c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:40.760 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.779+0000 7f7b5ad9d640 1 -- 172.21.15.3:0/2530249194 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7b5c06ed10 msgr2=0x7f7b5c159710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:40.760 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.779+0000 7f7b5ad9d640 1 --2- 172.21.15.3:0/2530249194 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7b5c06ed10 0x7f7b5c159710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:40.760 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.779+0000 7f7b5ad9d640 1 -- 172.21.15.3:0/2530249194 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7b5c0cede0 msgr2=0x7f7b5c1f2c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:40.760 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.779+0000 7f7b5ad9d640 1 --2- 172.21.15.3:0/2530249194 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7b5c0cede0 0x7f7b5c1f2c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:40.760 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.779+0000 7f7b5ad9d640 1 -- 172.21.15.3:0/2530249194 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b5c1f3180 con 0x7f7b5c145a50 2024-05-09T06:36:40.760 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.779+0000 7f7b5ad9d640 1 --2- 172.21.15.3:0/2530249194 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7b5c145a50 0x7f7b5c13a580 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f7b5402abc0 tx=0x7f7b54004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:40.760 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.779+0000 7f7b537fe640 1 -- 172.21.15.3:0/2530249194 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b54040070 con 0x7f7b5c145a50 2024-05-09T06:36:40.761 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.779+0000 7f7b537fe640 1 -- 172.21.15.3:0/2530249194 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b54030b60 con 0x7f7b5c145a50 2024-05-09T06:36:40.761 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.779+0000 7f7b60df9640 1 -- 172.21.15.3:0/2530249194 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b5c158670 con 0x7f7b5c145a50 2024-05-09T06:36:40.761 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.779+0000 7f7b537fe640 1 -- 172.21.15.3:0/2530249194 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b54009450 con 0x7f7b5c145a50 2024-05-09T06:36:40.761 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.779+0000 7f7b60df9640 1 -- 172.21.15.3:0/2530249194 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b5c158c30 con 0x7f7b5c145a50 2024-05-09T06:36:40.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.780+0000 7f7b60df9640 1 -- 172.21.15.3:0/2530249194 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b5c0b0710 con 0x7f7b5c145a50 2024-05-09T06:36:40.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.781+0000 7f7b537fe640 1 -- 172.21.15.3:0/2530249194 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7b54039040 con 0x7f7b5c145a50 2024-05-09T06:36:40.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.781+0000 7f7b537fe640 1 --2- 172.21.15.3:0/2530249194 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f7b3c06be30 0x7f7b3c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:40.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.781+0000 7f7b537fe640 1 -- 172.21.15.3:0/2530249194 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f7b540ac3f0 con 0x7f7b5c145a50 2024-05-09T06:36:40.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.782+0000 7f7b5a59c640 1 --2- 172.21.15.3:0/2530249194 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f7b3c06be30 0x7f7b3c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:40.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.782+0000 7f7b5a59c640 1 --2- 172.21.15.3:0/2530249194 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f7b3c06be30 0x7f7b3c06e2f0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f7b5c13edb0 tx=0x7f7b4c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:40.765 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.785+0000 7f7b537fe640 1 -- 172.21.15.3:0/2530249194 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f7b5c0b0710 con 0x7f7b5c145a50 2024-05-09T06:36:40.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.982+0000 7f7b60df9640 1 -- 172.21.15.3:0/2530249194 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f7b5c06c0e0 con 0x7f7b5c145a50 2024-05-09T06:36:40.963 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.983+0000 7f7b537fe640 1 -- 172.21.15.3:0/2530249194 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v20) ==== 80+0+0 (secure 0 0 0) 0x7f7b5407ae30 con 0x7f7b5c145a50 2024-05-09T06:36:40.963 INFO:teuthology.orchestra.run.smithi003.stderr:listed 0 entries 2024-05-09T06:36:40.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.984+0000 7f7b60df9640 1 -- 172.21.15.3:0/2530249194 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f7b3c06be30 msgr2=0x7f7b3c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:40.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.984+0000 7f7b60df9640 1 --2- 172.21.15.3:0/2530249194 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f7b3c06be30 0x7f7b3c06e2f0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f7b5c13edb0 tx=0x7f7b4c009040 comp rx=0 tx=0).stop 2024-05-09T06:36:40.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.985+0000 7f7b60df9640 1 -- 172.21.15.3:0/2530249194 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7b5c145a50 msgr2=0x7f7b5c13a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:40.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.985+0000 7f7b60df9640 1 --2- 172.21.15.3:0/2530249194 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7b5c145a50 0x7f7b5c13a580 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f7b5402abc0 tx=0x7f7b54004a40 comp rx=0 tx=0).stop 2024-05-09T06:36:40.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.985+0000 7f7b60df9640 1 -- 172.21.15.3:0/2530249194 shutdown_connections 2024-05-09T06:36:40.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.985+0000 7f7b60df9640 1 --2- 172.21.15.3:0/2530249194 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f7b3c06be30 0x7f7b3c06e2f0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:40.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.985+0000 7f7b60df9640 1 --2- 172.21.15.3:0/2530249194 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7b5c145a50 0x7f7b5c13a580 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:40.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.985+0000 7f7b60df9640 1 --2- 172.21.15.3:0/2530249194 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7b5c0cede0 0x7f7b5c1f2c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:40.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.985+0000 7f7b60df9640 1 --2- 172.21.15.3:0/2530249194 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7b5c06ed10 0x7f7b5c159710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:40.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.985+0000 7f7b60df9640 1 -- 172.21.15.3:0/2530249194 >> 172.21.15.3:0/2530249194 conn(0x7f7b5c0b6530 msgr2=0x7f7b5c0b6910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:40.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.985+0000 7f7b60df9640 1 -- 172.21.15.3:0/2530249194 shutdown_connections 2024-05-09T06:36:40.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:40.985+0000 7f7b60df9640 1 -- 172.21.15.3:0/2530249194 wait complete. 2024-05-09T06:36:40.976 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-05-09T06:36:40.976 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:36:40.976 DEBUG:teuthology.orchestra.run.smithi190:> sudo umount /home/ubuntu/cephtest/mnt.1 -f 2024-05-09T06:36:41.008 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi190.front.sepia.ceph.com 2024-05-09T06:36:41.009 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:36:41.009 DEBUG:teuthology.orchestra.run.smithi190:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-05-09T06:36:41.072 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-05-09T06:36:41.072 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T06:36:41.089 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-05-09T06:36:41.090 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T06:36:41.147 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-05-09T06:36:41.147 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T06:36:41.204 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T06:36:41.348 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:41.368+0000 7f01e75f4640 1 Processor -- start 2024-05-09T06:36:41.348 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:41.368+0000 7f01e75f4640 1 -- start start 2024-05-09T06:36:41.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:41.369+0000 7f01e75f4640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f01e013e0c0 0x7f01e00cf720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:41.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:41.369+0000 7f01e75f4640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f01e00cc870 0x7f01e00cccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:41.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:41.369+0000 7f01e75f4640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f01e013e4a0 con 0x7f01e01408f0 2024-05-09T06:36:41.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:41.369+0000 7f01e75f4640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f01e00cd210 con 0x7f01e013e0c0 2024-05-09T06:36:41.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:41.369+0000 7f01e75f4640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f01e00ce960 con 0x7f01e00cc870 2024-05-09T06:36:41.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:41.369+0000 7f01e5b91640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f01e01408f0 0x7f01e0140d40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:51104/0 (socket says 172.21.15.3:51104) 2024-05-09T06:36:41.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:41.369+0000 7f01e5b91640 1 -- 172.21.15.3:0/4051682608 learned_addr learned my addr 172.21.15.3:0/4051682608 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-05-09T06:36:41.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:41.370+0000 7f01e5390640 1 --2- 172.21.15.3:0/4051682608 >> v2:172.21.15.190:3300/0 conn(0x7f01e013e0c0 0x7f01e00cf720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:41.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:41.370+0000 7f01e4b8f640 1 --2- 172.21.15.3:0/4051682608 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f01e00cc870 0x7f01e00cccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:41.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:41.370+0000 7f01dffff640 1 -- 172.21.15.3:0/4051682608 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 792657312 0 0) 0x7f01d0001490 con 0x7f01e01408f0 2024-05-09T06:36:41.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:41.370+0000 7f01dffff640 1 -- 172.21.15.3:0/4051682608 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f01c8003520 con 0x7f01e01408f0 2024-05-09T06:36:41.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:41.370+0000 7f01dffff640 1 -- 172.21.15.3:0/4051682608 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 7946643 0 0) 0x7f01c8003520 con 0x7f01e01408f0 2024-05-09T06:36:41.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:41.370+0000 7f01dffff640 1 -- 172.21.15.3:0/4051682608 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f01e00cc870 msgr2=0x7f01e00cccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:41.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:41.370+0000 7f01dffff640 1 --2- 172.21.15.3:0/4051682608 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f01e00cc870 0x7f01e00cccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:41.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:41.370+0000 7f01dffff640 1 -- 172.21.15.3:0/4051682608 >> v2:172.21.15.190:3300/0 conn(0x7f01e013e0c0 msgr2=0x7f01e00cf720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:41.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:41.370+0000 7f01dffff640 1 --2- 172.21.15.3:0/4051682608 >> v2:172.21.15.190:3300/0 conn(0x7f01e013e0c0 0x7f01e00cf720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:41.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:41.370+0000 7f01dffff640 1 -- 172.21.15.3:0/4051682608 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01e00cec40 con 0x7f01e01408f0 2024-05-09T06:36:41.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:41.370+0000 7f01dffff640 1 -- 172.21.15.3:0/4051682608 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 563312031 0 0) 0x7f01d0001bc0 con 0x7f01e01408f0 2024-05-09T06:36:41.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:41.370+0000 7f01dffff640 1 -- 172.21.15.3:0/4051682608 >> v1:172.21.15.3:6789/0 conn(0x7f01e01408f0 legacy=0x7f01e0140d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:41.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:41.370+0000 7f01dffff640 1 --2- 172.21.15.3:0/4051682608 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f01c80039b0 0x7f01c8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:41.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:41.370+0000 7f01dffff640 1 --2- 172.21.15.3:0/4051682608 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f01c8004800 0x7f01c8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:41.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:41.370+0000 7f01dffff640 1 --2- 172.21.15.3:0/4051682608 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f01e00cc870 0x7f01c80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:41.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:41.370+0000 7f01dffff640 1 -- 172.21.15.3:0/4051682608 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f01e00cd210 con 0x7f01c8004800 2024-05-09T06:36:41.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:41.370+0000 7f01dffff640 1 -- 172.21.15.3:0/4051682608 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f01e00ce960 con 0x7f01c80039b0 2024-05-09T06:36:41.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:41.370+0000 7f01dffff640 1 -- 172.21.15.3:0/4051682608 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f01c800b9c0 con 0x7f01e00cc870 2024-05-09T06:36:41.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:41.370+0000 7f01e4b8f640 1 --2- 172.21.15.3:0/4051682608 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f01c8004800 0x7f01c8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:41.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:41.370+0000 7f01e5390640 1 --2- 172.21.15.3:0/4051682608 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f01c80039b0 0x7f01c8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:41.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:41.370+0000 7f01e5b91640 1 --2- 172.21.15.3:0/4051682608 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f01e00cc870 0x7f01c80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:41.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:41.370+0000 7f01e4b8f640 1 -- 172.21.15.3:0/4051682608 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f01e00cc870 msgr2=0x7f01c80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:41.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:41.370+0000 7f01e4b8f640 1 --2- 172.21.15.3:0/4051682608 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f01e00cc870 0x7f01c80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:41.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:41.370+0000 7f01e4b8f640 1 -- 172.21.15.3:0/4051682608 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f01c80039b0 msgr2=0x7f01c8003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:41.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:41.370+0000 7f01e4b8f640 1 --2- 172.21.15.3:0/4051682608 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f01c80039b0 0x7f01c8003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:41.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:41.370+0000 7f01e4b8f640 1 -- 172.21.15.3:0/4051682608 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01c8003520 con 0x7f01c8004800 2024-05-09T06:36:41.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:41.370+0000 7f01e5b91640 1 --2- 172.21.15.3:0/4051682608 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f01e00cc870 0x7f01c80051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:36:41.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:41.371+0000 7f01e4b8f640 1 --2- 172.21.15.3:0/4051682608 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f01c8004800 0x7f01c8004c80 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f01d4009ac0 tx=0x7f01c8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f0e0c906a7f5dd7 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:41.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:41.371+0000 7f01dffff640 1 -- 172.21.15.3:0/4051682608 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01d4036070 con 0x7f01c8004800 2024-05-09T06:36:41.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:41.371+0000 7f01dffff640 1 -- 172.21.15.3:0/4051682608 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01d4031400 con 0x7f01c8004800 2024-05-09T06:36:41.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:41.371+0000 7f01dffff640 1 -- 172.21.15.3:0/4051682608 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01d4009690 con 0x7f01c8004800 2024-05-09T06:36:41.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:41.372+0000 7f01e75f4640 1 -- 172.21.15.3:0/4051682608 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f01c8004800 msgr2=0x7f01c8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:41.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:41.372+0000 7f01e75f4640 1 --2- 172.21.15.3:0/4051682608 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f01c8004800 0x7f01c8004c80 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f01d4009ac0 tx=0x7f01c8005830 comp rx=0 tx=0).stop 2024-05-09T06:36:41.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:41.372+0000 7f01e5b91640 1 -- 172.21.15.3:0/4051682608 reap_dead start 2024-05-09T06:36:41.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:41.372+0000 7f01e75f4640 1 -- 172.21.15.3:0/4051682608 shutdown_connections 2024-05-09T06:36:41.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:41.372+0000 7f01e75f4640 1 -- 172.21.15.3:0/4051682608 >> 172.21.15.3:0/4051682608 conn(0x7f01e006b430 msgr2=0x7f01e006b810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:41.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:41.372+0000 7f01e75f4640 1 -- 172.21.15.3:0/4051682608 shutdown_connections 2024-05-09T06:36:41.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:41.372+0000 7f01e75f4640 1 -- 172.21.15.3:0/4051682608 wait complete. 2024-05-09T06:36:41.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:41.373+0000 7f01e75f4640 1 Processor -- start 2024-05-09T06:36:41.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:41.373+0000 7f01e75f4640 1 -- start start 2024-05-09T06:36:41.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:41.374+0000 7f01e75f4640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f01c8004800 0x7f01e013c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:41.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:41.374+0000 7f01e75f4640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f01e015d780 0x7f01e015dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:41.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:41.374+0000 7f01e75f4640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f01e00b1370 0x7f01e00b17f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:41.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:41.374+0000 7f01e75f4640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f01e0141660 con 0x7f01e00b1370 2024-05-09T06:36:41.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:41.374+0000 7f01e75f4640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f01e00cf200 con 0x7f01e015d780 2024-05-09T06:36:41.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:41.374+0000 7f01e75f4640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f01e00cccd0 con 0x7f01c8004800 2024-05-09T06:36:41.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:41.374+0000 7f01e5b91640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f01e00b1370 0x7f01e00b17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:41.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:41.374+0000 7f01e5b91640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f01e00b1370 0x7f01e00b17f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46506/0 (socket says 172.21.15.3:46506) 2024-05-09T06:36:41.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:41.374+0000 7f01e5b91640 1 -- 172.21.15.3:0/2920937168 learned_addr learned my addr 172.21.15.3:0/2920937168 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:36:41.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:41.374+0000 7f01e5b91640 1 -- 172.21.15.3:0/2920937168 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f01c8004800 msgr2=0x7f01e013c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:41.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:41.374+0000 7f01e5b91640 1 --2- 172.21.15.3:0/2920937168 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f01c8004800 0x7f01e013c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:41.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:41.374+0000 7f01e5b91640 1 -- 172.21.15.3:0/2920937168 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f01e015d780 msgr2=0x7f01e015dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:41.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:41.374+0000 7f01e5b91640 1 --2- 172.21.15.3:0/2920937168 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f01e015d780 0x7f01e015dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:41.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:41.374+0000 7f01e5b91640 1 -- 172.21.15.3:0/2920937168 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01e014f030 con 0x7f01e00b1370 2024-05-09T06:36:41.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:41.375+0000 7f01e5b91640 1 --2- 172.21.15.3:0/2920937168 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f01e00b1370 0x7f01e00b17f0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f01d0000f80 tx=0x7f01d000a550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:41.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:41.375+0000 7f01ddffb640 1 -- 172.21.15.3:0/2920937168 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01d000abb0 con 0x7f01e00b1370 2024-05-09T06:36:41.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:41.375+0000 7f01ddffb640 1 -- 172.21.15.3:0/2920937168 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01d0018070 con 0x7f01e00b1370 2024-05-09T06:36:41.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:41.375+0000 7f01e75f4640 1 -- 172.21.15.3:0/2920937168 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01e014f2d0 con 0x7f01e00b1370 2024-05-09T06:36:41.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:41.375+0000 7f01ddffb640 1 -- 172.21.15.3:0/2920937168 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01d0013420 con 0x7f01e00b1370 2024-05-09T06:36:41.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:41.375+0000 7f01e75f4640 1 -- 172.21.15.3:0/2920937168 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01e014f890 con 0x7f01e00b1370 2024-05-09T06:36:41.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:41.376+0000 7f01e75f4640 1 -- 172.21.15.3:0/2920937168 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01e004f660 con 0x7f01e00b1370 2024-05-09T06:36:41.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:41.376+0000 7f01ddffb640 1 -- 172.21.15.3:0/2920937168 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f01d00135c0 con 0x7f01e00b1370 2024-05-09T06:36:41.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:41.377+0000 7f01ddffb640 1 --2- 172.21.15.3:0/2920937168 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f01cc06be30 0x7f01cc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:41.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:41.377+0000 7f01ddffb640 1 -- 172.21.15.3:0/2920937168 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f01d008b250 con 0x7f01e00b1370 2024-05-09T06:36:41.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:41.377+0000 7f01e5390640 1 --2- 172.21.15.3:0/2920937168 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f01cc06be30 0x7f01cc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:41.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:41.378+0000 7f01e5390640 1 --2- 172.21.15.3:0/2920937168 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f01cc06be30 0x7f01cc06e2f0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f01d8006e80 tx=0x7f01d8006df0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:41.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:41.381+0000 7f01ddffb640 1 -- 172.21.15.3:0/2920937168 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f01e013e4a0 con 0x7f01e00b1370 2024-05-09T06:36:41.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:41.567+0000 7f01e75f4640 1 -- 172.21.15.3:0/2920937168 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f01e00b45c0 con 0x7f01e00b1370 2024-05-09T06:36:41.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:41.568+0000 7f01ddffb640 1 -- 172.21.15.3:0/2920937168 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16521 (secure 0 0 0) 0x7f01d0059c90 con 0x7f01e00b1370 2024-05-09T06:36:41.548 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-09T06:36:41.548 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":20,"fsid":"16ed737d-2665-431a-8ac6-ef242be3975a","created":"2024-05-09T06:35:42.541368+0000","modified":"2024-05-09T06:36:08.856391+0000","last_up_change":"2024-05-09T06:35:53.778274+0000","last_in_change":"2024-05-09T06:35:44.780586+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T06:35:54.819999+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T06:36:04.931314+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T06:36:06.200433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fae3a8db-4826-4f3c-b33a-12c09bb5df6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":1897849827},{"type":"v1","addr":"172.21.15.3:6817","nonce":1897849827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":1897849827},{"type":"v1","addr":"172.21.15.3:6819","nonce":1897849827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":1897849827},{"type":"v1","addr":"172.21.15.3:6823","nonce":1897849827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":1897849827},{"type":"v1","addr":"172.21.15.3:6821","nonce":1897849827}]},"public_addr":"172.21.15.3:6817/1897849827","cluster_addr":"172.21.15.3:6819/1897849827","heartbeat_back_addr":"172.21.15.3:6823/1897849827","heartbeat_front_addr":"172.21.15.3:6821/1897849827","state":["exists","up"]},{"osd":1,"uuid":"646e882d-4ce4-4720-8c84-f71df563a35e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":926805498},{"type":"v1","addr":"172.21.15.3:6801","nonce":926805498}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":926805498},{"type":"v1","addr":"172.21.15.3:6803","nonce":926805498}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":926805498},{"type":"v1","addr":"172.21.15.3:6807","nonce":926805498}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":926805498},{"type":"v1","addr":"172.21.15.3:6805","nonce":926805498}]},"public_addr":"172.21.15.3:6801/926805498","cluster_addr":"172.21.15.3:6803/926805498","heartbeat_back_addr":"172.21.15.3:6807/926805498","heartbeat_front_addr":"172.21.15.3:6805/926805498","state":["exists","up"]},{"osd":2,"uuid":"1abbadb5-a570-4359-9643-acff3ede35a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":801295451},{"type":"v1","addr":"172.21.15.3:6809","nonce":801295451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":801295451},{"type":"v1","addr":"172.21.15.3:6811","nonce":801295451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":801295451},{"type":"v1","addr":"172.21.15.3:6815","nonce":801295451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":801295451},{"type":"v1","addr":"172.21.15.3:6813","nonce":801295451}]},"public_addr":"172.21.15.3:6809/801295451","cluster_addr":"172.21.15.3:6811/801295451","heartbeat_back_addr":"172.21.15.3:6815/801295451","heartbeat_front_addr":"172.21.15.3:6813/801295451","state":["exists","up"]},{"osd":3,"uuid":"62ac4dc1-2371-4dd2-acd3-e2dbb11aedf5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":1903389857},{"type":"v1","addr":"172.21.15.3:6825","nonce":1903389857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":1903389857},{"type":"v1","addr":"172.21.15.3:6827","nonce":1903389857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":1903389857},{"type":"v1","addr":"172.21.15.3:6831","nonce":1903389857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":1903389857},{"type":"v1","addr":"172.21.15.3:6829","nonce":1903389857}]},"public_addr":"172.21.15.3:6825/1903389857","cluster_addr":"172.21.15.3:6827/1903389857","heartbeat_back_addr":"172.21.15.3:6831/1903389857","heartbeat_front_addr":"172.21.15.3:6829/1903389857","state":["exists","up"]},{"osd":4,"uuid":"ae808398-8326-4d7c-9319-a2a0eafffb59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":2143624576},{"type":"v1","addr":"172.21.15.190:6801","nonce":2143624576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":2143624576},{"type":"v1","addr":"172.21.15.190:6803","nonce":2143624576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":2143624576},{"type":"v1","addr":"172.21.15.190:6807","nonce":2143624576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":2143624576},{"type":"v1","addr":"172.21.15.190:6805","nonce":2143624576}]},"public_addr":"172.21.15.190:6801/2143624576","cluster_addr":"172.21.15.190:6803/2143624576","heartbeat_back_addr":"172.21.15.190:6807/2143624576","heartbeat_front_addr":"172.21.15.190:6805/2143624576","state":["exists","up"]},{"osd":5,"uuid":"918adf50-c440-48ac-9a8d-4837562d7275","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":1766923126},{"type":"v1","addr":"172.21.15.190:6814","nonce":1766923126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":1766923126},{"type":"v1","addr":"172.21.15.190:6818","nonce":1766923126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":1766923126},{"type":"v1","addr":"172.21.15.190:6823","nonce":1766923126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":1766923126},{"type":"v1","addr":"172.21.15.190:6821","nonce":1766923126}]},"public_addr":"172.21.15.190:6814/1766923126","cluster_addr":"172.21.15.190:6818/1766923126","heartbeat_back_addr":"172.21.15.190:6823/1766923126","heartbeat_front_addr":"172.21.15.190:6821/1766923126","state":["exists","up"]},{"osd":6,"uuid":"ac2882bc-2a7c-463c-8f6c-76ff49828adc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":342414248},{"type":"v1","addr":"172.21.15.190:6825","nonce":342414248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":342414248},{"type":"v1","addr":"172.21.15.190:6827","nonce":342414248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":342414248},{"type":"v1","addr":"172.21.15.190:6831","nonce":342414248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":342414248},{"type":"v1","addr":"172.21.15.190:6829","nonce":342414248}]},"public_addr":"172.21.15.190:6825/342414248","cluster_addr":"172.21.15.190:6827/342414248","heartbeat_back_addr":"172.21.15.190:6831/342414248","heartbeat_front_addr":"172.21.15.190:6829/342414248","state":["exists","up"]},{"osd":7,"uuid":"dd4e767a-4730-4c70-8bd5-3f088140a5a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":537849891},{"type":"v1","addr":"172.21.15.190:6809","nonce":537849891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":537849891},{"type":"v1","addr":"172.21.15.190:6811","nonce":537849891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6817","nonce":537849891},{"type":"v1","addr":"172.21.15.190:6819","nonce":537849891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6813","nonce":537849891},{"type":"v1","addr":"172.21.15.190:6815","nonce":537849891}]},"public_addr":"172.21.15.190:6809/537849891","cluster_addr":"172.21.15.190:6811/537849891","heartbeat_back_addr":"172.21.15.190:6819/537849891","heartbeat_front_addr":"172.21.15.190:6815/537849891","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.889784+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.974002+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.973549+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.955396+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.928471+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.902418+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.958696+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.926474+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T06:36:41.550 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:41.570+0000 7f01e75f4640 1 -- 172.21.15.3:0/2920937168 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f01cc06be30 msgr2=0x7f01cc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:41.550 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:41.570+0000 7f01e75f4640 1 --2- 172.21.15.3:0/2920937168 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f01cc06be30 0x7f01cc06e2f0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f01d8006e80 tx=0x7f01d8006df0 comp rx=0 tx=0).stop 2024-05-09T06:36:41.550 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:41.570+0000 7f01e75f4640 1 -- 172.21.15.3:0/2920937168 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f01e00b1370 msgr2=0x7f01e00b17f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:41.550 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:41.570+0000 7f01e75f4640 1 --2- 172.21.15.3:0/2920937168 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f01e00b1370 0x7f01e00b17f0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f01d0000f80 tx=0x7f01d000a550 comp rx=0 tx=0).stop 2024-05-09T06:36:41.550 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:41.570+0000 7f01e75f4640 1 -- 172.21.15.3:0/2920937168 shutdown_connections 2024-05-09T06:36:41.550 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:41.570+0000 7f01e75f4640 1 --2- 172.21.15.3:0/2920937168 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f01cc06be30 0x7f01cc06e2f0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:41.550 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:41.570+0000 7f01e75f4640 1 --2- 172.21.15.3:0/2920937168 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f01e00b1370 0x7f01e00b17f0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:41.550 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:41.570+0000 7f01e75f4640 1 --2- 172.21.15.3:0/2920937168 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f01e015d780 0x7f01e015dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:41.550 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:41.570+0000 7f01e75f4640 1 --2- 172.21.15.3:0/2920937168 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f01c8004800 0x7f01e013c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:41.550 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:41.570+0000 7f01e75f4640 1 -- 172.21.15.3:0/2920937168 >> 172.21.15.3:0/2920937168 conn(0x7f01e006b430 msgr2=0x7f01e013d620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:41.550 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:41.570+0000 7f01e75f4640 1 -- 172.21.15.3:0/2920937168 shutdown_connections 2024-05-09T06:36:41.551 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:41.570+0000 7f01e75f4640 1 -- 172.21.15.3:0/2920937168 wait complete. 2024-05-09T06:36:41.561 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-09T06:35:54.819999+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-09T06:36:04.931314+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-09T06:36:06.200433+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-09T06:36:41.562 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-09T06:36:41.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:41.689+0000 7f9b7b438640 1 Processor -- start 2024-05-09T06:36:41.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:41.689+0000 7f9b7b438640 1 -- start start 2024-05-09T06:36:41.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:41.690+0000 7f9b7b438640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9b740b4840 0x7f9b740cf8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:41.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:41.690+0000 7f9b7b438640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9b74071860 con 0x7f9b74144a10 2024-05-09T06:36:41.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:41.690+0000 7f9b7b438640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9b74071b40 con 0x7f9b74072900 2024-05-09T06:36:41.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:41.690+0000 7f9b7b438640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f9b74072470 con 0x7f9b740b4840 2024-05-09T06:36:41.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:41.690+0000 7f9b791d4640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f9b74144a10 0x7f9b74071080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:51116/0 (socket says 172.21.15.3:51116) 2024-05-09T06:36:41.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:41.690+0000 7f9b791d4640 1 -- 172.21.15.3:0/2242794644 learned_addr learned my addr 172.21.15.3:0/2242794644 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-05-09T06:36:41.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:41.690+0000 7f9b799d5640 1 --2- 172.21.15.3:0/2242794644 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9b740b4840 0x7f9b740cf8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:41.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:41.690+0000 7f9b73fff640 1 -- 172.21.15.3:0/2242794644 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2481785963 0 0) 0x7f9b74071860 con 0x7f9b74144a10 2024-05-09T06:36:41.672 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:41.690+0000 7f9b73fff640 1 -- 172.21.15.3:0/2242794644 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9b5c005730 con 0x7f9b74144a10 2024-05-09T06:36:41.672 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:41.690+0000 7f9b73fff640 1 -- 172.21.15.3:0/2242794644 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 262172770 0 0) 0x7f9b74071b40 con 0x7f9b74072900 2024-05-09T06:36:41.672 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:41.690+0000 7f9b73fff640 1 -- 172.21.15.3:0/2242794644 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9b74071860 con 0x7f9b74072900 2024-05-09T06:36:41.672 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:41.690+0000 7f9b73fff640 1 -- 172.21.15.3:0/2242794644 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2074239968 0 0) 0x7f9b5c005730 con 0x7f9b74144a10 2024-05-09T06:36:41.672 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:41.690+0000 7f9b799d5640 1 -- 172.21.15.3:0/2242794644 >> v1:172.21.15.190:6789/0 conn(0x7f9b74072900 legacy=0x7f9b740b4130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:41.672 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:41.690+0000 7f9b799d5640 1 -- 172.21.15.3:0/2242794644 >> v1:172.21.15.3:6789/0 conn(0x7f9b74144a10 legacy=0x7f9b74071080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:41.672 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:41.690+0000 7f9b799d5640 1 -- 172.21.15.3:0/2242794644 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b74071e20 con 0x7f9b740b4840 2024-05-09T06:36:41.672 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:41.691+0000 7f9b799d5640 1 --2- 172.21.15.3:0/2242794644 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9b740b4840 0x7f9b740cf8a0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f9b740725f0 tx=0x7f9b6402a5d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c7adeb16b795d550 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:41.672 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:41.691+0000 7f9b73fff640 1 -- 172.21.15.3:0/2242794644 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b6c001670 con 0x7f9b740b4840 2024-05-09T06:36:41.672 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:41.691+0000 7f9b73fff640 1 -- 172.21.15.3:0/2242794644 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b64007540 con 0x7f9b740b4840 2024-05-09T06:36:41.672 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:41.691+0000 7f9b73fff640 1 -- 172.21.15.3:0/2242794644 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b640078f0 con 0x7f9b740b4840 2024-05-09T06:36:41.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:41.692+0000 7f9b7b438640 1 -- 172.21.15.3:0/2242794644 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9b740b4840 msgr2=0x7f9b740cf8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:41.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:41.692+0000 7f9b7b438640 1 --2- 172.21.15.3:0/2242794644 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9b740b4840 0x7f9b740cf8a0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f9b740725f0 tx=0x7f9b6402a5d0 comp rx=0 tx=0).stop 2024-05-09T06:36:41.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:41.692+0000 7f9b7b438640 1 -- 172.21.15.3:0/2242794644 shutdown_connections 2024-05-09T06:36:41.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:41.692+0000 7f9b7b438640 1 --2- 172.21.15.3:0/2242794644 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9b740b4840 0x7f9b740cf8a0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:41.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:41.692+0000 7f9b7b438640 1 -- 172.21.15.3:0/2242794644 >> 172.21.15.3:0/2242794644 conn(0x7f9b740b1c20 msgr2=0x7f9b740b2030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:41.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:41.692+0000 7f9b7b438640 1 -- 172.21.15.3:0/2242794644 shutdown_connections 2024-05-09T06:36:41.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:41.692+0000 7f9b7b438640 1 -- 172.21.15.3:0/2242794644 wait complete. 2024-05-09T06:36:41.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:41.693+0000 7f9b7b438640 1 Processor -- start 2024-05-09T06:36:41.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:41.693+0000 7f9b7b438640 1 -- start start 2024-05-09T06:36:41.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:41.694+0000 7f9b7b438640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9b74072900 0x7f9b74150f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:41.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:41.694+0000 7f9b7b438640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9b740b4840 0x7f9b7415df30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:41.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:41.694+0000 7f9b7b438640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9b74144a10 0x7f9b7415e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:41.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:41.694+0000 7f9b7b438640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f9b740b31a0 con 0x7f9b74072900 2024-05-09T06:36:41.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:41.694+0000 7f9b7b438640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f9b740b3020 con 0x7f9b740b4840 2024-05-09T06:36:41.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:41.694+0000 7f9b7b438640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f9b740b3320 con 0x7f9b74144a10 2024-05-09T06:36:41.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:41.694+0000 7f9b791d4640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9b74072900 0x7f9b74150f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:41.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:41.694+0000 7f9b791d4640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9b74072900 0x7f9b74150f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46512/0 (socket says 172.21.15.3:46512) 2024-05-09T06:36:41.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:41.694+0000 7f9b791d4640 1 -- 172.21.15.3:0/2187083978 learned_addr learned my addr 172.21.15.3:0/2187083978 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:36:41.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:41.694+0000 7f9b789d3640 1 --2- 172.21.15.3:0/2187083978 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9b740b4840 0x7f9b7415df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:41.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:41.694+0000 7f9b791d4640 1 -- 172.21.15.3:0/2187083978 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9b74144a10 msgr2=0x7f9b7415e470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:41.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:41.694+0000 7f9b791d4640 1 --2- 172.21.15.3:0/2187083978 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9b74144a10 0x7f9b7415e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:41.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:41.694+0000 7f9b791d4640 1 -- 172.21.15.3:0/2187083978 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9b740b4840 msgr2=0x7f9b7415df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:41.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:41.694+0000 7f9b791d4640 1 --2- 172.21.15.3:0/2187083978 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9b740b4840 0x7f9b7415df30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:41.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:41.694+0000 7f9b791d4640 1 -- 172.21.15.3:0/2187083978 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b740ae280 con 0x7f9b74072900 2024-05-09T06:36:41.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:41.695+0000 7f9b791d4640 1 --2- 172.21.15.3:0/2187083978 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9b74072900 0x7f9b74150f00 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f9b6c00c5d0 tx=0x7f9b6c00ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:41.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:41.695+0000 7f9b71ffb640 1 -- 172.21.15.3:0/2187083978 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b6c017070 con 0x7f9b74072900 2024-05-09T06:36:41.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:41.695+0000 7f9b71ffb640 1 -- 172.21.15.3:0/2187083978 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b6c005d00 con 0x7f9b74072900 2024-05-09T06:36:41.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:41.695+0000 7f9b7b438640 1 -- 172.21.15.3:0/2187083978 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b74151af0 con 0x7f9b74072900 2024-05-09T06:36:41.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:41.695+0000 7f9b71ffb640 1 -- 172.21.15.3:0/2187083978 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b6c0072b0 con 0x7f9b74072900 2024-05-09T06:36:41.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:41.695+0000 7f9b7b438640 1 -- 172.21.15.3:0/2187083978 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b740ae7e0 con 0x7f9b74072900 2024-05-09T06:36:41.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:41.696+0000 7f9b7b438640 1 -- 172.21.15.3:0/2187083978 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b740af180 con 0x7f9b74072900 2024-05-09T06:36:41.680 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:41.696+0000 7f9b71ffb640 1 -- 172.21.15.3:0/2187083978 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9b6c01a070 con 0x7f9b74072900 2024-05-09T06:36:41.680 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:41.697+0000 7f9b71ffb640 1 --2- 172.21.15.3:0/2187083978 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f9b6006be30 0x7f9b6006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:41.680 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:41.697+0000 7f9b71ffb640 1 -- 172.21.15.3:0/2187083978 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f9b6c08a280 con 0x7f9b74072900 2024-05-09T06:36:41.680 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:41.697+0000 7f9b789d3640 1 --2- 172.21.15.3:0/2187083978 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f9b6006be30 0x7f9b6006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:41.680 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:41.698+0000 7f9b789d3640 1 --2- 172.21.15.3:0/2187083978 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f9b6006be30 0x7f9b6006e2f0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f9b680010e0 tx=0x7f9b68009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:41.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:41.701+0000 7f9b71ffb640 1 -- 172.21.15.3:0/2187083978 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f9b740af180 con 0x7f9b74072900 2024-05-09T06:36:41.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:41.884+0000 7f9b7b438640 1 -- 172.21.15.3:0/2187083978 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f9b74140a70 con 0x7f9b74072900 2024-05-09T06:36:41.865 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:41.885+0000 7f9b71ffb640 1 -- 172.21.15.3:0/2187083978 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7f9b74151af0 con 0x7f9b74072900 2024-05-09T06:36:41.865 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 1 2024-05-09T06:36:41.867 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:41.887+0000 7f9b7b438640 1 -- 172.21.15.3:0/2187083978 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f9b6006be30 msgr2=0x7f9b6006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:41.867 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:41.887+0000 7f9b7b438640 1 --2- 172.21.15.3:0/2187083978 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f9b6006be30 0x7f9b6006e2f0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f9b680010e0 tx=0x7f9b68009040 comp rx=0 tx=0).stop 2024-05-09T06:36:41.867 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:41.887+0000 7f9b7b438640 1 -- 172.21.15.3:0/2187083978 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9b74072900 msgr2=0x7f9b74150f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:41.867 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:41.887+0000 7f9b7b438640 1 --2- 172.21.15.3:0/2187083978 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9b74072900 0x7f9b74150f00 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f9b6c00c5d0 tx=0x7f9b6c00ca90 comp rx=0 tx=0).stop 2024-05-09T06:36:41.867 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:41.887+0000 7f9b7b438640 1 -- 172.21.15.3:0/2187083978 shutdown_connections 2024-05-09T06:36:41.867 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:41.887+0000 7f9b7b438640 1 --2- 172.21.15.3:0/2187083978 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f9b6006be30 0x7f9b6006e2f0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:41.867 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:41.887+0000 7f9b7b438640 1 --2- 172.21.15.3:0/2187083978 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9b74144a10 0x7f9b7415e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:41.868 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:41.887+0000 7f9b7b438640 1 --2- 172.21.15.3:0/2187083978 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9b740b4840 0x7f9b7415df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:41.868 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:41.887+0000 7f9b7b438640 1 --2- 172.21.15.3:0/2187083978 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9b74072900 0x7f9b74150f00 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:41.868 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:41.887+0000 7f9b7b438640 1 -- 172.21.15.3:0/2187083978 >> 172.21.15.3:0/2187083978 conn(0x7f9b740b1c20 msgr2=0x7f9b740b0f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:41.868 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:41.887+0000 7f9b7b438640 1 -- 172.21.15.3:0/2187083978 shutdown_connections 2024-05-09T06:36:41.868 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:41.887+0000 7f9b7b438640 1 -- 172.21.15.3:0/2187083978 wait complete. 2024-05-09T06:36:41.878 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-09T06:36:41.984 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.005+0000 7ffa1c7e7640 1 Processor -- start 2024-05-09T06:36:41.985 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.005+0000 7ffa1c7e7640 1 -- start start 2024-05-09T06:36:41.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.006+0000 7ffa1c7e7640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffa1415e500 0x7ffa1409aa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:41.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.006+0000 7ffa1c7e7640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffa1406caa0 con 0x7ffa140d11b0 2024-05-09T06:36:41.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.006+0000 7ffa1c7e7640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffa1406c7c0 con 0x7ffa140b4620 2024-05-09T06:36:41.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.006+0000 7ffa1c7e7640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7ffa1406c570 con 0x7ffa1415e500 2024-05-09T06:36:41.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.006+0000 7ffa1ad84640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7ffa140d11b0 0x7ffa140cd610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:51120/0 (socket says 172.21.15.3:51120) 2024-05-09T06:36:41.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.006+0000 7ffa1ad84640 1 -- 172.21.15.3:0/2293199565 learned_addr learned my addr 172.21.15.3:0/2293199565 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-05-09T06:36:41.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.006+0000 7ffa1a583640 1 --2- 172.21.15.3:0/2293199565 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffa1415e500 0x7ffa1409aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:41.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.006+0000 7ffa1a583640 1 -- 172.21.15.3:0/2293199565 >> v1:172.21.15.190:6789/0 conn(0x7ffa140b4620 legacy=0x7ffa140ccf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:41.987 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.006+0000 7ffa1a583640 1 -- 172.21.15.3:0/2293199565 >> v1:172.21.15.3:6789/0 conn(0x7ffa140d11b0 legacy=0x7ffa140cd610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:41.987 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.006+0000 7ffa1a583640 1 -- 172.21.15.3:0/2293199565 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa140cc700 con 0x7ffa1415e500 2024-05-09T06:36:41.988 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.007+0000 7ffa1a583640 1 --2- 172.21.15.3:0/2293199565 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffa1415e500 0x7ffa1409aa30 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7ffa100097d0 tx=0x7ffa1002a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3d99bf5efb5fb0f2 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:41.988 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.007+0000 7ffa19581640 1 -- 172.21.15.3:0/2293199565 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa10037070 con 0x7ffa1415e500 2024-05-09T06:36:41.988 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.007+0000 7ffa19581640 1 -- 172.21.15.3:0/2293199565 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa10032440 con 0x7ffa1415e500 2024-05-09T06:36:41.988 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.007+0000 7ffa19581640 1 -- 172.21.15.3:0/2293199565 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa10032810 con 0x7ffa1415e500 2024-05-09T06:36:41.988 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.008+0000 7ffa1c7e7640 1 -- 172.21.15.3:0/2293199565 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffa1415e500 msgr2=0x7ffa1409aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:41.988 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.008+0000 7ffa1c7e7640 1 --2- 172.21.15.3:0/2293199565 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffa1415e500 0x7ffa1409aa30 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7ffa100097d0 tx=0x7ffa1002a210 comp rx=0 tx=0).stop 2024-05-09T06:36:41.988 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.008+0000 7ffa1c7e7640 1 -- 172.21.15.3:0/2293199565 shutdown_connections 2024-05-09T06:36:41.988 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.008+0000 7ffa1c7e7640 1 --2- 172.21.15.3:0/2293199565 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffa1415e500 0x7ffa1409aa30 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:41.988 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.008+0000 7ffa1c7e7640 1 -- 172.21.15.3:0/2293199565 >> 172.21.15.3:0/2293199565 conn(0x7ffa140b3b00 msgr2=0x7ffa140b3f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:41.989 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.008+0000 7ffa1c7e7640 1 -- 172.21.15.3:0/2293199565 shutdown_connections 2024-05-09T06:36:41.989 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.008+0000 7ffa1c7e7640 1 -- 172.21.15.3:0/2293199565 wait complete. 2024-05-09T06:36:41.989 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.009+0000 7ffa1c7e7640 1 Processor -- start 2024-05-09T06:36:41.989 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.009+0000 7ffa1c7e7640 1 -- start start 2024-05-09T06:36:41.991 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.010+0000 7ffa1c7e7640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffa140b4620 0x7ffa140c6990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:41.991 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.010+0000 7ffa1c7e7640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffa140d11b0 0x7ffa1415fe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:41.991 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.010+0000 7ffa1c7e7640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffa1415e500 0x7ffa14155800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:41.991 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.010+0000 7ffa1c7e7640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7ffa1409b6f0 con 0x7ffa140b4620 2024-05-09T06:36:41.992 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.010+0000 7ffa1c7e7640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7ffa1415e8e0 con 0x7ffa1415e500 2024-05-09T06:36:41.992 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.010+0000 7ffa1c7e7640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7ffa1409b870 con 0x7ffa140d11b0 2024-05-09T06:36:41.992 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.010+0000 7ffa1a583640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffa140b4620 0x7ffa140c6990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:41.992 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.010+0000 7ffa1a583640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffa140b4620 0x7ffa140c6990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46516/0 (socket says 172.21.15.3:46516) 2024-05-09T06:36:41.992 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.010+0000 7ffa1a583640 1 -- 172.21.15.3:0/1268863334 learned_addr learned my addr 172.21.15.3:0/1268863334 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:36:41.992 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.010+0000 7ffa19d82640 1 --2- 172.21.15.3:0/1268863334 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffa140d11b0 0x7ffa1415fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:41.992 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.010+0000 7ffa1ad84640 1 --2- 172.21.15.3:0/1268863334 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffa1415e500 0x7ffa14155800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:41.992 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.010+0000 7ffa1a583640 1 -- 172.21.15.3:0/1268863334 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffa140d11b0 msgr2=0x7ffa1415fe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:41.992 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.010+0000 7ffa1a583640 1 --2- 172.21.15.3:0/1268863334 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffa140d11b0 0x7ffa1415fe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:41.992 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.010+0000 7ffa1a583640 1 -- 172.21.15.3:0/1268863334 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffa1415e500 msgr2=0x7ffa14155800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:41.992 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.010+0000 7ffa1a583640 1 --2- 172.21.15.3:0/1268863334 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffa1415e500 0x7ffa14155800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:41.992 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.010+0000 7ffa1a583640 1 -- 172.21.15.3:0/1268863334 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa141605b0 con 0x7ffa140b4620 2024-05-09T06:36:41.993 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.010+0000 7ffa19d82640 1 --2- 172.21.15.3:0/1268863334 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffa140d11b0 0x7ffa1415fe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:36:41.993 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.011+0000 7ffa1a583640 1 --2- 172.21.15.3:0/1268863334 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffa140b4620 0x7ffa140c6990 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7ffa10035040 tx=0x7ffa100046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:41.993 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.011+0000 7ffa0affd640 1 -- 172.21.15.3:0/1268863334 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa10037040 con 0x7ffa140b4620 2024-05-09T06:36:41.993 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.011+0000 7ffa0affd640 1 -- 172.21.15.3:0/1268863334 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa10030770 con 0x7ffa140b4620 2024-05-09T06:36:41.993 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.011+0000 7ffa0affd640 1 -- 172.21.15.3:0/1268863334 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa10030b60 con 0x7ffa140b4620 2024-05-09T06:36:41.993 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.011+0000 7ffa1c7e7640 1 -- 172.21.15.3:0/1268863334 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa14160890 con 0x7ffa140b4620 2024-05-09T06:36:41.993 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.011+0000 7ffa1c7e7640 1 -- 172.21.15.3:0/1268863334 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa14160e50 con 0x7ffa140b4620 2024-05-09T06:36:41.994 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.012+0000 7ffa1c7e7640 1 -- 172.21.15.3:0/1268863334 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa14069550 con 0x7ffa140b4620 2024-05-09T06:36:41.994 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.012+0000 7ffa0affd640 1 -- 172.21.15.3:0/1268863334 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ffa10044020 con 0x7ffa140b4620 2024-05-09T06:36:41.994 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.013+0000 7ffa0affd640 1 --2- 172.21.15.3:0/1268863334 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7ff9f806be30 0x7ff9f806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:41.994 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.013+0000 7ffa0affd640 1 -- 172.21.15.3:0/1268863334 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7ffa100ab380 con 0x7ffa140b4620 2024-05-09T06:36:41.994 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.013+0000 7ffa19d82640 1 --2- 172.21.15.3:0/1268863334 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7ff9f806be30 0x7ff9f806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:41.994 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.014+0000 7ffa19d82640 1 --2- 172.21.15.3:0/1268863334 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7ff9f806be30 0x7ff9f806e2f0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7ffa04007fd0 tx=0x7ffa04009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:41.997 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.017+0000 7ffa0affd640 1 -- 172.21.15.3:0/1268863334 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7ffa10032e20 con 0x7ffa140b4620 2024-05-09T06:36:42.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.202+0000 7ffa1c7e7640 1 -- 172.21.15.3:0/1268863334 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7ffa14074fb0 con 0x7ffa140b4620 2024-05-09T06:36:42.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.202+0000 7ffa0affd640 1 -- 172.21.15.3:0/1268863334 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7ffa10079dc0 con 0x7ffa140b4620 2024-05-09T06:36:42.182 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 32 2024-05-09T06:36:42.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.204+0000 7ffa1c7e7640 1 -- 172.21.15.3:0/1268863334 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7ff9f806be30 msgr2=0x7ff9f806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:42.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.204+0000 7ffa1c7e7640 1 --2- 172.21.15.3:0/1268863334 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7ff9f806be30 0x7ff9f806e2f0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7ffa04007fd0 tx=0x7ffa04009040 comp rx=0 tx=0).stop 2024-05-09T06:36:42.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.204+0000 7ffa1c7e7640 1 -- 172.21.15.3:0/1268863334 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffa140b4620 msgr2=0x7ffa140c6990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:42.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.204+0000 7ffa1c7e7640 1 --2- 172.21.15.3:0/1268863334 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffa140b4620 0x7ffa140c6990 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7ffa10035040 tx=0x7ffa100046c0 comp rx=0 tx=0).stop 2024-05-09T06:36:42.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.204+0000 7ffa1c7e7640 1 -- 172.21.15.3:0/1268863334 shutdown_connections 2024-05-09T06:36:42.185 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.204+0000 7ffa1c7e7640 1 --2- 172.21.15.3:0/1268863334 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7ff9f806be30 0x7ff9f806e2f0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:42.185 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.204+0000 7ffa1c7e7640 1 --2- 172.21.15.3:0/1268863334 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffa1415e500 0x7ffa14155800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:42.185 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.204+0000 7ffa1c7e7640 1 --2- 172.21.15.3:0/1268863334 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffa140d11b0 0x7ffa1415fe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:42.185 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.204+0000 7ffa1c7e7640 1 --2- 172.21.15.3:0/1268863334 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffa140b4620 0x7ffa140c6990 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:42.185 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.204+0000 7ffa1c7e7640 1 -- 172.21.15.3:0/1268863334 >> 172.21.15.3:0/1268863334 conn(0x7ffa140b3b00 msgr2=0x7ffa140cf200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:42.185 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.204+0000 7ffa1c7e7640 1 -- 172.21.15.3:0/1268863334 shutdown_connections 2024-05-09T06:36:42.185 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.204+0000 7ffa1c7e7640 1 -- 172.21.15.3:0/1268863334 wait complete. 2024-05-09T06:36:42.195 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-09T06:36:42.302 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.322+0000 7fa716415640 1 Processor -- start 2024-05-09T06:36:42.302 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.322+0000 7fa716415640 1 -- start start 2024-05-09T06:36:42.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.323+0000 7fa716415640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fa710144a10 0x7fa7100cf8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:42.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.323+0000 7fa716415640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa710071860 con 0x7fa7100b4130 2024-05-09T06:36:42.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.323+0000 7fa716415640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa710071b40 con 0x7fa710072900 2024-05-09T06:36:42.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.323+0000 7fa716415640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7fa710072470 con 0x7fa710144a10 2024-05-09T06:36:42.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.323+0000 7fa7149b2640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fa7100b4130 0x7fa71015b5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:51136/0 (socket says 172.21.15.3:51136) 2024-05-09T06:36:42.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.323+0000 7fa7149b2640 1 -- 172.21.15.3:0/2854768885 learned_addr learned my addr 172.21.15.3:0/2854768885 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-05-09T06:36:42.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.323+0000 7fa70f7fe640 1 -- 172.21.15.3:0/2854768885 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 463388391 0 0) 0x7fa710071860 con 0x7fa7100b4130 2024-05-09T06:36:42.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.324+0000 7fa70f7fe640 1 -- 172.21.15.3:0/2854768885 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa6fc003520 con 0x7fa7100b4130 2024-05-09T06:36:42.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.324+0000 7fa70ffff640 1 --2- 172.21.15.3:0/2854768885 >> v2:172.21.15.190:3300/0 conn(0x7fa710144a10 0x7fa7100cf8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:42.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.324+0000 7fa70f7fe640 1 -- 172.21.15.3:0/2854768885 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2923074866 0 0) 0x7fa6fc003520 con 0x7fa7100b4130 2024-05-09T06:36:42.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.324+0000 7fa70f7fe640 1 -- 172.21.15.3:0/2854768885 >> v2:172.21.15.190:3300/0 conn(0x7fa710144a10 msgr2=0x7fa7100cf8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:42.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.324+0000 7fa70f7fe640 1 --2- 172.21.15.3:0/2854768885 >> v2:172.21.15.190:3300/0 conn(0x7fa710144a10 0x7fa7100cf8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:42.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.324+0000 7fa70f7fe640 1 -- 172.21.15.3:0/2854768885 >> v1:172.21.15.190:6789/0 conn(0x7fa710072900 legacy=0x7fa710071080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:42.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.324+0000 7fa70f7fe640 1 -- 172.21.15.3:0/2854768885 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa710071e20 con 0x7fa7100b4130 2024-05-09T06:36:42.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.324+0000 7fa70f7fe640 1 -- 172.21.15.3:0/2854768885 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 563312031 0 0) 0x7fa7080019f0 con 0x7fa7100b4130 2024-05-09T06:36:42.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.324+0000 7fa70f7fe640 1 -- 172.21.15.3:0/2854768885 >> v1:172.21.15.3:6789/0 conn(0x7fa7100b4130 legacy=0x7fa71015b5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:42.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.324+0000 7fa70f7fe640 1 --2- 172.21.15.3:0/2854768885 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa6fc0039b0 0x7fa6fc003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:42.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.324+0000 7fa70f7fe640 1 --2- 172.21.15.3:0/2854768885 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa6fc004800 0x7fa6fc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:42.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.324+0000 7fa70f7fe640 1 --2- 172.21.15.3:0/2854768885 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa6fc0051c0 0x7fa6fc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:42.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.324+0000 7fa70f7fe640 1 -- 172.21.15.3:0/2854768885 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fa710072470 con 0x7fa6fc004800 2024-05-09T06:36:42.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.324+0000 7fa70f7fe640 1 -- 172.21.15.3:0/2854768885 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fa6fc00c160 con 0x7fa6fc0039b0 2024-05-09T06:36:42.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.324+0000 7fa70f7fe640 1 -- 172.21.15.3:0/2854768885 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fa6fc00c360 con 0x7fa6fc0051c0 2024-05-09T06:36:42.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.324+0000 7fa707fff640 1 --2- 172.21.15.3:0/2854768885 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa6fc004800 0x7fa6fc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:42.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.324+0000 7fa70ffff640 1 --2- 172.21.15.3:0/2854768885 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa6fc0039b0 0x7fa6fc003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:42.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.324+0000 7fa7149b2640 1 --2- 172.21.15.3:0/2854768885 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa6fc0051c0 0x7fa6fc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:42.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.324+0000 7fa707fff640 1 -- 172.21.15.3:0/2854768885 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa6fc0051c0 msgr2=0x7fa6fc00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:42.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.324+0000 7fa707fff640 1 --2- 172.21.15.3:0/2854768885 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa6fc0051c0 0x7fa6fc00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:42.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.324+0000 7fa707fff640 1 -- 172.21.15.3:0/2854768885 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa6fc0039b0 msgr2=0x7fa6fc003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:42.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.324+0000 7fa707fff640 1 --2- 172.21.15.3:0/2854768885 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa6fc0039b0 0x7fa6fc003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:42.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.324+0000 7fa707fff640 1 -- 172.21.15.3:0/2854768885 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6fc003520 con 0x7fa6fc004800 2024-05-09T06:36:42.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.324+0000 7fa7149b2640 1 -- 172.21.15.3:0/2854768885 reap_dead start 2024-05-09T06:36:42.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.325+0000 7fa707fff640 1 --2- 172.21.15.3:0/2854768885 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa6fc004800 0x7fa6fc004c80 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fa6f8001330 tx=0x7fa6fc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=799fed01fe844d94 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:42.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.325+0000 7fa70f7fe640 1 -- 172.21.15.3:0/2854768885 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6f8001370 con 0x7fa6fc004800 2024-05-09T06:36:42.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.325+0000 7fa70f7fe640 1 -- 172.21.15.3:0/2854768885 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6f802cd60 con 0x7fa6fc004800 2024-05-09T06:36:42.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.325+0000 7fa70f7fe640 1 -- 172.21.15.3:0/2854768885 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6f8005310 con 0x7fa6fc004800 2024-05-09T06:36:42.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.326+0000 7fa716415640 1 -- 172.21.15.3:0/2854768885 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa6fc004800 msgr2=0x7fa6fc004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:42.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.326+0000 7fa716415640 1 --2- 172.21.15.3:0/2854768885 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa6fc004800 0x7fa6fc004c80 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fa6f8001330 tx=0x7fa6fc005830 comp rx=0 tx=0).stop 2024-05-09T06:36:42.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.326+0000 7fa716415640 1 -- 172.21.15.3:0/2854768885 shutdown_connections 2024-05-09T06:36:42.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.326+0000 7fa716415640 1 --2- 172.21.15.3:0/2854768885 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa6fc004800 0x7fa6fc004c80 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:42.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.326+0000 7fa716415640 1 -- 172.21.15.3:0/2854768885 >> 172.21.15.3:0/2854768885 conn(0x7fa7100b1c20 msgr2=0x7fa7100b2030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:42.309 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.326+0000 7fa716415640 1 -- 172.21.15.3:0/2854768885 shutdown_connections 2024-05-09T06:36:42.309 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.326+0000 7fa716415640 1 -- 172.21.15.3:0/2854768885 wait complete. 2024-05-09T06:36:42.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.327+0000 7fa716415640 1 Processor -- start 2024-05-09T06:36:42.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.327+0000 7fa716415640 1 -- start start 2024-05-09T06:36:42.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.328+0000 7fa716415640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa6fc004800 0x7fa7100c7c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:42.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.328+0000 7fa716415640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa7100ad8a0 0x7fa7100add20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:42.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.328+0000 7fa716415640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa710138f00 0x7fa710139380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:42.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.328+0000 7fa716415640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fa7100cfa20 con 0x7fa710138f00 2024-05-09T06:36:42.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.328+0000 7fa716415640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fa7100cf8a0 con 0x7fa6fc004800 2024-05-09T06:36:42.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.328+0000 7fa716415640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fa7100cfba0 con 0x7fa7100ad8a0 2024-05-09T06:36:42.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.328+0000 7fa7149b2640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa710138f00 0x7fa710139380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:42.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.328+0000 7fa7149b2640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa710138f00 0x7fa710139380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46530/0 (socket says 172.21.15.3:46530) 2024-05-09T06:36:42.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.328+0000 7fa7149b2640 1 -- 172.21.15.3:0/3455620795 learned_addr learned my addr 172.21.15.3:0/3455620795 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:36:42.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.328+0000 7fa707fff640 1 --2- 172.21.15.3:0/3455620795 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa7100ad8a0 0x7fa7100add20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:42.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.328+0000 7fa70ffff640 1 --2- 172.21.15.3:0/3455620795 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa6fc004800 0x7fa7100c7c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:42.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.328+0000 7fa7149b2640 1 -- 172.21.15.3:0/3455620795 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa7100ad8a0 msgr2=0x7fa7100add20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:42.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.328+0000 7fa7149b2640 1 --2- 172.21.15.3:0/3455620795 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa7100ad8a0 0x7fa7100add20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:42.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.328+0000 7fa7149b2640 1 -- 172.21.15.3:0/3455620795 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa6fc004800 msgr2=0x7fa7100c7c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:42.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.328+0000 7fa7149b2640 1 --2- 172.21.15.3:0/3455620795 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa6fc004800 0x7fa7100c7c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:42.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.328+0000 7fa7149b2640 1 -- 172.21.15.3:0/3455620795 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7100ae590 con 0x7fa710138f00 2024-05-09T06:36:42.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.328+0000 7fa707fff640 1 --2- 172.21.15.3:0/3455620795 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa7100ad8a0 0x7fa7100add20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:36:42.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.329+0000 7fa7149b2640 1 --2- 172.21.15.3:0/3455620795 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa710138f00 0x7fa710139380 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fa7100752d0 tx=0x7fa708007b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:42.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.329+0000 7fa70d7fa640 1 -- 172.21.15.3:0/3455620795 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa70800ab20 con 0x7fa710138f00 2024-05-09T06:36:42.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.329+0000 7fa70d7fa640 1 -- 172.21.15.3:0/3455620795 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa708018070 con 0x7fa710138f00 2024-05-09T06:36:42.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.329+0000 7fa70d7fa640 1 -- 172.21.15.3:0/3455620795 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7080133f0 con 0x7fa710138f00 2024-05-09T06:36:42.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.329+0000 7fa716415640 1 -- 172.21.15.3:0/3455620795 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7100ae870 con 0x7fa710138f00 2024-05-09T06:36:42.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.329+0000 7fa716415640 1 -- 172.21.15.3:0/3455620795 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7100aee30 con 0x7fa710138f00 2024-05-09T06:36:42.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.330+0000 7fa716415640 1 -- 172.21.15.3:0/3455620795 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa710069550 con 0x7fa710138f00 2024-05-09T06:36:42.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.330+0000 7fa70d7fa640 1 -- 172.21.15.3:0/3455620795 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa708013590 con 0x7fa710138f00 2024-05-09T06:36:42.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.331+0000 7fa70d7fa640 1 --2- 172.21.15.3:0/3455620795 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fa6f006be30 0x7fa6f006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:42.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.331+0000 7fa70d7fa640 1 -- 172.21.15.3:0/3455620795 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fa70808b3c0 con 0x7fa710138f00 2024-05-09T06:36:42.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.331+0000 7fa70ffff640 1 --2- 172.21.15.3:0/3455620795 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fa6f006be30 0x7fa6f006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:42.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.332+0000 7fa70ffff640 1 --2- 172.21.15.3:0/3455620795 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fa6f006be30 0x7fa6f006e2f0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fa700009a00 tx=0x7fa700008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:42.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.335+0000 7fa70d7fa640 1 -- 172.21.15.3:0/3455620795 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fa708059d70 con 0x7fa710138f00 2024-05-09T06:36:42.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.522+0000 7fa716415640 1 -- 172.21.15.3:0/3455620795 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fa71006d800 con 0x7fa710138f00 2024-05-09T06:36:42.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.522+0000 7fa70d7fa640 1 -- 172.21.15.3:0/3455620795 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7fa708059670 con 0x7fa710138f00 2024-05-09T06:36:42.502 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 64 2024-05-09T06:36:42.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.524+0000 7fa716415640 1 -- 172.21.15.3:0/3455620795 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fa6f006be30 msgr2=0x7fa6f006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:42.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.524+0000 7fa716415640 1 --2- 172.21.15.3:0/3455620795 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fa6f006be30 0x7fa6f006e2f0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fa700009a00 tx=0x7fa700008040 comp rx=0 tx=0).stop 2024-05-09T06:36:42.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.524+0000 7fa716415640 1 -- 172.21.15.3:0/3455620795 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa710138f00 msgr2=0x7fa710139380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:42.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.524+0000 7fa716415640 1 --2- 172.21.15.3:0/3455620795 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa710138f00 0x7fa710139380 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fa7100752d0 tx=0x7fa708007b20 comp rx=0 tx=0).stop 2024-05-09T06:36:42.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.524+0000 7fa716415640 1 -- 172.21.15.3:0/3455620795 shutdown_connections 2024-05-09T06:36:42.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.524+0000 7fa716415640 1 --2- 172.21.15.3:0/3455620795 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fa6f006be30 0x7fa6f006e2f0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:42.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.524+0000 7fa716415640 1 --2- 172.21.15.3:0/3455620795 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa710138f00 0x7fa710139380 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:42.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.524+0000 7fa716415640 1 --2- 172.21.15.3:0/3455620795 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa7100ad8a0 0x7fa7100add20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:42.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.524+0000 7fa716415640 1 --2- 172.21.15.3:0/3455620795 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa6fc004800 0x7fa7100c7c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:42.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.524+0000 7fa716415640 1 -- 172.21.15.3:0/3455620795 >> 172.21.15.3:0/3455620795 conn(0x7fa7100b1c20 msgr2=0x7fa7100b45e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:42.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.524+0000 7fa716415640 1 -- 172.21.15.3:0/3455620795 shutdown_connections 2024-05-09T06:36:42.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.524+0000 7fa716415640 1 -- 172.21.15.3:0/3455620795 wait complete. 2024-05-09T06:36:42.515 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-09T06:36:42.622 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.642+0000 7fa51f02a640 1 Processor -- start 2024-05-09T06:36:42.622 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.642+0000 7fa51f02a640 1 -- start start 2024-05-09T06:36:42.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.643+0000 7fa51f02a640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa5180d1fd0 0x7fa5180d27d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:42.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.643+0000 7fa51f02a640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa5180d2d10 0x7fa5180cf530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:42.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.643+0000 7fa51f02a640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa5180d15d0 con 0x7fa5180d11f0 2024-05-09T06:36:42.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.643+0000 7fa51f02a640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fa5180cf0a0 con 0x7fa5180d1fd0 2024-05-09T06:36:42.624 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.643+0000 7fa51f02a640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fa5180ced70 con 0x7fa5180d2d10 2024-05-09T06:36:42.624 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.643+0000 7fa50ffff640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa5180d1fd0 0x7fa5180d27d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:42.624 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.643+0000 7fa50ffff640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa5180d1fd0 0x7fa5180d27d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46544/0 (socket says 172.21.15.3:46544) 2024-05-09T06:36:42.624 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.643+0000 7fa50ffff640 1 -- 172.21.15.3:0/1594955857 learned_addr learned my addr 172.21.15.3:0/1594955857 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:36:42.624 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.643+0000 7fa51d5c7640 1 --2- 172.21.15.3:0/1594955857 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa5180d2d10 0x7fa5180cf530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:42.624 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.643+0000 7fa50ffff640 1 -- 172.21.15.3:0/1594955857 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa5180d2d10 msgr2=0x7fa5180cf530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:42.624 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.643+0000 7fa50ffff640 1 --2- 172.21.15.3:0/1594955857 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa5180d2d10 0x7fa5180cf530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:42.624 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.643+0000 7fa50ffff640 1 -- 172.21.15.3:0/1594955857 >> v1:172.21.15.190:6789/0 conn(0x7fa5180d11f0 legacy=0x7fa51815ff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:42.624 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.643+0000 7fa50ffff640 1 -- 172.21.15.3:0/1594955857 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa51815f340 con 0x7fa5180d1fd0 2024-05-09T06:36:42.625 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.644+0000 7fa50ffff640 1 --2- 172.21.15.3:0/1594955857 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa5180d1fd0 0x7fa5180d27d0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fa508009870 tx=0x7fa50802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee0675a21c13ad1e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:42.625 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.644+0000 7fa50f7fe640 1 -- 172.21.15.3:0/1594955857 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa508037070 con 0x7fa5180d1fd0 2024-05-09T06:36:42.625 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.644+0000 7fa50f7fe640 1 -- 172.21.15.3:0/1594955857 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa508032440 con 0x7fa5180d1fd0 2024-05-09T06:36:42.625 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.644+0000 7fa50f7fe640 1 -- 172.21.15.3:0/1594955857 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa508032810 con 0x7fa5180d1fd0 2024-05-09T06:36:42.626 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.645+0000 7fa51f02a640 1 -- 172.21.15.3:0/1594955857 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa5180d1fd0 msgr2=0x7fa5180d27d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:42.626 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.645+0000 7fa51f02a640 1 --2- 172.21.15.3:0/1594955857 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa5180d1fd0 0x7fa5180d27d0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fa508009870 tx=0x7fa50802a270 comp rx=0 tx=0).stop 2024-05-09T06:36:42.626 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.645+0000 7fa51f02a640 1 -- 172.21.15.3:0/1594955857 shutdown_connections 2024-05-09T06:36:42.626 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.645+0000 7fa51f02a640 1 --2- 172.21.15.3:0/1594955857 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa5180d2d10 0x7fa5180cf530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:42.626 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.645+0000 7fa51f02a640 1 --2- 172.21.15.3:0/1594955857 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa5180d1fd0 0x7fa5180d27d0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:42.626 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.645+0000 7fa51f02a640 1 -- 172.21.15.3:0/1594955857 >> 172.21.15.3:0/1594955857 conn(0x7fa51809d4d0 msgr2=0x7fa5180d41f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:42.626 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.645+0000 7fa51f02a640 1 -- 172.21.15.3:0/1594955857 shutdown_connections 2024-05-09T06:36:42.626 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.645+0000 7fa51f02a640 1 -- 172.21.15.3:0/1594955857 wait complete. 2024-05-09T06:36:42.627 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.646+0000 7fa51f02a640 1 Processor -- start 2024-05-09T06:36:42.627 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.647+0000 7fa51f02a640 1 -- start start 2024-05-09T06:36:42.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.647+0000 7fa51f02a640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa5180d11f0 0x7fa518155730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:42.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.647+0000 7fa51f02a640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa5180d1fd0 0x7fa5180ccc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:42.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.647+0000 7fa51f02a640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa5180d2d10 0x7fa5180b8f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:42.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.647+0000 7fa51f02a640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fa5180cfa70 con 0x7fa5180d11f0 2024-05-09T06:36:42.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.647+0000 7fa51f02a640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fa51815f900 con 0x7fa5180d2d10 2024-05-09T06:36:42.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.647+0000 7fa51f02a640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fa5180cfbf0 con 0x7fa5180d1fd0 2024-05-09T06:36:42.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.648+0000 7fa51cdc6640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa5180d11f0 0x7fa518155730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:42.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.648+0000 7fa51cdc6640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa5180d11f0 0x7fa518155730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46552/0 (socket says 172.21.15.3:46552) 2024-05-09T06:36:42.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.648+0000 7fa51cdc6640 1 -- 172.21.15.3:0/2036940592 learned_addr learned my addr 172.21.15.3:0/2036940592 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:36:42.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.648+0000 7fa50ffff640 1 --2- 172.21.15.3:0/2036940592 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa5180d1fd0 0x7fa5180ccc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:42.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.648+0000 7fa51d5c7640 1 --2- 172.21.15.3:0/2036940592 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa5180d2d10 0x7fa5180b8f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:42.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.648+0000 7fa51cdc6640 1 -- 172.21.15.3:0/2036940592 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa5180d1fd0 msgr2=0x7fa5180ccc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:42.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.648+0000 7fa51cdc6640 1 --2- 172.21.15.3:0/2036940592 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa5180d1fd0 0x7fa5180ccc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:42.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.648+0000 7fa51cdc6640 1 -- 172.21.15.3:0/2036940592 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa5180d2d10 msgr2=0x7fa5180b8f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:42.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.648+0000 7fa51cdc6640 1 --2- 172.21.15.3:0/2036940592 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa5180d2d10 0x7fa5180b8f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:42.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.648+0000 7fa51cdc6640 1 -- 172.21.15.3:0/2036940592 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5180cd6b0 con 0x7fa5180d11f0 2024-05-09T06:36:42.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.648+0000 7fa51cdc6640 1 --2- 172.21.15.3:0/2036940592 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa5180d11f0 0x7fa518155730 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fa51400c400 tx=0x7fa51400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:42.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.648+0000 7fa50d7fa640 1 -- 172.21.15.3:0/2036940592 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5140052d0 con 0x7fa5180d11f0 2024-05-09T06:36:42.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.648+0000 7fa50d7fa640 1 -- 172.21.15.3:0/2036940592 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa514005490 con 0x7fa5180d11f0 2024-05-09T06:36:42.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.648+0000 7fa51f02a640 1 -- 172.21.15.3:0/2036940592 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa51809e0e0 con 0x7fa5180d11f0 2024-05-09T06:36:42.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.648+0000 7fa50d7fa640 1 -- 172.21.15.3:0/2036940592 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa514007260 con 0x7fa5180d11f0 2024-05-09T06:36:42.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.648+0000 7fa51f02a640 1 -- 172.21.15.3:0/2036940592 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa51809e6a0 con 0x7fa5180d11f0 2024-05-09T06:36:42.631 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.649+0000 7fa51f02a640 1 -- 172.21.15.3:0/2036940592 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa518069550 con 0x7fa5180d11f0 2024-05-09T06:36:42.631 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.650+0000 7fa50d7fa640 1 -- 172.21.15.3:0/2036940592 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa514007400 con 0x7fa5180d11f0 2024-05-09T06:36:42.631 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.650+0000 7fa50d7fa640 1 --2- 172.21.15.3:0/2036940592 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fa50406be30 0x7fa50406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:42.631 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.651+0000 7fa50d7fa640 1 -- 172.21.15.3:0/2036940592 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fa51408a2f0 con 0x7fa5180d11f0 2024-05-09T06:36:42.631 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.651+0000 7fa50ffff640 1 --2- 172.21.15.3:0/2036940592 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fa50406be30 0x7fa50406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:42.631 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.651+0000 7fa50ffff640 1 --2- 172.21.15.3:0/2036940592 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fa50406be30 0x7fa50406e2f0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fa508034000 tx=0x7fa50802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:42.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.655+0000 7fa50d7fa640 1 -- 172.21.15.3:0/2036940592 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fa514058650 con 0x7fa5180d11f0 2024-05-09T06:36:42.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.857+0000 7fa51f02a640 1 -- 172.21.15.3:0/2036940592 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fa5180d25b0 con 0x7fa5180d11f0 2024-05-09T06:36:42.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.857+0000 7fa50d7fa640 1 -- 172.21.15.3:0/2036940592 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v20) ==== 80+0+0 (secure 0 0 0) 0x7fa51405c950 con 0x7fa5180d11f0 2024-05-09T06:36:42.837 INFO:teuthology.orchestra.run.smithi003.stderr:listed 0 entries 2024-05-09T06:36:42.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.859+0000 7fa51f02a640 1 -- 172.21.15.3:0/2036940592 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fa50406be30 msgr2=0x7fa50406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:42.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.859+0000 7fa51f02a640 1 --2- 172.21.15.3:0/2036940592 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fa50406be30 0x7fa50406e2f0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fa508034000 tx=0x7fa50802e000 comp rx=0 tx=0).stop 2024-05-09T06:36:42.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.859+0000 7fa51f02a640 1 -- 172.21.15.3:0/2036940592 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa5180d11f0 msgr2=0x7fa518155730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:42.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.859+0000 7fa51f02a640 1 --2- 172.21.15.3:0/2036940592 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa5180d11f0 0x7fa518155730 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fa51400c400 tx=0x7fa51400c8c0 comp rx=0 tx=0).stop 2024-05-09T06:36:42.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.859+0000 7fa51f02a640 1 -- 172.21.15.3:0/2036940592 shutdown_connections 2024-05-09T06:36:42.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.859+0000 7fa51f02a640 1 --2- 172.21.15.3:0/2036940592 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fa50406be30 0x7fa50406e2f0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:42.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.859+0000 7fa51f02a640 1 --2- 172.21.15.3:0/2036940592 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa5180d2d10 0x7fa5180b8f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:42.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.859+0000 7fa51f02a640 1 --2- 172.21.15.3:0/2036940592 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa5180d1fd0 0x7fa5180ccc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:42.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.859+0000 7fa51f02a640 1 --2- 172.21.15.3:0/2036940592 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa5180d11f0 0x7fa518155730 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:42.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.859+0000 7fa51f02a640 1 -- 172.21.15.3:0/2036940592 >> 172.21.15.3:0/2036940592 conn(0x7fa51809d4d0 msgr2=0x7fa5180b5640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:42.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.859+0000 7fa51f02a640 1 -- 172.21.15.3:0/2036940592 shutdown_connections 2024-05-09T06:36:42.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:42.859+0000 7fa51f02a640 1 -- 172.21.15.3:0/2036940592 wait complete. 2024-05-09T06:36:42.850 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.2... 2024-05-09T06:36:42.850 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:36:42.851 DEBUG:teuthology.orchestra.run.smithi003:> sudo umount /home/ubuntu/cephtest/mnt.2 -f 2024-05-09T06:36:42.878 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi003.front.sepia.ceph.com 2024-05-09T06:36:42.878 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:36:42.879 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-05-09T06:36:42.935 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-05-09T06:36:42.936 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T06:36:42.955 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-05-09T06:36:42.956 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T06:36:43.014 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-05-09T06:36:43.015 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T06:36:43.073 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T06:36:43.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:43.197+0000 7f5d086d6640 1 Processor -- start 2024-05-09T06:36:43.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:43.197+0000 7f5d086d6640 1 -- start start 2024-05-09T06:36:43.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:43.198+0000 7f5d086d6640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5d000ccad0 0x7f5d000b4250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:43.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:43.198+0000 7f5d086d6640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f5d000cde50 0x7f5d0015d160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:43.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:43.198+0000 7f5d086d6640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5d000b48e0 con 0x7f5d000cc6f0 2024-05-09T06:36:43.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:43.198+0000 7f5d086d6640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f5d000b4bc0 con 0x7f5d000cde50 2024-05-09T06:36:43.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:43.198+0000 7f5d086d6640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f5d000b4ea0 con 0x7f5d000ccad0 2024-05-09T06:36:43.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:43.198+0000 7f5d06c73640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f5d000cde50 0x7f5d0015d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:43.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:43.198+0000 7f5d06c73640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f5d000cde50 0x7f5d0015d160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46566/0 (socket says 172.21.15.3:46566) 2024-05-09T06:36:43.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:43.198+0000 7f5d06c73640 1 -- 172.21.15.3:0/2877855647 learned_addr learned my addr 172.21.15.3:0/2877855647 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:36:43.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:43.198+0000 7f5d05c71640 1 --2- 172.21.15.3:0/2877855647 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5d000ccad0 0x7f5d000b4250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:43.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:43.198+0000 7f5d06c73640 1 -- 172.21.15.3:0/2877855647 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5d000ccad0 msgr2=0x7f5d000b4250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:43.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:43.198+0000 7f5d06c73640 1 --2- 172.21.15.3:0/2877855647 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5d000ccad0 0x7f5d000b4250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:43.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:43.198+0000 7f5d06c73640 1 -- 172.21.15.3:0/2877855647 >> v1:172.21.15.190:6789/0 conn(0x7f5d000cc6f0 legacy=0x7f5d000cd740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:43.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:43.198+0000 7f5d06c73640 1 -- 172.21.15.3:0/2877855647 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d000b5180 con 0x7f5d000cde50 2024-05-09T06:36:43.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:43.199+0000 7f5d06c73640 1 --2- 172.21.15.3:0/2877855647 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5d000cde50 0x7f5d0015d160 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f5cf8009e50 tx=0x7f5cf802a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7462d81a3961ae3 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:43.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:43.199+0000 7f5d05470640 1 -- 172.21.15.3:0/2877855647 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cf8036070 con 0x7f5d000cde50 2024-05-09T06:36:43.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:43.199+0000 7f5d05470640 1 -- 172.21.15.3:0/2877855647 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5cf8031600 con 0x7f5d000cde50 2024-05-09T06:36:43.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:43.199+0000 7f5d05470640 1 -- 172.21.15.3:0/2877855647 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cf80319d0 con 0x7f5d000cde50 2024-05-09T06:36:43.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:43.200+0000 7f5d086d6640 1 -- 172.21.15.3:0/2877855647 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5d000cde50 msgr2=0x7f5d0015d160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:43.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:43.200+0000 7f5d086d6640 1 --2- 172.21.15.3:0/2877855647 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5d000cde50 0x7f5d0015d160 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f5cf8009e50 tx=0x7f5cf802a710 comp rx=0 tx=0).stop 2024-05-09T06:36:43.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:43.200+0000 7f5d086d6640 1 -- 172.21.15.3:0/2877855647 shutdown_connections 2024-05-09T06:36:43.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:43.200+0000 7f5d086d6640 1 --2- 172.21.15.3:0/2877855647 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5d000cde50 0x7f5d0015d160 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:43.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:43.200+0000 7f5d086d6640 1 --2- 172.21.15.3:0/2877855647 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5d000ccad0 0x7f5d000b4250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:43.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:43.200+0000 7f5d086d6640 1 -- 172.21.15.3:0/2877855647 >> 172.21.15.3:0/2877855647 conn(0x7f5d0006b430 msgr2=0x7f5d0006b810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:43.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:43.200+0000 7f5d086d6640 1 -- 172.21.15.3:0/2877855647 shutdown_connections 2024-05-09T06:36:43.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:43.200+0000 7f5d086d6640 1 -- 172.21.15.3:0/2877855647 wait complete. 2024-05-09T06:36:43.185 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:43.201+0000 7f5d086d6640 1 Processor -- start 2024-05-09T06:36:43.185 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:43.201+0000 7f5d086d6640 1 -- start start 2024-05-09T06:36:43.185 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:43.202+0000 7f5d086d6640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5d000cc6f0 0x7f5d001f22b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:43.186 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:43.202+0000 7f5d086d6640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5d000ccad0 0x7f5d00151190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:43.186 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:43.202+0000 7f5d086d6640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5d000cde50 0x7f5d0014f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:43.186 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:43.202+0000 7f5d086d6640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f5d0006dba0 con 0x7f5d000cde50 2024-05-09T06:36:43.186 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:43.202+0000 7f5d086d6640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f5d0006da20 con 0x7f5d000cc6f0 2024-05-09T06:36:43.186 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:43.202+0000 7f5d086d6640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f5d0006dd20 con 0x7f5d000ccad0 2024-05-09T06:36:43.186 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:43.202+0000 7f5d06c73640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5d000cde50 0x7f5d0014f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:43.186 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:43.202+0000 7f5d06c73640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5d000cde50 0x7f5d0014f180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46574/0 (socket says 172.21.15.3:46574) 2024-05-09T06:36:43.186 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:43.202+0000 7f5d06c73640 1 -- 172.21.15.3:0/2347185784 learned_addr learned my addr 172.21.15.3:0/2347185784 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:36:43.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:43.202+0000 7f5d06472640 1 --2- 172.21.15.3:0/2347185784 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5d000cc6f0 0x7f5d001f22b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:43.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:43.202+0000 7f5d05c71640 1 --2- 172.21.15.3:0/2347185784 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5d000ccad0 0x7f5d00151190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:43.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:43.202+0000 7f5d06c73640 1 -- 172.21.15.3:0/2347185784 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5d000ccad0 msgr2=0x7f5d00151190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:43.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:43.202+0000 7f5d06c73640 1 --2- 172.21.15.3:0/2347185784 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5d000ccad0 0x7f5d00151190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:43.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:43.202+0000 7f5d06c73640 1 -- 172.21.15.3:0/2347185784 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5d000cc6f0 msgr2=0x7f5d001f22b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:43.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:43.202+0000 7f5d06c73640 1 --2- 172.21.15.3:0/2347185784 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5d000cc6f0 0x7f5d001f22b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:43.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:43.202+0000 7f5d06c73640 1 -- 172.21.15.3:0/2347185784 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d001f2a00 con 0x7f5d000cde50 2024-05-09T06:36:43.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:43.202+0000 7f5d05c71640 1 --2- 172.21.15.3:0/2347185784 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5d000ccad0 0x7f5d00151190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:36:43.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:43.203+0000 7f5d06c73640 1 --2- 172.21.15.3:0/2347185784 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5d000cde50 0x7f5d0014f180 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f5cf8009f70 tx=0x7f5cf8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:43.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:43.203+0000 7f5cf6ffd640 1 -- 172.21.15.3:0/2347185784 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cf8041020 con 0x7f5d000cde50 2024-05-09T06:36:43.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:43.203+0000 7f5cf6ffd640 1 -- 172.21.15.3:0/2347185784 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5cf8007780 con 0x7f5d000cde50 2024-05-09T06:36:43.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:43.203+0000 7f5d086d6640 1 -- 172.21.15.3:0/2347185784 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d001f2ce0 con 0x7f5d000cde50 2024-05-09T06:36:43.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:43.203+0000 7f5cf6ffd640 1 -- 172.21.15.3:0/2347185784 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cf8036040 con 0x7f5d000cde50 2024-05-09T06:36:43.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:43.203+0000 7f5d086d6640 1 -- 172.21.15.3:0/2347185784 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d001f32a0 con 0x7f5d000cde50 2024-05-09T06:36:43.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:43.204+0000 7f5d086d6640 1 -- 172.21.15.3:0/2347185784 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d001e5bb0 con 0x7f5d000cde50 2024-05-09T06:36:43.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:43.204+0000 7f5cf6ffd640 1 -- 172.21.15.3:0/2347185784 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5cf8044020 con 0x7f5d000cde50 2024-05-09T06:36:43.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:43.205+0000 7f5cf6ffd640 1 --2- 172.21.15.3:0/2347185784 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f5ce406be30 0x7f5ce406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:43.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:43.205+0000 7f5cf6ffd640 1 -- 172.21.15.3:0/2347185784 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f5cf80ab9f0 con 0x7f5d000cde50 2024-05-09T06:36:43.189 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:43.205+0000 7f5d06472640 1 --2- 172.21.15.3:0/2347185784 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f5ce406be30 0x7f5ce406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:43.189 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:43.206+0000 7f5d06472640 1 --2- 172.21.15.3:0/2347185784 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f5ce406be30 0x7f5ce406e2f0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f5cfc001780 tx=0x7f5cfc009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:43.189 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:43.209+0000 7f5cf6ffd640 1 -- 172.21.15.3:0/2347185784 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f5cf804b030 con 0x7f5d000cde50 2024-05-09T06:36:43.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:43.393+0000 7f5d086d6640 1 -- 172.21.15.3:0/2347185784 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5d0004f660 con 0x7f5d000cde50 2024-05-09T06:36:43.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:43.394+0000 7f5cf6ffd640 1 -- 172.21.15.3:0/2347185784 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16521 (secure 0 0 0) 0x7f5cf807a430 con 0x7f5d000cde50 2024-05-09T06:36:43.374 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-09T06:36:43.374 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":20,"fsid":"16ed737d-2665-431a-8ac6-ef242be3975a","created":"2024-05-09T06:35:42.541368+0000","modified":"2024-05-09T06:36:08.856391+0000","last_up_change":"2024-05-09T06:35:53.778274+0000","last_in_change":"2024-05-09T06:35:44.780586+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T06:35:54.819999+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T06:36:04.931314+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T06:36:06.200433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fae3a8db-4826-4f3c-b33a-12c09bb5df6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":1897849827},{"type":"v1","addr":"172.21.15.3:6817","nonce":1897849827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":1897849827},{"type":"v1","addr":"172.21.15.3:6819","nonce":1897849827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":1897849827},{"type":"v1","addr":"172.21.15.3:6823","nonce":1897849827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":1897849827},{"type":"v1","addr":"172.21.15.3:6821","nonce":1897849827}]},"public_addr":"172.21.15.3:6817/1897849827","cluster_addr":"172.21.15.3:6819/1897849827","heartbeat_back_addr":"172.21.15.3:6823/1897849827","heartbeat_front_addr":"172.21.15.3:6821/1897849827","state":["exists","up"]},{"osd":1,"uuid":"646e882d-4ce4-4720-8c84-f71df563a35e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":926805498},{"type":"v1","addr":"172.21.15.3:6801","nonce":926805498}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":926805498},{"type":"v1","addr":"172.21.15.3:6803","nonce":926805498}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":926805498},{"type":"v1","addr":"172.21.15.3:6807","nonce":926805498}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":926805498},{"type":"v1","addr":"172.21.15.3:6805","nonce":926805498}]},"public_addr":"172.21.15.3:6801/926805498","cluster_addr":"172.21.15.3:6803/926805498","heartbeat_back_addr":"172.21.15.3:6807/926805498","heartbeat_front_addr":"172.21.15.3:6805/926805498","state":["exists","up"]},{"osd":2,"uuid":"1abbadb5-a570-4359-9643-acff3ede35a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":801295451},{"type":"v1","addr":"172.21.15.3:6809","nonce":801295451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":801295451},{"type":"v1","addr":"172.21.15.3:6811","nonce":801295451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":801295451},{"type":"v1","addr":"172.21.15.3:6815","nonce":801295451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":801295451},{"type":"v1","addr":"172.21.15.3:6813","nonce":801295451}]},"public_addr":"172.21.15.3:6809/801295451","cluster_addr":"172.21.15.3:6811/801295451","heartbeat_back_addr":"172.21.15.3:6815/801295451","heartbeat_front_addr":"172.21.15.3:6813/801295451","state":["exists","up"]},{"osd":3,"uuid":"62ac4dc1-2371-4dd2-acd3-e2dbb11aedf5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":1903389857},{"type":"v1","addr":"172.21.15.3:6825","nonce":1903389857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":1903389857},{"type":"v1","addr":"172.21.15.3:6827","nonce":1903389857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":1903389857},{"type":"v1","addr":"172.21.15.3:6831","nonce":1903389857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":1903389857},{"type":"v1","addr":"172.21.15.3:6829","nonce":1903389857}]},"public_addr":"172.21.15.3:6825/1903389857","cluster_addr":"172.21.15.3:6827/1903389857","heartbeat_back_addr":"172.21.15.3:6831/1903389857","heartbeat_front_addr":"172.21.15.3:6829/1903389857","state":["exists","up"]},{"osd":4,"uuid":"ae808398-8326-4d7c-9319-a2a0eafffb59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":2143624576},{"type":"v1","addr":"172.21.15.190:6801","nonce":2143624576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":2143624576},{"type":"v1","addr":"172.21.15.190:6803","nonce":2143624576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":2143624576},{"type":"v1","addr":"172.21.15.190:6807","nonce":2143624576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":2143624576},{"type":"v1","addr":"172.21.15.190:6805","nonce":2143624576}]},"public_addr":"172.21.15.190:6801/2143624576","cluster_addr":"172.21.15.190:6803/2143624576","heartbeat_back_addr":"172.21.15.190:6807/2143624576","heartbeat_front_addr":"172.21.15.190:6805/2143624576","state":["exists","up"]},{"osd":5,"uuid":"918adf50-c440-48ac-9a8d-4837562d7275","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":1766923126},{"type":"v1","addr":"172.21.15.190:6814","nonce":1766923126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":1766923126},{"type":"v1","addr":"172.21.15.190:6818","nonce":1766923126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":1766923126},{"type":"v1","addr":"172.21.15.190:6823","nonce":1766923126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":1766923126},{"type":"v1","addr":"172.21.15.190:6821","nonce":1766923126}]},"public_addr":"172.21.15.190:6814/1766923126","cluster_addr":"172.21.15.190:6818/1766923126","heartbeat_back_addr":"172.21.15.190:6823/1766923126","heartbeat_front_addr":"172.21.15.190:6821/1766923126","state":["exists","up"]},{"osd":6,"uuid":"ac2882bc-2a7c-463c-8f6c-76ff49828adc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":342414248},{"type":"v1","addr":"172.21.15.190:6825","nonce":342414248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":342414248},{"type":"v1","addr":"172.21.15.190:6827","nonce":342414248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":342414248},{"type":"v1","addr":"172.21.15.190:6831","nonce":342414248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":342414248},{"type":"v1","addr":"172.21.15.190:6829","nonce":342414248}]},"public_addr":"172.21.15.190:6825/342414248","cluster_addr":"172.21.15.190:6827/342414248","heartbeat_back_addr":"172.21.15.190:6831/342414248","heartbeat_front_addr":"172.21.15.190:6829/342414248","state":["exists","up"]},{"osd":7,"uuid":"dd4e767a-4730-4c70-8bd5-3f088140a5a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":537849891},{"type":"v1","addr":"172.21.15.190:6809","nonce":537849891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":537849891},{"type":"v1","addr":"172.21.15.190:6811","nonce":537849891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6817","nonce":537849891},{"type":"v1","addr":"172.21.15.190:6819","nonce":537849891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6813","nonce":537849891},{"type":"v1","addr":"172.21.15.190:6815","nonce":537849891}]},"public_addr":"172.21.15.190:6809/537849891","cluster_addr":"172.21.15.190:6811/537849891","heartbeat_back_addr":"172.21.15.190:6819/537849891","heartbeat_front_addr":"172.21.15.190:6815/537849891","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.889784+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.974002+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.973549+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.955396+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.928471+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.902418+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.958696+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.926474+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T06:36:43.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:43.396+0000 7f5d086d6640 1 -- 172.21.15.3:0/2347185784 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f5ce406be30 msgr2=0x7f5ce406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:43.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:43.396+0000 7f5d086d6640 1 --2- 172.21.15.3:0/2347185784 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f5ce406be30 0x7f5ce406e2f0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f5cfc001780 tx=0x7f5cfc009040 comp rx=0 tx=0).stop 2024-05-09T06:36:43.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:43.396+0000 7f5d086d6640 1 -- 172.21.15.3:0/2347185784 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5d000cde50 msgr2=0x7f5d0014f180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:43.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:43.396+0000 7f5d086d6640 1 --2- 172.21.15.3:0/2347185784 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5d000cde50 0x7f5d0014f180 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f5cf8009f70 tx=0x7f5cf8004a40 comp rx=0 tx=0).stop 2024-05-09T06:36:43.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:43.396+0000 7f5d086d6640 1 -- 172.21.15.3:0/2347185784 shutdown_connections 2024-05-09T06:36:43.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:43.396+0000 7f5d086d6640 1 --2- 172.21.15.3:0/2347185784 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f5ce406be30 0x7f5ce406e2f0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:43.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:43.396+0000 7f5d086d6640 1 --2- 172.21.15.3:0/2347185784 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5d000cde50 0x7f5d0014f180 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:43.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:43.396+0000 7f5d086d6640 1 --2- 172.21.15.3:0/2347185784 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5d000ccad0 0x7f5d00151190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:43.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:43.396+0000 7f5d086d6640 1 --2- 172.21.15.3:0/2347185784 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5d000cc6f0 0x7f5d001f22b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:43.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:43.396+0000 7f5d086d6640 1 -- 172.21.15.3:0/2347185784 >> 172.21.15.3:0/2347185784 conn(0x7f5d0006b430 msgr2=0x7f5d000cd740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:43.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:43.396+0000 7f5d086d6640 1 -- 172.21.15.3:0/2347185784 shutdown_connections 2024-05-09T06:36:43.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:43.396+0000 7f5d086d6640 1 -- 172.21.15.3:0/2347185784 wait complete. 2024-05-09T06:36:43.387 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-09T06:35:54.819999+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-09T06:36:04.931314+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-09T06:36:06.200433+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-09T06:36:43.387 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-09T06:36:43.495 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:43.515+0000 7f74540d4640 1 Processor -- start 2024-05-09T06:36:43.551 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:43.515+0000 7f74540d4640 1 -- start start 2024-05-09T06:36:43.557 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:43.516+0000 7f74540d4640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f744c144910 0x7f744c0cf8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:43.557 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:43.516+0000 7f74540d4640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f744c144cf0 con 0x7f744c072970 2024-05-09T06:36:43.557 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:43.516+0000 7f74540d4640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f744c0718d0 con 0x7f744c0b4130 2024-05-09T06:36:43.557 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:43.516+0000 7f74540d4640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f744c0724e0 con 0x7f744c144910 2024-05-09T06:36:43.557 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:43.516+0000 7f745166f640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f744c072970 0x7f744c071080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:51140/0 (socket says 172.21.15.3:51140) 2024-05-09T06:36:43.557 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:43.516+0000 7f745166f640 1 -- 172.21.15.3:0/2811457129 learned_addr learned my addr 172.21.15.3:0/2811457129 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-05-09T06:36:43.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:43.516+0000 7f7451e70640 1 --2- 172.21.15.3:0/2811457129 >> v2:172.21.15.190:3300/0 conn(0x7f744c144910 0x7f744c0cf8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:43.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:43.516+0000 7f7450e6e640 1 -- 172.21.15.3:0/2811457129 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3242367412 0 0) 0x7f744c144cf0 con 0x7f744c072970 2024-05-09T06:36:43.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:43.516+0000 7f7450e6e640 1 -- 172.21.15.3:0/2811457129 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7438003520 con 0x7f744c072970 2024-05-09T06:36:43.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:43.517+0000 7f7450e6e640 1 -- 172.21.15.3:0/2811457129 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1958939490 0 0) 0x7f7438003520 con 0x7f744c072970 2024-05-09T06:36:43.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:43.517+0000 7f7450e6e640 1 -- 172.21.15.3:0/2811457129 >> v2:172.21.15.190:3300/0 conn(0x7f744c144910 msgr2=0x7f744c0cf8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:43.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:43.517+0000 7f7450e6e640 1 --2- 172.21.15.3:0/2811457129 >> v2:172.21.15.190:3300/0 conn(0x7f744c144910 0x7f744c0cf8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:43.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:43.517+0000 7f7450e6e640 1 -- 172.21.15.3:0/2811457129 >> v1:172.21.15.190:6789/0 conn(0x7f744c0b4130 legacy=0x7f744c15b5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:43.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:43.517+0000 7f7450e6e640 1 -- 172.21.15.3:0/2811457129 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f744c071bb0 con 0x7f744c072970 2024-05-09T06:36:43.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:43.517+0000 7f7450e6e640 1 -- 172.21.15.3:0/2811457129 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 563312031 0 0) 0x7f743c001670 con 0x7f744c072970 2024-05-09T06:36:43.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:43.517+0000 7f7450e6e640 1 -- 172.21.15.3:0/2811457129 >> v1:172.21.15.3:6789/0 conn(0x7f744c072970 legacy=0x7f744c071080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:43.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:43.517+0000 7f7450e6e640 1 --2- 172.21.15.3:0/2811457129 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f74380039b0 0x7f7438003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:43.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:43.517+0000 7f7450e6e640 1 --2- 172.21.15.3:0/2811457129 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f74380043b0 0x7f7438004880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:43.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:43.517+0000 7f7450e6e640 1 --2- 172.21.15.3:0/2811457129 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7438005080 0x7f743800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:43.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:43.517+0000 7f7450e6e640 1 -- 172.21.15.3:0/2811457129 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f744c0724e0 con 0x7f74380043b0 2024-05-09T06:36:43.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:43.517+0000 7f7450e6e640 1 -- 172.21.15.3:0/2811457129 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f743800bfe0 con 0x7f74380039b0 2024-05-09T06:36:43.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:43.517+0000 7f7450e6e640 1 -- 172.21.15.3:0/2811457129 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f743800c1e0 con 0x7f7438005080 2024-05-09T06:36:43.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:43.517+0000 7f745166f640 1 --2- 172.21.15.3:0/2811457129 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f74380043b0 0x7f7438004880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:43.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:43.517+0000 7f7451e70640 1 --2- 172.21.15.3:0/2811457129 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f74380039b0 0x7f7438003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:43.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:43.517+0000 7f7452671640 1 --2- 172.21.15.3:0/2811457129 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7438005080 0x7f743800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:43.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:43.517+0000 7f745166f640 1 -- 172.21.15.3:0/2811457129 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7438005080 msgr2=0x7f743800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:43.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:43.517+0000 7f745166f640 1 --2- 172.21.15.3:0/2811457129 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7438005080 0x7f743800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:43.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:43.517+0000 7f745166f640 1 -- 172.21.15.3:0/2811457129 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f74380039b0 msgr2=0x7f7438003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:43.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:43.517+0000 7f745166f640 1 --2- 172.21.15.3:0/2811457129 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f74380039b0 0x7f7438003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:43.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:43.517+0000 7f745166f640 1 -- 172.21.15.3:0/2811457129 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7438003520 con 0x7f74380043b0 2024-05-09T06:36:43.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:43.517+0000 7f7452671640 1 -- 172.21.15.3:0/2811457129 reap_dead start 2024-05-09T06:36:43.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:43.518+0000 7f745166f640 1 --2- 172.21.15.3:0/2811457129 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f74380043b0 0x7f7438004880 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f743c00c240 tx=0x7f7438005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50e2e5927d11ad6a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:43.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:43.518+0000 7f7450e6e640 1 -- 172.21.15.3:0/2811457129 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f743c007d30 con 0x7f74380043b0 2024-05-09T06:36:43.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:43.518+0000 7f7450e6e640 1 -- 172.21.15.3:0/2811457129 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f743c0058d0 con 0x7f74380043b0 2024-05-09T06:36:43.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:43.518+0000 7f7450e6e640 1 -- 172.21.15.3:0/2811457129 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f743c005d20 con 0x7f74380043b0 2024-05-09T06:36:43.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:43.518+0000 7f74540d4640 1 -- 172.21.15.3:0/2811457129 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f74380043b0 msgr2=0x7f7438004880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:43.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:43.518+0000 7f74540d4640 1 --2- 172.21.15.3:0/2811457129 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f74380043b0 0x7f7438004880 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f743c00c240 tx=0x7f7438005830 comp rx=0 tx=0).stop 2024-05-09T06:36:43.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:43.519+0000 7f74540d4640 1 -- 172.21.15.3:0/2811457129 shutdown_connections 2024-05-09T06:36:43.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:43.519+0000 7f74540d4640 1 --2- 172.21.15.3:0/2811457129 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f74380043b0 0x7f7438004880 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:43.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:43.519+0000 7f74540d4640 1 -- 172.21.15.3:0/2811457129 >> 172.21.15.3:0/2811457129 conn(0x7f744c0b1c20 msgr2=0x7f744c0b2030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:43.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:43.519+0000 7f74540d4640 1 -- 172.21.15.3:0/2811457129 shutdown_connections 2024-05-09T06:36:43.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:43.519+0000 7f74540d4640 1 -- 172.21.15.3:0/2811457129 wait complete. 2024-05-09T06:36:43.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:43.520+0000 7f74540d4640 1 Processor -- start 2024-05-09T06:36:43.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:43.520+0000 7f74540d4640 1 -- start start 2024-05-09T06:36:43.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:43.521+0000 7f74540d4640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f74380043b0 0x7f744c13c060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:43.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:43.521+0000 7f74540d4640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f744c1f3fd0 0x7f744c0ae0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:43.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:43.521+0000 7f74540d4640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f744c0b3020 0x7f744c0b34a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:43.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:43.521+0000 7f74540d4640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f744c0cf8c0 con 0x7f744c1f3fd0 2024-05-09T06:36:43.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:43.521+0000 7f74540d4640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f744c072170 con 0x7f744c0b3020 2024-05-09T06:36:43.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:43.521+0000 7f74540d4640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f744c0cfa40 con 0x7f74380043b0 2024-05-09T06:36:43.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:43.521+0000 7f745166f640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f744c1f3fd0 0x7f744c0ae0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:43.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:43.521+0000 7f745166f640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f744c1f3fd0 0x7f744c0ae0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46594/0 (socket says 172.21.15.3:46594) 2024-05-09T06:36:43.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:43.521+0000 7f745166f640 1 -- 172.21.15.3:0/1087337212 learned_addr learned my addr 172.21.15.3:0/1087337212 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:36:43.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:43.521+0000 7f7451e70640 1 --2- 172.21.15.3:0/1087337212 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f74380043b0 0x7f744c13c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:43.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:43.521+0000 7f7452671640 1 --2- 172.21.15.3:0/1087337212 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f744c0b3020 0x7f744c0b34a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:43.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:43.521+0000 7f745166f640 1 -- 172.21.15.3:0/1087337212 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f74380043b0 msgr2=0x7f744c13c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:43.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:43.521+0000 7f745166f640 1 --2- 172.21.15.3:0/1087337212 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f74380043b0 0x7f744c13c060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:43.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:43.521+0000 7f745166f640 1 -- 172.21.15.3:0/1087337212 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f744c0b3020 msgr2=0x7f744c0b34a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:43.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:43.521+0000 7f745166f640 1 --2- 172.21.15.3:0/1087337212 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f744c0b3020 0x7f744c0b34a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:43.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:43.521+0000 7f745166f640 1 -- 172.21.15.3:0/1087337212 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f744c0ad940 con 0x7f744c1f3fd0 2024-05-09T06:36:43.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:43.521+0000 7f7452671640 1 --2- 172.21.15.3:0/1087337212 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f744c0b3020 0x7f744c0b34a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:36:43.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:43.521+0000 7f745166f640 1 --2- 172.21.15.3:0/1087337212 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f744c1f3fd0 0x7f744c0ae0d0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f743c001970 tx=0x7f744c0cfe00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:43.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:43.521+0000 7f74427fc640 1 -- 172.21.15.3:0/1087337212 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f743c041030 con 0x7f744c1f3fd0 2024-05-09T06:36:43.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:43.521+0000 7f74427fc640 1 -- 172.21.15.3:0/1087337212 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f743c004c10 con 0x7f744c1f3fd0 2024-05-09T06:36:43.562 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:43.521+0000 7f74540d4640 1 -- 172.21.15.3:0/1087337212 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f744c1f4870 con 0x7f744c1f3fd0 2024-05-09T06:36:43.562 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:43.521+0000 7f74427fc640 1 -- 172.21.15.3:0/1087337212 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f743c038040 con 0x7f744c1f3fd0 2024-05-09T06:36:43.562 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:43.521+0000 7f74540d4640 1 -- 172.21.15.3:0/1087337212 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f744c0adbb0 con 0x7f744c1f3fd0 2024-05-09T06:36:43.562 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:43.522+0000 7f74540d4640 1 -- 172.21.15.3:0/1087337212 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f744c069550 con 0x7f744c1f3fd0 2024-05-09T06:36:43.562 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:43.523+0000 7f74427fc640 1 -- 172.21.15.3:0/1087337212 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f743c044070 con 0x7f744c1f3fd0 2024-05-09T06:36:43.562 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:43.523+0000 7f74427fc640 1 --2- 172.21.15.3:0/1087337212 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f743006be60 0x7f743006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:43.562 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:43.523+0000 7f74427fc640 1 -- 172.21.15.3:0/1087337212 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f743c0aa790 con 0x7f744c1f3fd0 2024-05-09T06:36:43.562 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:43.524+0000 7f7451e70640 1 --2- 172.21.15.3:0/1087337212 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f743006be60 0x7f743006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:43.562 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:43.524+0000 7f7451e70640 1 --2- 172.21.15.3:0/1087337212 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f743006be60 0x7f743006e320 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f744c06a040 tx=0x7f7448008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:43.562 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:43.528+0000 7f74427fc640 1 -- 172.21.15.3:0/1087337212 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f743c04c020 con 0x7f744c1f3fd0 2024-05-09T06:36:43.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:43.709+0000 7f74540d4640 1 -- 172.21.15.3:0/1087337212 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f744c13aa40 con 0x7f744c1f3fd0 2024-05-09T06:36:43.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:43.709+0000 7f74427fc640 1 -- 172.21.15.3:0/1087337212 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7f743c0791d0 con 0x7f744c1f3fd0 2024-05-09T06:36:43.690 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 1 2024-05-09T06:36:43.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:43.711+0000 7f74540d4640 1 -- 172.21.15.3:0/1087337212 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f743006be60 msgr2=0x7f743006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:43.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:43.711+0000 7f74540d4640 1 --2- 172.21.15.3:0/1087337212 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f743006be60 0x7f743006e320 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f744c06a040 tx=0x7f7448008040 comp rx=0 tx=0).stop 2024-05-09T06:36:43.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:43.711+0000 7f74540d4640 1 -- 172.21.15.3:0/1087337212 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f744c1f3fd0 msgr2=0x7f744c0ae0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:43.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:43.711+0000 7f74540d4640 1 --2- 172.21.15.3:0/1087337212 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f744c1f3fd0 0x7f744c0ae0d0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f743c001970 tx=0x7f744c0cfe00 comp rx=0 tx=0).stop 2024-05-09T06:36:43.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:43.711+0000 7f74540d4640 1 -- 172.21.15.3:0/1087337212 shutdown_connections 2024-05-09T06:36:43.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:43.711+0000 7f74540d4640 1 --2- 172.21.15.3:0/1087337212 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f743006be60 0x7f743006e320 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:43.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:43.711+0000 7f74540d4640 1 --2- 172.21.15.3:0/1087337212 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f744c0b3020 0x7f744c0b34a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:43.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:43.711+0000 7f74540d4640 1 --2- 172.21.15.3:0/1087337212 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f744c1f3fd0 0x7f744c0ae0d0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:43.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:43.711+0000 7f74540d4640 1 --2- 172.21.15.3:0/1087337212 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f74380043b0 0x7f744c13c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:43.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:43.711+0000 7f74540d4640 1 -- 172.21.15.3:0/1087337212 >> 172.21.15.3:0/1087337212 conn(0x7f744c0b1c20 msgr2=0x7f744c0afb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:43.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:43.711+0000 7f74540d4640 1 -- 172.21.15.3:0/1087337212 shutdown_connections 2024-05-09T06:36:43.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:43.711+0000 7f74540d4640 1 -- 172.21.15.3:0/1087337212 wait complete. 2024-05-09T06:36:43.702 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-09T06:36:43.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:43.831+0000 7fc6504a5640 1 Processor -- start 2024-05-09T06:36:43.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:43.831+0000 7fc6504a5640 1 -- start start 2024-05-09T06:36:43.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:43.832+0000 7fc6504a5640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fc648161df0 0x7fc6480ae600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:43.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:43.832+0000 7fc6504a5640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fc6480b4520 0x7fc6480728f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:43.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:43.832+0000 7fc6504a5640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc648162280 con 0x7fc64815dff0 2024-05-09T06:36:43.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:43.832+0000 7fc6504a5640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fc648162560 con 0x7fc648161df0 2024-05-09T06:36:43.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:43.832+0000 7fc6504a5640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7fc648162840 con 0x7fc6480b4520 2024-05-09T06:36:43.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:43.832+0000 7fc64da40640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fc648161df0 0x7fc6480ae600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:43.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:43.832+0000 7fc64da40640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fc648161df0 0x7fc6480ae600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46598/0 (socket says 172.21.15.3:46598) 2024-05-09T06:36:43.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:43.832+0000 7fc64da40640 1 -- 172.21.15.3:0/550176493 learned_addr learned my addr 172.21.15.3:0/550176493 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:36:43.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:43.832+0000 7fc64ea42640 1 --2- 172.21.15.3:0/550176493 >> v2:172.21.15.190:3300/0 conn(0x7fc6480b4520 0x7fc6480728f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:43.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:43.832+0000 7fc64da40640 1 -- 172.21.15.3:0/550176493 >> v2:172.21.15.190:3300/0 conn(0x7fc6480b4520 msgr2=0x7fc6480728f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:43.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:43.832+0000 7fc64da40640 1 --2- 172.21.15.3:0/550176493 >> v2:172.21.15.190:3300/0 conn(0x7fc6480b4520 0x7fc6480728f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:43.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:43.832+0000 7fc64da40640 1 -- 172.21.15.3:0/550176493 >> v1:172.21.15.190:6789/0 conn(0x7fc64815dff0 legacy=0x7fc64806e600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:43.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:43.833+0000 7fc64da40640 1 -- 172.21.15.3:0/550176493 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc648162b20 con 0x7fc648161df0 2024-05-09T06:36:43.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:43.833+0000 7fc64da40640 1 --2- 172.21.15.3:0/550176493 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc648161df0 0x7fc6480ae600 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fc6480b0c50 tx=0x7fc63c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e58c838341edca76 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:43.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:43.833+0000 7fc64d23f640 1 -- 172.21.15.3:0/550176493 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc63c037070 con 0x7fc648161df0 2024-05-09T06:36:43.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:43.833+0000 7fc64d23f640 1 -- 172.21.15.3:0/550176493 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc63c032440 con 0x7fc648161df0 2024-05-09T06:36:43.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:43.833+0000 7fc64d23f640 1 -- 172.21.15.3:0/550176493 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc63c032810 con 0x7fc648161df0 2024-05-09T06:36:43.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:43.834+0000 7fc6504a5640 1 -- 172.21.15.3:0/550176493 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc648161df0 msgr2=0x7fc6480ae600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:43.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:43.834+0000 7fc6504a5640 1 --2- 172.21.15.3:0/550176493 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc648161df0 0x7fc6480ae600 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fc6480b0c50 tx=0x7fc63c02a1e0 comp rx=0 tx=0).stop 2024-05-09T06:36:43.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:43.834+0000 7fc6504a5640 1 -- 172.21.15.3:0/550176493 shutdown_connections 2024-05-09T06:36:43.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:43.834+0000 7fc6504a5640 1 --2- 172.21.15.3:0/550176493 >> v2:172.21.15.190:3300/0 conn(0x7fc6480b4520 0x7fc6480728f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:43.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:43.834+0000 7fc6504a5640 1 --2- 172.21.15.3:0/550176493 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc648161df0 0x7fc6480ae600 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:43.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:43.834+0000 7fc6504a5640 1 -- 172.21.15.3:0/550176493 >> 172.21.15.3:0/550176493 conn(0x7fc6480b6530 msgr2=0x7fc6480b6910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:43.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:43.834+0000 7fc6504a5640 1 -- 172.21.15.3:0/550176493 shutdown_connections 2024-05-09T06:36:43.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:43.834+0000 7fc6504a5640 1 -- 172.21.15.3:0/550176493 wait complete. 2024-05-09T06:36:43.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:43.835+0000 7fc6504a5640 1 Processor -- start 2024-05-09T06:36:43.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:43.835+0000 7fc6504a5640 1 -- start start 2024-05-09T06:36:43.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:43.836+0000 7fc6504a5640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc6480b4520 0x7fc64814ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:43.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:43.836+0000 7fc6504a5640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc64815dff0 0x7fc648154400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:43.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:43.836+0000 7fc6504a5640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc648154d40 0x7fc64815c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:43.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:43.836+0000 7fc6504a5640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fc64815e3d0 con 0x7fc648154d40 2024-05-09T06:36:43.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:43.836+0000 7fc6504a5640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fc64806ef60 con 0x7fc64815dff0 2024-05-09T06:36:43.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:43.836+0000 7fc6504a5640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fc64815e550 con 0x7fc6480b4520 2024-05-09T06:36:43.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:43.836+0000 7fc64ea42640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc648154d40 0x7fc64815c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:43.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:43.836+0000 7fc64ea42640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc648154d40 0x7fc64815c1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46612/0 (socket says 172.21.15.3:46612) 2024-05-09T06:36:43.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:43.836+0000 7fc64ea42640 1 -- 172.21.15.3:0/1765839444 learned_addr learned my addr 172.21.15.3:0/1765839444 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:36:43.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:43.836+0000 7fc64e241640 1 --2- 172.21.15.3:0/1765839444 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc6480b4520 0x7fc64814ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:43.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:43.836+0000 7fc64da40640 1 --2- 172.21.15.3:0/1765839444 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc64815dff0 0x7fc648154400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:43.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:43.836+0000 7fc64ea42640 1 -- 172.21.15.3:0/1765839444 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc6480b4520 msgr2=0x7fc64814ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:43.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:43.836+0000 7fc64ea42640 1 --2- 172.21.15.3:0/1765839444 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc6480b4520 0x7fc64814ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:43.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:43.836+0000 7fc64ea42640 1 -- 172.21.15.3:0/1765839444 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc64815dff0 msgr2=0x7fc648154400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:43.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:43.836+0000 7fc64ea42640 1 --2- 172.21.15.3:0/1765839444 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc64815dff0 0x7fc648154400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:43.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:43.836+0000 7fc64ea42640 1 -- 172.21.15.3:0/1765839444 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc648155790 con 0x7fc648154d40 2024-05-09T06:36:43.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:43.837+0000 7fc64ea42640 1 --2- 172.21.15.3:0/1765839444 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc648154d40 0x7fc64815c1a0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fc64000d5a0 tx=0x7fc64000da60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:43.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:43.837+0000 7fc646ffd640 1 -- 172.21.15.3:0/1765839444 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6400125f0 con 0x7fc648154d40 2024-05-09T06:36:43.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:43.837+0000 7fc646ffd640 1 -- 172.21.15.3:0/1765839444 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc640019070 con 0x7fc648154d40 2024-05-09T06:36:43.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:43.837+0000 7fc6504a5640 1 -- 172.21.15.3:0/1765839444 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc648153920 con 0x7fc648154d40 2024-05-09T06:36:43.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:43.837+0000 7fc646ffd640 1 -- 172.21.15.3:0/1765839444 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6400129f0 con 0x7fc648154d40 2024-05-09T06:36:43.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:43.837+0000 7fc6504a5640 1 -- 172.21.15.3:0/1765839444 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc648153ee0 con 0x7fc648154d40 2024-05-09T06:36:43.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:43.838+0000 7fc6504a5640 1 -- 172.21.15.3:0/1765839444 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc648069550 con 0x7fc648154d40 2024-05-09T06:36:43.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:43.838+0000 7fc646ffd640 1 -- 172.21.15.3:0/1765839444 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc640012b90 con 0x7fc648154d40 2024-05-09T06:36:43.821 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:43.839+0000 7fc646ffd640 1 --2- 172.21.15.3:0/1765839444 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fc63406be30 0x7fc63406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:43.821 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:43.839+0000 7fc646ffd640 1 -- 172.21.15.3:0/1765839444 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fc64008b220 con 0x7fc648154d40 2024-05-09T06:36:43.821 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:43.839+0000 7fc64e241640 1 --2- 172.21.15.3:0/1765839444 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fc63406be30 0x7fc63406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:43.821 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:43.840+0000 7fc64e241640 1 --2- 172.21.15.3:0/1765839444 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fc63406be30 0x7fc63406e2f0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fc638001d40 tx=0x7fc638009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:43.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:43.843+0000 7fc646ffd640 1 -- 172.21.15.3:0/1765839444 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fc640059be0 con 0x7fc648154d40 2024-05-09T06:36:44.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.026+0000 7fc6504a5640 1 -- 172.21.15.3:0/1765839444 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fc6480b5a70 con 0x7fc648154d40 2024-05-09T06:36:44.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.027+0000 7fc646ffd640 1 -- 172.21.15.3:0/1765839444 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7fc6400594e0 con 0x7fc648154d40 2024-05-09T06:36:44.007 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 32 2024-05-09T06:36:44.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.029+0000 7fc6504a5640 1 -- 172.21.15.3:0/1765839444 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fc63406be30 msgr2=0x7fc63406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:44.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.029+0000 7fc6504a5640 1 --2- 172.21.15.3:0/1765839444 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fc63406be30 0x7fc63406e2f0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fc638001d40 tx=0x7fc638009040 comp rx=0 tx=0).stop 2024-05-09T06:36:44.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.029+0000 7fc6504a5640 1 -- 172.21.15.3:0/1765839444 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc648154d40 msgr2=0x7fc64815c1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:44.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.029+0000 7fc6504a5640 1 --2- 172.21.15.3:0/1765839444 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc648154d40 0x7fc64815c1a0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fc64000d5a0 tx=0x7fc64000da60 comp rx=0 tx=0).stop 2024-05-09T06:36:44.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.029+0000 7fc6504a5640 1 -- 172.21.15.3:0/1765839444 shutdown_connections 2024-05-09T06:36:44.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.029+0000 7fc6504a5640 1 --2- 172.21.15.3:0/1765839444 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fc63406be30 0x7fc63406e2f0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:44.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.029+0000 7fc6504a5640 1 --2- 172.21.15.3:0/1765839444 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc648154d40 0x7fc64815c1a0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:44.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.029+0000 7fc6504a5640 1 --2- 172.21.15.3:0/1765839444 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc64815dff0 0x7fc648154400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:44.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.029+0000 7fc6504a5640 1 --2- 172.21.15.3:0/1765839444 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc6480b4520 0x7fc64814ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:44.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.029+0000 7fc6504a5640 1 -- 172.21.15.3:0/1765839444 >> 172.21.15.3:0/1765839444 conn(0x7fc6480b6530 msgr2=0x7fc6480727f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:44.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.029+0000 7fc6504a5640 1 -- 172.21.15.3:0/1765839444 shutdown_connections 2024-05-09T06:36:44.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.029+0000 7fc6504a5640 1 -- 172.21.15.3:0/1765839444 wait complete. 2024-05-09T06:36:44.020 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-09T06:36:44.127 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.147+0000 7f40a01fb640 1 Processor -- start 2024-05-09T06:36:44.127 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.147+0000 7f40a01fb640 1 -- start start 2024-05-09T06:36:44.128 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.148+0000 7f40a01fb640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f40980b4e30 0x7f409806e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:44.128 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.148+0000 7f40a01fb640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f409806e790 0x7f409815c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:44.128 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.148+0000 7f40a01fb640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f40980b40f0 con 0x7f409815c0f0 2024-05-09T06:36:44.128 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.148+0000 7f40a01fb640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f409806ec00 con 0x7f40980b4e30 2024-05-09T06:36:44.128 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.148+0000 7f40a01fb640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f40980b3050 con 0x7f409806e790 2024-05-09T06:36:44.128 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.148+0000 7f409d796640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f40980b4e30 0x7f409806e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:44.128 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.148+0000 7f409d796640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f40980b4e30 0x7f409806e250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46622/0 (socket says 172.21.15.3:46622) 2024-05-09T06:36:44.128 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.148+0000 7f409d796640 1 -- 172.21.15.3:0/1128579274 learned_addr learned my addr 172.21.15.3:0/1128579274 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:36:44.129 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.148+0000 7f409e798640 1 --2- 172.21.15.3:0/1128579274 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f409806e790 0x7f409815c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:44.129 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.148+0000 7f409d796640 1 -- 172.21.15.3:0/1128579274 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f409806e790 msgr2=0x7f409815c4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:44.129 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.148+0000 7f409d796640 1 --2- 172.21.15.3:0/1128579274 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f409806e790 0x7f409815c4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:44.129 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.148+0000 7f409d796640 1 -- 172.21.15.3:0/1128579274 >> v1:172.21.15.190:6789/0 conn(0x7f409815c0f0 legacy=0x7f4098160730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:44.129 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.148+0000 7f409d796640 1 -- 172.21.15.3:0/1128579274 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40980b3330 con 0x7f40980b4e30 2024-05-09T06:36:44.130 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.149+0000 7f409d796640 1 --2- 172.21.15.3:0/1128579274 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f40980b4e30 0x7f409806e250 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f408c009870 tx=0x7f408c02a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfea16b2434ab925 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:44.130 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.149+0000 7f409cf95640 1 -- 172.21.15.3:0/1128579274 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f408c037070 con 0x7f40980b4e30 2024-05-09T06:36:44.130 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.149+0000 7f409cf95640 1 -- 172.21.15.3:0/1128579274 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f408c032440 con 0x7f40980b4e30 2024-05-09T06:36:44.130 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.149+0000 7f409cf95640 1 -- 172.21.15.3:0/1128579274 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f408c032810 con 0x7f40980b4e30 2024-05-09T06:36:44.130 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.150+0000 7f40a01fb640 1 -- 172.21.15.3:0/1128579274 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f40980b4e30 msgr2=0x7f409806e250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:44.130 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.150+0000 7f40a01fb640 1 --2- 172.21.15.3:0/1128579274 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f40980b4e30 0x7f409806e250 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f408c009870 tx=0x7f408c02a1c0 comp rx=0 tx=0).stop 2024-05-09T06:36:44.130 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.150+0000 7f40a01fb640 1 -- 172.21.15.3:0/1128579274 shutdown_connections 2024-05-09T06:36:44.131 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.150+0000 7f40a01fb640 1 --2- 172.21.15.3:0/1128579274 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f409806e790 0x7f409815c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:44.131 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.150+0000 7f40a01fb640 1 --2- 172.21.15.3:0/1128579274 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f40980b4e30 0x7f409806e250 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:44.131 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.150+0000 7f40a01fb640 1 -- 172.21.15.3:0/1128579274 >> 172.21.15.3:0/1128579274 conn(0x7f40980b1c20 msgr2=0x7f40980b2030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:44.131 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.150+0000 7f40a01fb640 1 -- 172.21.15.3:0/1128579274 shutdown_connections 2024-05-09T06:36:44.131 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.150+0000 7f40a01fb640 1 -- 172.21.15.3:0/1128579274 wait complete. 2024-05-09T06:36:44.131 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.151+0000 7f40a01fb640 1 Processor -- start 2024-05-09T06:36:44.131 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.151+0000 7f40a01fb640 1 -- start start 2024-05-09T06:36:44.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.152+0000 7f40a01fb640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f409806e790 0x7f4098072a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:44.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.152+0000 7f40a01fb640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f40980b4e30 0x7f409815de40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:44.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.152+0000 7f40a01fb640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f409815c0f0 0x7f409815e380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:44.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.152+0000 7f40a01fb640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f40980b3bd0 con 0x7f409806e790 2024-05-09T06:36:44.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.152+0000 7f40a01fb640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f40980b38f0 con 0x7f409815c0f0 2024-05-09T06:36:44.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.152+0000 7f40a01fb640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f409815ca10 con 0x7f40980b4e30 2024-05-09T06:36:44.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.152+0000 7f409df97640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f409806e790 0x7f4098072a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:44.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.152+0000 7f409df97640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f409806e790 0x7f4098072a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46626/0 (socket says 172.21.15.3:46626) 2024-05-09T06:36:44.137 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.152+0000 7f409df97640 1 -- 172.21.15.3:0/280327156 learned_addr learned my addr 172.21.15.3:0/280327156 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:36:44.137 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.152+0000 7f409d796640 1 --2- 172.21.15.3:0/280327156 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f40980b4e30 0x7f409815de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:44.137 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.152+0000 7f409e798640 1 --2- 172.21.15.3:0/280327156 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f409815c0f0 0x7f409815e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:44.137 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.152+0000 7f409df97640 1 -- 172.21.15.3:0/280327156 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f40980b4e30 msgr2=0x7f409815de40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:44.137 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.152+0000 7f409df97640 1 --2- 172.21.15.3:0/280327156 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f40980b4e30 0x7f409815de40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:44.137 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.152+0000 7f409df97640 1 -- 172.21.15.3:0/280327156 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f409815c0f0 msgr2=0x7f409815e380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:44.137 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.152+0000 7f409df97640 1 --2- 172.21.15.3:0/280327156 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f409815c0f0 0x7f409815e380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:44.137 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.152+0000 7f409df97640 1 -- 172.21.15.3:0/280327156 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f409814e140 con 0x7f409806e790 2024-05-09T06:36:44.137 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.152+0000 7f409e798640 1 --2- 172.21.15.3:0/280327156 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f409815c0f0 0x7f409815e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:36:44.137 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.153+0000 7f409df97640 1 --2- 172.21.15.3:0/280327156 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f409806e790 0x7f4098072a90 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f409000c3c0 tx=0x7f409000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:44.137 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.153+0000 7f40967fc640 1 -- 172.21.15.3:0/280327156 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40900052d0 con 0x7f409806e790 2024-05-09T06:36:44.137 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.153+0000 7f40967fc640 1 -- 172.21.15.3:0/280327156 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4090005490 con 0x7f409806e790 2024-05-09T06:36:44.137 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.153+0000 7f40967fc640 1 -- 172.21.15.3:0/280327156 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4090007260 con 0x7f409806e790 2024-05-09T06:36:44.137 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.153+0000 7f40a01fb640 1 -- 172.21.15.3:0/280327156 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40981f8ab0 con 0x7f409806e790 2024-05-09T06:36:44.138 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.153+0000 7f40a01fb640 1 -- 172.21.15.3:0/280327156 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40981f9070 con 0x7f409806e790 2024-05-09T06:36:44.138 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.154+0000 7f40a01fb640 1 -- 172.21.15.3:0/280327156 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4098069550 con 0x7f409806e790 2024-05-09T06:36:44.138 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.154+0000 7f40967fc640 1 -- 172.21.15.3:0/280327156 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4090007400 con 0x7f409806e790 2024-05-09T06:36:44.138 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.155+0000 7f40967fc640 1 --2- 172.21.15.3:0/280327156 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f408406be30 0x7f408406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:44.138 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.155+0000 7f40967fc640 1 -- 172.21.15.3:0/280327156 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f409008a290 con 0x7f409806e790 2024-05-09T06:36:44.138 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.155+0000 7f409d796640 1 --2- 172.21.15.3:0/280327156 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f408406be30 0x7f408406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:44.138 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.155+0000 7f409d796640 1 --2- 172.21.15.3:0/280327156 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f408406be30 0x7f408406e2f0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f408c034000 tx=0x7f408c02e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:44.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.159+0000 7f40967fc640 1 -- 172.21.15.3:0/280327156 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f40900585f0 con 0x7f409806e790 2024-05-09T06:36:44.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.340+0000 7f40a01fb640 1 -- 172.21.15.3:0/280327156 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f40980b18f0 con 0x7f409806e790 2024-05-09T06:36:44.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.340+0000 7f40967fc640 1 -- 172.21.15.3:0/280327156 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7f409005c8f0 con 0x7f409806e790 2024-05-09T06:36:44.321 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 64 2024-05-09T06:36:44.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.342+0000 7f40a01fb640 1 -- 172.21.15.3:0/280327156 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f408406be30 msgr2=0x7f408406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:44.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.342+0000 7f40a01fb640 1 --2- 172.21.15.3:0/280327156 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f408406be30 0x7f408406e2f0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f408c034000 tx=0x7f408c02e000 comp rx=0 tx=0).stop 2024-05-09T06:36:44.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.342+0000 7f40a01fb640 1 -- 172.21.15.3:0/280327156 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f409806e790 msgr2=0x7f4098072a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:44.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.342+0000 7f40a01fb640 1 --2- 172.21.15.3:0/280327156 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f409806e790 0x7f4098072a90 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f409000c3c0 tx=0x7f409000c880 comp rx=0 tx=0).stop 2024-05-09T06:36:44.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.342+0000 7f40a01fb640 1 -- 172.21.15.3:0/280327156 shutdown_connections 2024-05-09T06:36:44.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.342+0000 7f40a01fb640 1 --2- 172.21.15.3:0/280327156 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f408406be30 0x7f408406e2f0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:44.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.342+0000 7f40a01fb640 1 --2- 172.21.15.3:0/280327156 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f409815c0f0 0x7f409815e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:44.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.342+0000 7f40a01fb640 1 --2- 172.21.15.3:0/280327156 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f40980b4e30 0x7f409815de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:44.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.342+0000 7f40a01fb640 1 --2- 172.21.15.3:0/280327156 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f409806e790 0x7f4098072a90 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:44.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.342+0000 7f40a01fb640 1 -- 172.21.15.3:0/280327156 >> 172.21.15.3:0/280327156 conn(0x7f40980b1c20 msgr2=0x7f409815a780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:44.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.342+0000 7f40a01fb640 1 -- 172.21.15.3:0/280327156 shutdown_connections 2024-05-09T06:36:44.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.342+0000 7f40a01fb640 1 -- 172.21.15.3:0/280327156 wait complete. 2024-05-09T06:36:44.334 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-09T06:36:44.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.462+0000 7f12ee722640 1 Processor -- start 2024-05-09T06:36:44.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.462+0000 7f12ee722640 1 -- start start 2024-05-09T06:36:44.451 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.463+0000 7f12ee722640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f12e80b4e30 0x7f12e806e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:44.451 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.463+0000 7f12ee722640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f12e80b40f0 con 0x7f12e815c0f0 2024-05-09T06:36:44.451 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.463+0000 7f12ee722640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f12e80b3050 con 0x7f12e806e790 2024-05-09T06:36:44.451 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.463+0000 7f12ee722640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f12e806ec00 con 0x7f12e80b4e30 2024-05-09T06:36:44.451 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.463+0000 7f12e7fff640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f12e815c0f0 0x7f12e8160730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:51152/0 (socket says 172.21.15.3:51152) 2024-05-09T06:36:44.452 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.463+0000 7f12e7fff640 1 -- 172.21.15.3:0/2251351115 learned_addr learned my addr 172.21.15.3:0/2251351115 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-05-09T06:36:44.452 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.463+0000 7f12e6ffd640 1 -- 172.21.15.3:0/2251351115 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3462809473 0 0) 0x7f12e80b40f0 con 0x7f12e815c0f0 2024-05-09T06:36:44.452 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.463+0000 7f12e77fe640 1 --2- 172.21.15.3:0/2251351115 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f12e80b4e30 0x7f12e806e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:44.452 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.463+0000 7f12e6ffd640 1 -- 172.21.15.3:0/2251351115 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f12d0003520 con 0x7f12e815c0f0 2024-05-09T06:36:44.452 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.463+0000 7f12e6ffd640 1 -- 172.21.15.3:0/2251351115 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 976569114 0 0) 0x7f12d0003520 con 0x7f12e815c0f0 2024-05-09T06:36:44.452 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.463+0000 7f12e6ffd640 1 -- 172.21.15.3:0/2251351115 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f12e80b4e30 msgr2=0x7f12e806e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:44.453 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.463+0000 7f12e6ffd640 1 --2- 172.21.15.3:0/2251351115 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f12e80b4e30 0x7f12e806e250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:44.453 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.463+0000 7f12e6ffd640 1 -- 172.21.15.3:0/2251351115 >> v1:172.21.15.190:6789/0 conn(0x7f12e806e790 legacy=0x7f12e815c4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:44.453 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.463+0000 7f12e6ffd640 1 -- 172.21.15.3:0/2251351115 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12e80b3330 con 0x7f12e815c0f0 2024-05-09T06:36:44.453 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.463+0000 7f12e6ffd640 1 -- 172.21.15.3:0/2251351115 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 563312031 0 0) 0x7f12d8001670 con 0x7f12e815c0f0 2024-05-09T06:36:44.453 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.463+0000 7f12e6ffd640 1 -- 172.21.15.3:0/2251351115 >> v1:172.21.15.3:6789/0 conn(0x7f12e815c0f0 legacy=0x7f12e8160730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:44.453 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.463+0000 7f12e6ffd640 1 --2- 172.21.15.3:0/2251351115 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f12d00039e0 0x7f12d0003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:44.454 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.463+0000 7f12e6ffd640 1 --2- 172.21.15.3:0/2251351115 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f12d0004800 0x7f12d0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:44.454 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.463+0000 7f12e6ffd640 1 --2- 172.21.15.3:0/2251351115 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f12e80b4e30 0x7f12d0005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:44.454 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.463+0000 7f12e6ffd640 1 -- 172.21.15.3:0/2251351115 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f12e806ec00 con 0x7f12d00039e0 2024-05-09T06:36:44.454 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.463+0000 7f12e6ffd640 1 -- 172.21.15.3:0/2251351115 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f12d000baf0 con 0x7f12d0004800 2024-05-09T06:36:44.454 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.463+0000 7f12e6ffd640 1 -- 172.21.15.3:0/2251351115 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f12d000bcf0 con 0x7f12e80b4e30 2024-05-09T06:36:44.454 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.463+0000 7f12e7fff640 1 --2- 172.21.15.3:0/2251351115 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f12d00039e0 0x7f12d0003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:44.455 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.464+0000 7f12e77fe640 1 --2- 172.21.15.3:0/2251351115 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f12d0004800 0x7f12d0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:44.455 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.464+0000 7f12eccbf640 1 --2- 172.21.15.3:0/2251351115 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f12e80b4e30 0x7f12d0005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:44.455 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.464+0000 7f12e7fff640 1 -- 172.21.15.3:0/2251351115 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f12e80b4e30 msgr2=0x7f12d0005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:44.455 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.464+0000 7f12e7fff640 1 --2- 172.21.15.3:0/2251351115 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f12e80b4e30 0x7f12d0005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:44.455 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.464+0000 7f12e7fff640 1 -- 172.21.15.3:0/2251351115 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f12d0004800 msgr2=0x7f12d0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:44.455 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.464+0000 7f12e7fff640 1 --2- 172.21.15.3:0/2251351115 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f12d0004800 0x7f12d0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:44.456 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.464+0000 7f12e7fff640 1 -- 172.21.15.3:0/2251351115 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12d0003520 con 0x7f12d00039e0 2024-05-09T06:36:44.456 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.464+0000 7f12e7fff640 1 --2- 172.21.15.3:0/2251351115 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f12d00039e0 0x7f12d0003e00 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f12d800c240 tx=0x7f12d0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13735c16277835f6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:44.456 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.465+0000 7f12e6ffd640 1 -- 172.21.15.3:0/2251351115 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12d8007d10 con 0x7f12d00039e0 2024-05-09T06:36:44.456 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.465+0000 7f12e6ffd640 1 -- 172.21.15.3:0/2251351115 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f12d8005ad0 con 0x7f12d00039e0 2024-05-09T06:36:44.456 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.465+0000 7f12e6ffd640 1 -- 172.21.15.3:0/2251351115 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12d8007070 con 0x7f12d00039e0 2024-05-09T06:36:44.456 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.465+0000 7f12ee722640 1 -- 172.21.15.3:0/2251351115 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f12d00039e0 msgr2=0x7f12d0003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:44.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.465+0000 7f12ee722640 1 --2- 172.21.15.3:0/2251351115 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f12d00039e0 0x7f12d0003e00 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f12d800c240 tx=0x7f12d0005830 comp rx=0 tx=0).stop 2024-05-09T06:36:44.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.465+0000 7f12eccbf640 1 -- 172.21.15.3:0/2251351115 reap_dead start 2024-05-09T06:36:44.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.465+0000 7f12ee722640 1 -- 172.21.15.3:0/2251351115 shutdown_connections 2024-05-09T06:36:44.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.465+0000 7f12ee722640 1 -- 172.21.15.3:0/2251351115 >> 172.21.15.3:0/2251351115 conn(0x7f12e80b1c20 msgr2=0x7f12e80b2030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:44.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.465+0000 7f12ee722640 1 -- 172.21.15.3:0/2251351115 shutdown_connections 2024-05-09T06:36:44.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.465+0000 7f12ee722640 1 -- 172.21.15.3:0/2251351115 wait complete. 2024-05-09T06:36:44.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.466+0000 7f12ee722640 1 Processor -- start 2024-05-09T06:36:44.458 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.467+0000 7f12ee722640 1 -- start start 2024-05-09T06:36:44.458 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.467+0000 7f12ee722640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f12d00039e0 0x7f12e81479d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:44.458 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.467+0000 7f12ee722640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f12e814b2d0 0x7f12e815e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:44.458 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.467+0000 7f12ee722640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f12e80710c0 0x7f12e815ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:44.458 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.467+0000 7f12ee722640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f12e80b3bd0 con 0x7f12e80710c0 2024-05-09T06:36:44.458 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.467+0000 7f12ee722640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f12e80b38f0 con 0x7f12e814b2d0 2024-05-09T06:36:44.458 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.467+0000 7f12ee722640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f12e806e250 con 0x7f12d00039e0 2024-05-09T06:36:44.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.467+0000 7f12eccbf640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f12e80710c0 0x7f12e815ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:44.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.467+0000 7f12eccbf640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f12e80710c0 0x7f12e815ebe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46640/0 (socket says 172.21.15.3:46640) 2024-05-09T06:36:44.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.467+0000 7f12eccbf640 1 -- 172.21.15.3:0/2769486118 learned_addr learned my addr 172.21.15.3:0/2769486118 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:36:44.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.467+0000 7f12e7fff640 1 --2- 172.21.15.3:0/2769486118 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f12d00039e0 0x7f12e81479d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:44.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.467+0000 7f12e77fe640 1 --2- 172.21.15.3:0/2769486118 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f12e814b2d0 0x7f12e815e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:44.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.468+0000 7f12eccbf640 1 -- 172.21.15.3:0/2769486118 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f12d00039e0 msgr2=0x7f12e81479d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:44.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.468+0000 7f12eccbf640 1 --2- 172.21.15.3:0/2769486118 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f12d00039e0 0x7f12e81479d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:44.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.468+0000 7f12eccbf640 1 -- 172.21.15.3:0/2769486118 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f12e814b2d0 msgr2=0x7f12e815e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:44.460 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.468+0000 7f12eccbf640 1 --2- 172.21.15.3:0/2769486118 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f12e814b2d0 0x7f12e815e6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:44.460 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.468+0000 7f12eccbf640 1 -- 172.21.15.3:0/2769486118 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12e81f88b0 con 0x7f12e80710c0 2024-05-09T06:36:44.460 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.468+0000 7f12e7fff640 1 --2- 172.21.15.3:0/2769486118 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f12d00039e0 0x7f12e81479d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:36:44.460 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.468+0000 7f12e77fe640 1 --2- 172.21.15.3:0/2769486118 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f12e814b2d0 0x7f12e815e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:36:44.460 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.468+0000 7f12eccbf640 1 --2- 172.21.15.3:0/2769486118 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f12e80710c0 0x7f12e815ebe0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f12e80af6f0 tx=0x7f12e000ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:44.460 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.468+0000 7f12e4ff9640 1 -- 172.21.15.3:0/2769486118 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12e0003970 con 0x7f12e80710c0 2024-05-09T06:36:44.460 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.468+0000 7f12e4ff9640 1 -- 172.21.15.3:0/2769486118 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f12e0018070 con 0x7f12e80710c0 2024-05-09T06:36:44.460 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.468+0000 7f12ee722640 1 -- 172.21.15.3:0/2769486118 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12e81f7d30 con 0x7f12e80710c0 2024-05-09T06:36:44.460 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.468+0000 7f12e4ff9640 1 -- 172.21.15.3:0/2769486118 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12e0003d50 con 0x7f12e80710c0 2024-05-09T06:36:44.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.468+0000 7f12ee722640 1 -- 172.21.15.3:0/2769486118 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12e81f82f0 con 0x7f12e80710c0 2024-05-09T06:36:44.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.469+0000 7f12ee722640 1 -- 172.21.15.3:0/2769486118 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12e8069550 con 0x7f12e80710c0 2024-05-09T06:36:44.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.470+0000 7f12e4ff9640 1 -- 172.21.15.3:0/2769486118 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f12e00050d0 con 0x7f12e80710c0 2024-05-09T06:36:44.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.470+0000 7f12e4ff9640 1 --2- 172.21.15.3:0/2769486118 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f12d406be60 0x7f12d406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:44.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.470+0000 7f12e4ff9640 1 -- 172.21.15.3:0/2769486118 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f12e008ac50 con 0x7f12e80710c0 2024-05-09T06:36:44.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.470+0000 7f12e7fff640 1 --2- 172.21.15.3:0/2769486118 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f12d406be60 0x7f12d406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:44.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.471+0000 7f12e7fff640 1 --2- 172.21.15.3:0/2769486118 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f12d406be60 0x7f12d406e320 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f12d8033000 tx=0x7f12d802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:44.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.475+0000 7f12e4ff9640 1 -- 172.21.15.3:0/2769486118 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f12e0059560 con 0x7f12e80710c0 2024-05-09T06:36:44.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.675+0000 7f12ee722640 1 -- 172.21.15.3:0/2769486118 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f12e806efe0 con 0x7f12e80710c0 2024-05-09T06:36:44.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.675+0000 7f12e4ff9640 1 -- 172.21.15.3:0/2769486118 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v20) ==== 80+0+0 (secure 0 0 0) 0x7f12e0058e60 con 0x7f12e80710c0 2024-05-09T06:36:44.655 INFO:teuthology.orchestra.run.smithi003.stderr:listed 0 entries 2024-05-09T06:36:44.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.677+0000 7f12ee722640 1 -- 172.21.15.3:0/2769486118 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f12d406be60 msgr2=0x7f12d406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:44.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.677+0000 7f12ee722640 1 --2- 172.21.15.3:0/2769486118 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f12d406be60 0x7f12d406e320 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f12d8033000 tx=0x7f12d802e000 comp rx=0 tx=0).stop 2024-05-09T06:36:44.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.677+0000 7f12ee722640 1 -- 172.21.15.3:0/2769486118 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f12e80710c0 msgr2=0x7f12e815ebe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:44.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.677+0000 7f12ee722640 1 --2- 172.21.15.3:0/2769486118 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f12e80710c0 0x7f12e815ebe0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f12e80af6f0 tx=0x7f12e000ec10 comp rx=0 tx=0).stop 2024-05-09T06:36:44.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.677+0000 7f12ee722640 1 -- 172.21.15.3:0/2769486118 shutdown_connections 2024-05-09T06:36:44.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.677+0000 7f12ee722640 1 --2- 172.21.15.3:0/2769486118 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f12d406be60 0x7f12d406e320 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:44.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.677+0000 7f12ee722640 1 --2- 172.21.15.3:0/2769486118 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f12e80710c0 0x7f12e815ebe0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:44.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.677+0000 7f12ee722640 1 --2- 172.21.15.3:0/2769486118 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f12e814b2d0 0x7f12e815e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:44.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.677+0000 7f12ee722640 1 --2- 172.21.15.3:0/2769486118 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f12d00039e0 0x7f12e81479d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:44.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.677+0000 7f12ee722640 1 -- 172.21.15.3:0/2769486118 >> 172.21.15.3:0/2769486118 conn(0x7f12e80b1c20 msgr2=0x7f12e815aef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:44.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.677+0000 7f12ee722640 1 -- 172.21.15.3:0/2769486118 shutdown_connections 2024-05-09T06:36:44.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.677+0000 7f12ee722640 1 -- 172.21.15.3:0/2769486118 wait complete. 2024-05-09T06:36:44.668 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.3... 2024-05-09T06:36:44.668 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:36:44.668 DEBUG:teuthology.orchestra.run.smithi190:> sudo umount /home/ubuntu/cephtest/mnt.3 -f 2024-05-09T06:36:44.702 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi190.front.sepia.ceph.com 2024-05-09T06:36:44.702 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:36:44.703 DEBUG:teuthology.orchestra.run.smithi190:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-05-09T06:36:44.762 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-05-09T06:36:44.762 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T06:36:44.866 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.887+0000 7f57b01a5640 1 Processor -- start 2024-05-09T06:36:44.867 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.887+0000 7f57b01a5640 1 -- start start 2024-05-09T06:36:44.867 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.888+0000 7f57b01a5640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f57a815e100 0x7f57a80ae2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:44.867 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.888+0000 7f57b01a5640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f57a806d8f0 0x7f57a80ae7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:44.868 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.888+0000 7f57b01a5640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f57a806cab0 con 0x7f57a806bcf0 2024-05-09T06:36:44.868 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.888+0000 7f57b01a5640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f57a806c890 con 0x7f57a815e100 2024-05-09T06:36:44.868 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.888+0000 7f57b01a5640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f57a80b3ed0 con 0x7f57a806d8f0 2024-05-09T06:36:44.868 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.888+0000 7f57adf41640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f57a815e100 0x7f57a80ae2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:44.869 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.888+0000 7f57ad740640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f57a806d8f0 0x7f57a80ae7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:44.869 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.888+0000 7f57ae742640 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7f57a806bcf0 0x7f57a806c180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.3:58246/0 (socket says 172.21.15.3:58246) 2024-05-09T06:36:44.869 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.888+0000 7f57ae742640 1 -- 172.21.15.3:0/2208313683 learned_addr learned my addr 172.21.15.3:0/2208313683 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-05-09T06:36:44.869 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.889+0000 7f57adf41640 1 -- 172.21.15.3:0/2208313683 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f57a806d8f0 msgr2=0x7f57a80ae7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:44.869 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.889+0000 7f57adf41640 1 --2- 172.21.15.3:0/2208313683 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f57a806d8f0 0x7f57a80ae7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:44.869 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.889+0000 7f57adf41640 1 -- 172.21.15.3:0/2208313683 >> v1:172.21.15.190:6789/0 conn(0x7f57a806bcf0 legacy=0x7f57a806c180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:44.869 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.889+0000 7f57adf41640 1 -- 172.21.15.3:0/2208313683 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57a80b4120 con 0x7f57a815e100 2024-05-09T06:36:44.869 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.889+0000 7f57acf3f640 1 -- 172.21.15.3:0/2208313683 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4136136085 0 0) 0x7f57a806cab0 con 0x7f57a806bcf0 2024-05-09T06:36:44.869 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.889+0000 7f57ad740640 1 --2- 172.21.15.3:0/2208313683 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f57a806d8f0 0x7f57a80ae7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T06:36:44.870 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.890+0000 7f57adf41640 1 --2- 172.21.15.3:0/2208313683 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f57a815e100 0x7f57a80ae2b0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f57a40097a0 tx=0x7f57a402a100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=67eee1d5489f3b2 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:44.871 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.890+0000 7f57acf3f640 1 -- 172.21.15.3:0/2208313683 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57a4037070 con 0x7f57a815e100 2024-05-09T06:36:44.871 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.890+0000 7f57acf3f640 1 -- 172.21.15.3:0/2208313683 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57a402ae50 con 0x7f57a815e100 2024-05-09T06:36:44.871 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.890+0000 7f57acf3f640 1 -- 172.21.15.3:0/2208313683 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57a4032660 con 0x7f57a815e100 2024-05-09T06:36:44.871 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.890+0000 7f57b01a5640 1 -- 172.21.15.3:0/2208313683 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f57a815e100 msgr2=0x7f57a80ae2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:44.871 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.890+0000 7f57b01a5640 1 --2- 172.21.15.3:0/2208313683 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f57a815e100 0x7f57a80ae2b0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f57a40097a0 tx=0x7f57a402a100 comp rx=0 tx=0).stop 2024-05-09T06:36:44.871 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.890+0000 7f57b01a5640 1 -- 172.21.15.3:0/2208313683 shutdown_connections 2024-05-09T06:36:44.871 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.890+0000 7f57b01a5640 1 --2- 172.21.15.3:0/2208313683 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f57a806d8f0 0x7f57a80ae7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:44.871 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.890+0000 7f57b01a5640 1 --2- 172.21.15.3:0/2208313683 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f57a815e100 0x7f57a80ae2b0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:44.871 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.890+0000 7f57b01a5640 1 -- 172.21.15.3:0/2208313683 >> 172.21.15.3:0/2208313683 conn(0x7f57a80d05d0 msgr2=0x7f57a80d09e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:44.871 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.890+0000 7f57b01a5640 1 -- 172.21.15.3:0/2208313683 shutdown_connections 2024-05-09T06:36:44.871 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.890+0000 7f57b01a5640 1 -- 172.21.15.3:0/2208313683 wait complete. 2024-05-09T06:36:44.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.892+0000 7f57b01a5640 1 Processor -- start 2024-05-09T06:36:44.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.892+0000 7f57b01a5640 1 -- start start 2024-05-09T06:36:44.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.892+0000 7f57b01a5640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f57a806bcf0 0x7f57a814ec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:44.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.892+0000 7f57b01a5640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f57a806d8f0 0x7f57a815e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:44.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.892+0000 7f57b01a5640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f57a815e100 0x7f57a815ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:44.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.892+0000 7f57b01a5640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f57a815e4e0 con 0x7f57a815e100 2024-05-09T06:36:44.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.892+0000 7f57b01a5640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f57a80b4770 con 0x7f57a806d8f0 2024-05-09T06:36:44.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.892+0000 7f57b01a5640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f57a815e660 con 0x7f57a806bcf0 2024-05-09T06:36:44.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.892+0000 7f57ae742640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f57a815e100 0x7f57a815ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:44.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.892+0000 7f57ae742640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f57a815e100 0x7f57a815ed20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46644/0 (socket says 172.21.15.3:46644) 2024-05-09T06:36:44.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.893+0000 7f57ae742640 1 -- 172.21.15.3:0/1324644454 learned_addr learned my addr 172.21.15.3:0/1324644454 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:36:44.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.893+0000 7f57adf41640 1 --2- 172.21.15.3:0/1324644454 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f57a806bcf0 0x7f57a814ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:44.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.893+0000 7f57ad740640 1 --2- 172.21.15.3:0/1324644454 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f57a806d8f0 0x7f57a815e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:44.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.893+0000 7f57ae742640 1 -- 172.21.15.3:0/1324644454 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f57a806bcf0 msgr2=0x7f57a814ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:44.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.893+0000 7f57ae742640 1 --2- 172.21.15.3:0/1324644454 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f57a806bcf0 0x7f57a814ec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:44.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.893+0000 7f57ae742640 1 -- 172.21.15.3:0/1324644454 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f57a806d8f0 msgr2=0x7f57a815e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:44.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.893+0000 7f57ae742640 1 --2- 172.21.15.3:0/1324644454 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f57a806d8f0 0x7f57a815e7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:44.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.893+0000 7f57ae742640 1 -- 172.21.15.3:0/1324644454 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57a814f1b0 con 0x7f57a815e100 2024-05-09T06:36:44.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.893+0000 7f57ae742640 1 --2- 172.21.15.3:0/1324644454 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f57a815e100 0x7f57a815ed20 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f57a000c7b0 tx=0x7f57a000cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:44.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.893+0000 7f579e7fc640 1 -- 172.21.15.3:0/1324644454 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57a0018070 con 0x7f57a815e100 2024-05-09T06:36:44.875 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.893+0000 7f579e7fc640 1 -- 172.21.15.3:0/1324644454 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57a000a640 con 0x7f57a815e100 2024-05-09T06:36:44.875 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.893+0000 7f57b01a5640 1 -- 172.21.15.3:0/1324644454 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57a81520a0 con 0x7f57a815e100 2024-05-09T06:36:44.875 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.893+0000 7f579e7fc640 1 -- 172.21.15.3:0/1324644454 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57a000aa30 con 0x7f57a815e100 2024-05-09T06:36:44.875 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.893+0000 7f57b01a5640 1 -- 172.21.15.3:0/1324644454 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57a815aa10 con 0x7f57a815e100 2024-05-09T06:36:44.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.894+0000 7f57b01a5640 1 -- 172.21.15.3:0/1324644454 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57a804f660 con 0x7f57a815e100 2024-05-09T06:36:44.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.895+0000 7f579e7fc640 1 -- 172.21.15.3:0/1324644454 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f57a001b070 con 0x7f57a815e100 2024-05-09T06:36:44.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.895+0000 7f579e7fc640 1 --2- 172.21.15.3:0/1324644454 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f578c06be30 0x7f578c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:44.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.895+0000 7f579e7fc640 1 -- 172.21.15.3:0/1324644454 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f57a008a3e0 con 0x7f57a815e100 2024-05-09T06:36:44.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.896+0000 7f57adf41640 1 --2- 172.21.15.3:0/1324644454 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f578c06be30 0x7f578c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:44.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.896+0000 7f57adf41640 1 --2- 172.21.15.3:0/1324644454 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f578c06be30 0x7f578c06e2f0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f57a4034000 tx=0x7f57a402e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:44.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:44.899+0000 7f579e7fc640 1 -- 172.21.15.3:0/1324644454 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f57a00586b0 con 0x7f57a815e100 2024-05-09T06:36:45.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.125+0000 7f57b01a5640 1 -- 172.21.15.3:0/1324644454 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f57a80b3cb0 con 0x7f57a815e100 2024-05-09T06:36:45.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.126+0000 7f579e7fc640 1 -- 172.21.15.3:0/1324644454 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5172 (secure 0 0 0) 0x7f57a005c9b0 con 0x7f57a815e100 2024-05-09T06:36:45.106 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-09T06:36:45.106 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4272,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6834/126273055","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":126273055},{"type":"v1","addr":"172.21.15.3:6834","nonce":126273055}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4273,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6836/3885692063","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":3885692063},{"type":"v1","addr":"172.21.15.190:6836","nonce":3885692063}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4275,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/3461228756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6833","nonce":3461228756},{"type":"v1","addr":"172.21.15.3:6835","nonce":3461228756}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-09T06:36:07.221126+0000","modified":"2024-05-09T06:36:10.231051+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4276},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4276":{"gid":4276,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.190:6837/4111472052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6835","nonce":4111472052},{"type":"v1","addr":"172.21.15.190:6837","nonce":4111472052}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4276,"qdb_cluster":[4276]},"id":1}]} 2024-05-09T06:36:45.106 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 8 2024-05-09T06:36:45.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.128+0000 7f57b01a5640 1 -- 172.21.15.3:0/1324644454 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f578c06be30 msgr2=0x7f578c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:45.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.128+0000 7f57b01a5640 1 --2- 172.21.15.3:0/1324644454 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f578c06be30 0x7f578c06e2f0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f57a4034000 tx=0x7f57a402e000 comp rx=0 tx=0).stop 2024-05-09T06:36:45.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.128+0000 7f57b01a5640 1 -- 172.21.15.3:0/1324644454 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f57a815e100 msgr2=0x7f57a815ed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:45.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.128+0000 7f57b01a5640 1 --2- 172.21.15.3:0/1324644454 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f57a815e100 0x7f57a815ed20 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f57a000c7b0 tx=0x7f57a000cc70 comp rx=0 tx=0).stop 2024-05-09T06:36:45.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.128+0000 7f57b01a5640 1 -- 172.21.15.3:0/1324644454 shutdown_connections 2024-05-09T06:36:45.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.128+0000 7f57b01a5640 1 --2- 172.21.15.3:0/1324644454 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f578c06be30 0x7f578c06e2f0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:45.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.128+0000 7f57b01a5640 1 --2- 172.21.15.3:0/1324644454 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f57a815e100 0x7f57a815ed20 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:45.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.128+0000 7f57b01a5640 1 --2- 172.21.15.3:0/1324644454 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f57a806d8f0 0x7f57a815e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:45.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.128+0000 7f57b01a5640 1 --2- 172.21.15.3:0/1324644454 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f57a806bcf0 0x7f57a814ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:45.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.128+0000 7f57b01a5640 1 -- 172.21.15.3:0/1324644454 >> 172.21.15.3:0/1324644454 conn(0x7f57a80d05d0 msgr2=0x7f57a80b0470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:45.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.128+0000 7f57b01a5640 1 -- 172.21.15.3:0/1324644454 shutdown_connections 2024-05-09T06:36:45.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.128+0000 7f57b01a5640 1 -- 172.21.15.3:0/1324644454 wait complete. 2024-05-09T06:36:45.119 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T06:36:45.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.244+0000 7f56d8a7d640 1 Processor -- start 2024-05-09T06:36:45.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.245+0000 7f56d8a7d640 1 -- start start 2024-05-09T06:36:45.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.245+0000 7f56d8a7d640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f56d40d27c0 0x7f56d40d2c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:45.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.246+0000 7f56d8a7d640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f56d4166390 0x7f56d4168780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:45.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.246+0000 7f56d8a7d640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f56d40d4e50 con 0x7f56d415e480 2024-05-09T06:36:45.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.246+0000 7f56d8a7d640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f56d40d3160 con 0x7f56d4166390 2024-05-09T06:36:45.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.246+0000 7f56d8a7d640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f56d40d0270 con 0x7f56d40d27c0 2024-05-09T06:36:45.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.246+0000 7f56d259c640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f56d415e480 0x7f56d40d1dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:51154/0 (socket says 172.21.15.3:51154) 2024-05-09T06:36:45.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.246+0000 7f56d259c640 1 -- 172.21.15.3:0/1020047463 learned_addr learned my addr 172.21.15.3:0/1020047463 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-05-09T06:36:45.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.246+0000 7f56d2d9d640 1 --2- 172.21.15.3:0/1020047463 >> v2:172.21.15.190:3300/0 conn(0x7f56d4166390 0x7f56d4168780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:45.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.246+0000 7f56d159a640 1 -- 172.21.15.3:0/1020047463 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 676436643 0 0) 0x7f56cc001490 con 0x7f56d415e480 2024-05-09T06:36:45.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.246+0000 7f56d1d9b640 1 --2- 172.21.15.3:0/1020047463 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f56d40d27c0 0x7f56d40d2c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:45.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.246+0000 7f56d159a640 1 -- 172.21.15.3:0/1020047463 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f56c0003520 con 0x7f56d415e480 2024-05-09T06:36:45.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.246+0000 7f56d159a640 1 -- 172.21.15.3:0/1020047463 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3030400135 0 0) 0x7f56c0003520 con 0x7f56d415e480 2024-05-09T06:36:45.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.246+0000 7f56d159a640 1 -- 172.21.15.3:0/1020047463 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f56d40d27c0 msgr2=0x7f56d40d2c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:45.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.246+0000 7f56d159a640 1 --2- 172.21.15.3:0/1020047463 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f56d40d27c0 0x7f56d40d2c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:45.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.246+0000 7f56d159a640 1 -- 172.21.15.3:0/1020047463 >> v2:172.21.15.190:3300/0 conn(0x7f56d4166390 msgr2=0x7f56d4168780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:45.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.246+0000 7f56d159a640 1 --2- 172.21.15.3:0/1020047463 >> v2:172.21.15.190:3300/0 conn(0x7f56d4166390 0x7f56d4168780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:45.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.246+0000 7f56d159a640 1 -- 172.21.15.3:0/1020047463 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56d40cf1d0 con 0x7f56d415e480 2024-05-09T06:36:45.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.246+0000 7f56d159a640 1 -- 172.21.15.3:0/1020047463 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 563312031 0 0) 0x7f56cc001bc0 con 0x7f56d415e480 2024-05-09T06:36:45.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.246+0000 7f56d2d9d640 1 --2- 172.21.15.3:0/1020047463 >> v2:172.21.15.190:3300/0 conn(0x7f56d4166390 0x7f56d4168780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:36:45.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.246+0000 7f56d1d9b640 1 --2- 172.21.15.3:0/1020047463 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f56d40d27c0 0x7f56d40d2c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:36:45.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.246+0000 7f56d159a640 1 -- 172.21.15.3:0/1020047463 >> v1:172.21.15.3:6789/0 conn(0x7f56d415e480 legacy=0x7f56d40d1dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:45.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.246+0000 7f56d159a640 1 --2- 172.21.15.3:0/1020047463 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f56d40d27c0 0x7f56c00039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:45.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.246+0000 7f56d159a640 1 --2- 172.21.15.3:0/1020047463 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f56c0003fa0 0x7f56c0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:45.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.246+0000 7f56d159a640 1 --2- 172.21.15.3:0/1020047463 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f56c0004d40 0x7f56c00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:45.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.246+0000 7f56d159a640 1 -- 172.21.15.3:0/1020047463 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f56d40d0270 con 0x7f56c0003fa0 2024-05-09T06:36:45.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.246+0000 7f56d159a640 1 -- 172.21.15.3:0/1020047463 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f56c000ba20 con 0x7f56c0004d40 2024-05-09T06:36:45.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.246+0000 7f56d159a640 1 -- 172.21.15.3:0/1020047463 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f56c000bc20 con 0x7f56d40d27c0 2024-05-09T06:36:45.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.247+0000 7f56d1d9b640 1 --2- 172.21.15.3:0/1020047463 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f56c0003fa0 0x7f56c0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:45.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.247+0000 7f56d259c640 1 --2- 172.21.15.3:0/1020047463 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f56d40d27c0 0x7f56c00039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:45.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.247+0000 7f56d2d9d640 1 --2- 172.21.15.3:0/1020047463 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f56c0004d40 0x7f56c00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:45.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.247+0000 7f56d1d9b640 1 -- 172.21.15.3:0/1020047463 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f56d40d27c0 msgr2=0x7f56c00039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:45.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.247+0000 7f56d1d9b640 1 --2- 172.21.15.3:0/1020047463 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f56d40d27c0 0x7f56c00039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:45.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.247+0000 7f56d1d9b640 1 -- 172.21.15.3:0/1020047463 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f56c0004d40 msgr2=0x7f56c00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:45.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.247+0000 7f56d1d9b640 1 --2- 172.21.15.3:0/1020047463 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f56c0004d40 0x7f56c00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:45.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.247+0000 7f56d1d9b640 1 -- 172.21.15.3:0/1020047463 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56c0003520 con 0x7f56c0003fa0 2024-05-09T06:36:45.231 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.247+0000 7f56d2d9d640 1 --2- 172.21.15.3:0/1020047463 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f56c0004d40 0x7f56c00051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:36:45.231 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.247+0000 7f56d1d9b640 1 --2- 172.21.15.3:0/1020047463 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f56c0003fa0 0x7f56c0004800 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f56d408c750 tx=0x7f56c0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fa6d59a85897885 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:45.231 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.248+0000 7f56d159a640 1 -- 172.21.15.3:0/1020047463 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56bc037070 con 0x7f56c0003fa0 2024-05-09T06:36:45.231 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.248+0000 7f56d159a640 1 -- 172.21.15.3:0/1020047463 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f56bc02aca0 con 0x7f56c0003fa0 2024-05-09T06:36:45.231 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.248+0000 7f56d159a640 1 -- 172.21.15.3:0/1020047463 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56bc00f040 con 0x7f56c0003fa0 2024-05-09T06:36:45.231 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.248+0000 7f56d8a7d640 1 -- 172.21.15.3:0/1020047463 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f56c0003fa0 msgr2=0x7f56c0004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:45.231 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.248+0000 7f56d8a7d640 1 --2- 172.21.15.3:0/1020047463 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f56c0003fa0 0x7f56c0004800 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f56d408c750 tx=0x7f56c0005830 comp rx=0 tx=0).stop 2024-05-09T06:36:45.231 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.248+0000 7f56d2d9d640 1 -- 172.21.15.3:0/1020047463 reap_dead start 2024-05-09T06:36:45.231 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.248+0000 7f56d8a7d640 1 -- 172.21.15.3:0/1020047463 shutdown_connections 2024-05-09T06:36:45.231 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.248+0000 7f56d8a7d640 1 -- 172.21.15.3:0/1020047463 >> 172.21.15.3:0/1020047463 conn(0x7f56d409d4d0 msgr2=0x7f56d40d4200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:45.231 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.248+0000 7f56d8a7d640 1 -- 172.21.15.3:0/1020047463 shutdown_connections 2024-05-09T06:36:45.231 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.248+0000 7f56d8a7d640 1 -- 172.21.15.3:0/1020047463 wait complete. 2024-05-09T06:36:45.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.250+0000 7f56d8a7d640 1 Processor -- start 2024-05-09T06:36:45.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.250+0000 7f56d8a7d640 1 -- start start 2024-05-09T06:36:45.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.251+0000 7f56d8a7d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f56c0003fa0 0x7f56d41491a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:45.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.251+0000 7f56d8a7d640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f56d40b98c0 0x7f56d40b9d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:45.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.251+0000 7f56d8a7d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f56d40cd190 0x7f56d40cd610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:45.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.251+0000 7f56d8a7d640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f56d40cfa70 con 0x7f56d40cd190 2024-05-09T06:36:45.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.251+0000 7f56d8a7d640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f56d40cf790 con 0x7f56c0003fa0 2024-05-09T06:36:45.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.251+0000 7f56d8a7d640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f56d40cfd50 con 0x7f56d40b98c0 2024-05-09T06:36:45.235 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.251+0000 7f56d2d9d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f56d40cd190 0x7f56d40cd610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:45.235 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.251+0000 7f56d2d9d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f56d40cd190 0x7f56d40cd610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46658/0 (socket says 172.21.15.3:46658) 2024-05-09T06:36:45.235 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.251+0000 7f56d2d9d640 1 -- 172.21.15.3:0/1385350261 learned_addr learned my addr 172.21.15.3:0/1385350261 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:36:45.235 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.251+0000 7f56d259c640 1 --2- 172.21.15.3:0/1385350261 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f56c0003fa0 0x7f56d41491a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:45.235 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.251+0000 7f56d1d9b640 1 --2- 172.21.15.3:0/1385350261 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f56d40b98c0 0x7f56d40b9d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:45.235 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.251+0000 7f56d2d9d640 1 -- 172.21.15.3:0/1385350261 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f56d40b98c0 msgr2=0x7f56d40b9d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:45.235 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.251+0000 7f56d2d9d640 1 --2- 172.21.15.3:0/1385350261 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f56d40b98c0 0x7f56d40b9d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:45.235 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.251+0000 7f56d2d9d640 1 -- 172.21.15.3:0/1385350261 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f56c0003fa0 msgr2=0x7f56d41491a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:45.235 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.251+0000 7f56d2d9d640 1 --2- 172.21.15.3:0/1385350261 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f56c0003fa0 0x7f56d41491a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:45.235 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.251+0000 7f56d2d9d640 1 -- 172.21.15.3:0/1385350261 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56d40b79b0 con 0x7f56d40cd190 2024-05-09T06:36:45.235 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.251+0000 7f56d2d9d640 1 --2- 172.21.15.3:0/1385350261 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f56d40cd190 0x7f56d40cd610 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f56d40d41d0 tx=0x7f56c40109f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:45.236 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.251+0000 7f56caffd640 1 -- 172.21.15.3:0/1385350261 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56c400f040 con 0x7f56d40cd190 2024-05-09T06:36:45.236 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.251+0000 7f56caffd640 1 -- 172.21.15.3:0/1385350261 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f56c401b070 con 0x7f56d40cd190 2024-05-09T06:36:45.236 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.251+0000 7f56caffd640 1 -- 172.21.15.3:0/1385350261 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56c4003610 con 0x7f56d40cd190 2024-05-09T06:36:45.236 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.251+0000 7f56d8a7d640 1 -- 172.21.15.3:0/1385350261 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56d40b7c90 con 0x7f56d40cd190 2024-05-09T06:36:45.236 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.251+0000 7f56d8a7d640 1 -- 172.21.15.3:0/1385350261 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56d40b8250 con 0x7f56d40cd190 2024-05-09T06:36:45.236 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.253+0000 7f56d8a7d640 1 -- 172.21.15.3:0/1385350261 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56d404f660 con 0x7f56d40cd190 2024-05-09T06:36:45.236 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.253+0000 7f56caffd640 1 -- 172.21.15.3:0/1385350261 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f56c40037b0 con 0x7f56d40cd190 2024-05-09T06:36:45.236 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.254+0000 7f56caffd640 1 --2- 172.21.15.3:0/1385350261 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f56b406be60 0x7f56b406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:45.236 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.254+0000 7f56caffd640 1 -- 172.21.15.3:0/1385350261 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f56c408adc0 con 0x7f56d40cd190 2024-05-09T06:36:45.236 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.254+0000 7f56d259c640 1 --2- 172.21.15.3:0/1385350261 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f56b406be60 0x7f56b406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:45.236 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.254+0000 7f56d259c640 1 --2- 172.21.15.3:0/1385350261 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f56b406be60 0x7f56b406e320 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f56cc002230 tx=0x7f56cc006040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:45.238 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.258+0000 7f56caffd640 1 -- 172.21.15.3:0/1385350261 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f56c4059800 con 0x7f56d40cd190 2024-05-09T06:36:45.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.443+0000 7f56d8a7d640 1 -- 172.21.15.3:0/1385350261 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f56d40d0ae0 con 0x7f56d40cd190 2024-05-09T06:36:45.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.444+0000 7f56caffd640 1 -- 172.21.15.3:0/1385350261 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16521 (secure 0 0 0) 0x7f56c4059100 con 0x7f56d40cd190 2024-05-09T06:36:45.425 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-09T06:36:45.425 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":20,"fsid":"16ed737d-2665-431a-8ac6-ef242be3975a","created":"2024-05-09T06:35:42.541368+0000","modified":"2024-05-09T06:36:08.856391+0000","last_up_change":"2024-05-09T06:35:53.778274+0000","last_in_change":"2024-05-09T06:35:44.780586+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T06:35:54.819999+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T06:36:04.931314+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T06:36:06.200433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fae3a8db-4826-4f3c-b33a-12c09bb5df6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":1897849827},{"type":"v1","addr":"172.21.15.3:6817","nonce":1897849827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":1897849827},{"type":"v1","addr":"172.21.15.3:6819","nonce":1897849827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":1897849827},{"type":"v1","addr":"172.21.15.3:6823","nonce":1897849827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":1897849827},{"type":"v1","addr":"172.21.15.3:6821","nonce":1897849827}]},"public_addr":"172.21.15.3:6817/1897849827","cluster_addr":"172.21.15.3:6819/1897849827","heartbeat_back_addr":"172.21.15.3:6823/1897849827","heartbeat_front_addr":"172.21.15.3:6821/1897849827","state":["exists","up"]},{"osd":1,"uuid":"646e882d-4ce4-4720-8c84-f71df563a35e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":926805498},{"type":"v1","addr":"172.21.15.3:6801","nonce":926805498}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":926805498},{"type":"v1","addr":"172.21.15.3:6803","nonce":926805498}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":926805498},{"type":"v1","addr":"172.21.15.3:6807","nonce":926805498}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":926805498},{"type":"v1","addr":"172.21.15.3:6805","nonce":926805498}]},"public_addr":"172.21.15.3:6801/926805498","cluster_addr":"172.21.15.3:6803/926805498","heartbeat_back_addr":"172.21.15.3:6807/926805498","heartbeat_front_addr":"172.21.15.3:6805/926805498","state":["exists","up"]},{"osd":2,"uuid":"1abbadb5-a570-4359-9643-acff3ede35a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":801295451},{"type":"v1","addr":"172.21.15.3:6809","nonce":801295451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":801295451},{"type":"v1","addr":"172.21.15.3:6811","nonce":801295451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":801295451},{"type":"v1","addr":"172.21.15.3:6815","nonce":801295451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":801295451},{"type":"v1","addr":"172.21.15.3:6813","nonce":801295451}]},"public_addr":"172.21.15.3:6809/801295451","cluster_addr":"172.21.15.3:6811/801295451","heartbeat_back_addr":"172.21.15.3:6815/801295451","heartbeat_front_addr":"172.21.15.3:6813/801295451","state":["exists","up"]},{"osd":3,"uuid":"62ac4dc1-2371-4dd2-acd3-e2dbb11aedf5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":1903389857},{"type":"v1","addr":"172.21.15.3:6825","nonce":1903389857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":1903389857},{"type":"v1","addr":"172.21.15.3:6827","nonce":1903389857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":1903389857},{"type":"v1","addr":"172.21.15.3:6831","nonce":1903389857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":1903389857},{"type":"v1","addr":"172.21.15.3:6829","nonce":1903389857}]},"public_addr":"172.21.15.3:6825/1903389857","cluster_addr":"172.21.15.3:6827/1903389857","heartbeat_back_addr":"172.21.15.3:6831/1903389857","heartbeat_front_addr":"172.21.15.3:6829/1903389857","state":["exists","up"]},{"osd":4,"uuid":"ae808398-8326-4d7c-9319-a2a0eafffb59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":2143624576},{"type":"v1","addr":"172.21.15.190:6801","nonce":2143624576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":2143624576},{"type":"v1","addr":"172.21.15.190:6803","nonce":2143624576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":2143624576},{"type":"v1","addr":"172.21.15.190:6807","nonce":2143624576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":2143624576},{"type":"v1","addr":"172.21.15.190:6805","nonce":2143624576}]},"public_addr":"172.21.15.190:6801/2143624576","cluster_addr":"172.21.15.190:6803/2143624576","heartbeat_back_addr":"172.21.15.190:6807/2143624576","heartbeat_front_addr":"172.21.15.190:6805/2143624576","state":["exists","up"]},{"osd":5,"uuid":"918adf50-c440-48ac-9a8d-4837562d7275","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":1766923126},{"type":"v1","addr":"172.21.15.190:6814","nonce":1766923126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":1766923126},{"type":"v1","addr":"172.21.15.190:6818","nonce":1766923126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":1766923126},{"type":"v1","addr":"172.21.15.190:6823","nonce":1766923126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":1766923126},{"type":"v1","addr":"172.21.15.190:6821","nonce":1766923126}]},"public_addr":"172.21.15.190:6814/1766923126","cluster_addr":"172.21.15.190:6818/1766923126","heartbeat_back_addr":"172.21.15.190:6823/1766923126","heartbeat_front_addr":"172.21.15.190:6821/1766923126","state":["exists","up"]},{"osd":6,"uuid":"ac2882bc-2a7c-463c-8f6c-76ff49828adc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":342414248},{"type":"v1","addr":"172.21.15.190:6825","nonce":342414248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":342414248},{"type":"v1","addr":"172.21.15.190:6827","nonce":342414248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":342414248},{"type":"v1","addr":"172.21.15.190:6831","nonce":342414248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":342414248},{"type":"v1","addr":"172.21.15.190:6829","nonce":342414248}]},"public_addr":"172.21.15.190:6825/342414248","cluster_addr":"172.21.15.190:6827/342414248","heartbeat_back_addr":"172.21.15.190:6831/342414248","heartbeat_front_addr":"172.21.15.190:6829/342414248","state":["exists","up"]},{"osd":7,"uuid":"dd4e767a-4730-4c70-8bd5-3f088140a5a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":537849891},{"type":"v1","addr":"172.21.15.190:6809","nonce":537849891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":537849891},{"type":"v1","addr":"172.21.15.190:6811","nonce":537849891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6817","nonce":537849891},{"type":"v1","addr":"172.21.15.190:6819","nonce":537849891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6813","nonce":537849891},{"type":"v1","addr":"172.21.15.190:6815","nonce":537849891}]},"public_addr":"172.21.15.190:6809/537849891","cluster_addr":"172.21.15.190:6811/537849891","heartbeat_back_addr":"172.21.15.190:6819/537849891","heartbeat_front_addr":"172.21.15.190:6815/537849891","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.889784+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.974002+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.973549+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.955396+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.928471+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.902418+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.958696+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.926474+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T06:36:45.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.446+0000 7f56d8a7d640 1 -- 172.21.15.3:0/1385350261 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f56b406be60 msgr2=0x7f56b406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:45.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.446+0000 7f56d8a7d640 1 --2- 172.21.15.3:0/1385350261 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f56b406be60 0x7f56b406e320 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f56cc002230 tx=0x7f56cc006040 comp rx=0 tx=0).stop 2024-05-09T06:36:45.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.446+0000 7f56d8a7d640 1 -- 172.21.15.3:0/1385350261 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f56d40cd190 msgr2=0x7f56d40cd610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:45.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.446+0000 7f56d8a7d640 1 --2- 172.21.15.3:0/1385350261 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f56d40cd190 0x7f56d40cd610 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f56d40d41d0 tx=0x7f56c40109f0 comp rx=0 tx=0).stop 2024-05-09T06:36:45.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.446+0000 7f56d8a7d640 1 -- 172.21.15.3:0/1385350261 shutdown_connections 2024-05-09T06:36:45.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.446+0000 7f56d8a7d640 1 --2- 172.21.15.3:0/1385350261 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f56b406be60 0x7f56b406e320 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:45.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.446+0000 7f56d8a7d640 1 --2- 172.21.15.3:0/1385350261 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f56d40cd190 0x7f56d40cd610 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:45.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.446+0000 7f56d8a7d640 1 --2- 172.21.15.3:0/1385350261 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f56d40b98c0 0x7f56d40b9d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:45.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.446+0000 7f56d8a7d640 1 --2- 172.21.15.3:0/1385350261 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f56c0003fa0 0x7f56d41491a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:45.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.446+0000 7f56d8a7d640 1 -- 172.21.15.3:0/1385350261 >> 172.21.15.3:0/1385350261 conn(0x7f56d409d4d0 msgr2=0x7f56d40b26b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:45.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.446+0000 7f56d8a7d640 1 -- 172.21.15.3:0/1385350261 shutdown_connections 2024-05-09T06:36:45.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.446+0000 7f56d8a7d640 1 -- 172.21.15.3:0/1385350261 wait complete. 2024-05-09T06:36:45.437 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-09T06:35:54.819999+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-09T06:36:04.931314+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-09T06:36:06.200433+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-09T06:36:45.438 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-09T06:36:45.545 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.565+0000 7f573a093640 1 Processor -- start 2024-05-09T06:36:45.545 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.565+0000 7f573a093640 1 -- start start 2024-05-09T06:36:45.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.566+0000 7f573a093640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f573415e0b0 0x7f5734161e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:45.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.566+0000 7f573a093640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f573406cad0 con 0x7f573406b250 2024-05-09T06:36:45.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.566+0000 7f573a093640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f57340b3eb0 con 0x7f573415e490 2024-05-09T06:36:45.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.566+0000 7f573a093640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f57340b4190 con 0x7f573415e0b0 2024-05-09T06:36:45.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.566+0000 7f57337fe640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f573415e0b0 0x7f5734161e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:45.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.566+0000 7f5733fff640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f573406b250 0x7f573406b6c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:51156/0 (socket says 172.21.15.3:51156) 2024-05-09T06:36:45.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.566+0000 7f5733fff640 1 -- 172.21.15.3:0/2128407496 learned_addr learned my addr 172.21.15.3:0/2128407496 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-05-09T06:36:45.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.566+0000 7f57337fe640 1 --2- 172.21.15.3:0/2128407496 >> v2:172.21.15.3:3300/0 conn(0x7f573415e0b0 0x7f5734161e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46670/0 (socket says 172.21.15.3:46670) 2024-05-09T06:36:45.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.566+0000 7f57337fe640 1 -- 172.21.15.3:0/2128407496 >> v1:172.21.15.190:6789/0 conn(0x7f573415e490 legacy=0x7f57341623c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:45.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.566+0000 7f57337fe640 1 -- 172.21.15.3:0/2128407496 >> v1:172.21.15.3:6789/0 conn(0x7f573406b250 legacy=0x7f573406b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:45.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.566+0000 7f57327fc640 1 -- 172.21.15.3:0/2128407496 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3348207327 0 0) 0x7f573406cad0 con 0x7f573406b250 2024-05-09T06:36:45.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.566+0000 7f57337fe640 1 -- 172.21.15.3:0/2128407496 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57340b4470 con 0x7f573415e0b0 2024-05-09T06:36:45.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.567+0000 7f57337fe640 1 --2- 172.21.15.3:0/2128407496 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f573415e0b0 0x7f5734161e80 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f57240097a0 tx=0x7f572402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4c9ef8f37f8c6ae server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:45.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.567+0000 7f57327fc640 1 -- 172.21.15.3:0/2128407496 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5724037070 con 0x7f573415e0b0 2024-05-09T06:36:45.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.567+0000 7f57327fc640 1 -- 172.21.15.3:0/2128407496 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5724032440 con 0x7f573415e0b0 2024-05-09T06:36:45.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.567+0000 7f57327fc640 1 -- 172.21.15.3:0/2128407496 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5724032810 con 0x7f573415e0b0 2024-05-09T06:36:45.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.568+0000 7f573a093640 1 -- 172.21.15.3:0/2128407496 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f573415e0b0 msgr2=0x7f5734161e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:45.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.568+0000 7f573a093640 1 --2- 172.21.15.3:0/2128407496 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f573415e0b0 0x7f5734161e80 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f57240097a0 tx=0x7f572402a1e0 comp rx=0 tx=0).stop 2024-05-09T06:36:45.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.568+0000 7f573a093640 1 -- 172.21.15.3:0/2128407496 shutdown_connections 2024-05-09T06:36:45.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.568+0000 7f573a093640 1 --2- 172.21.15.3:0/2128407496 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f573415e0b0 0x7f5734161e80 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:45.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.568+0000 7f573a093640 1 -- 172.21.15.3:0/2128407496 >> 172.21.15.3:0/2128407496 conn(0x7f57340d07b0 msgr2=0x7f57340d0bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:45.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.568+0000 7f573a093640 1 -- 172.21.15.3:0/2128407496 shutdown_connections 2024-05-09T06:36:45.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.568+0000 7f573a093640 1 -- 172.21.15.3:0/2128407496 wait complete. 2024-05-09T06:36:45.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.569+0000 7f573a093640 1 Processor -- start 2024-05-09T06:36:45.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.569+0000 7f573a093640 1 -- start start 2024-05-09T06:36:45.551 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.570+0000 7f573a093640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f573406b250 0x7f5734144960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:45.551 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.570+0000 7f573a093640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f573415e0b0 0x7f57341e7b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:45.551 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.570+0000 7f573a093640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f573415e490 0x7f573415c910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:45.551 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.570+0000 7f573a093640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f573406da70 con 0x7f573415e0b0 2024-05-09T06:36:45.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.570+0000 7f573a093640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f573406d8f0 con 0x7f573415e490 2024-05-09T06:36:45.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.570+0000 7f573a093640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f5734162ad0 con 0x7f573406b250 2024-05-09T06:36:45.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.570+0000 7f5732ffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f573415e0b0 0x7f57341e7b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:45.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.570+0000 7f5732ffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f573415e0b0 0x7f57341e7b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46678/0 (socket says 172.21.15.3:46678) 2024-05-09T06:36:45.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.570+0000 7f5732ffd640 1 -- 172.21.15.3:0/1000180530 learned_addr learned my addr 172.21.15.3:0/1000180530 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:36:45.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.570+0000 7f57337fe640 1 --2- 172.21.15.3:0/1000180530 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f573406b250 0x7f5734144960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:45.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.570+0000 7f5733fff640 1 --2- 172.21.15.3:0/1000180530 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f573415e490 0x7f573415c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:45.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.570+0000 7f5732ffd640 1 -- 172.21.15.3:0/1000180530 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f573406b250 msgr2=0x7f5734144960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:45.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.570+0000 7f5732ffd640 1 --2- 172.21.15.3:0/1000180530 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f573406b250 0x7f5734144960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:45.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.570+0000 7f5732ffd640 1 -- 172.21.15.3:0/1000180530 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f573415e490 msgr2=0x7f573415c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:45.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.570+0000 7f5732ffd640 1 --2- 172.21.15.3:0/1000180530 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f573415e490 0x7f573415c910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:45.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.570+0000 7f5732ffd640 1 -- 172.21.15.3:0/1000180530 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57341e8450 con 0x7f573415e0b0 2024-05-09T06:36:45.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.570+0000 7f57337fe640 1 --2- 172.21.15.3:0/1000180530 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f573406b250 0x7f5734144960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:36:45.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.571+0000 7f5732ffd640 1 --2- 172.21.15.3:0/1000180530 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f573415e0b0 0x7f57341e7b00 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f572800c410 tx=0x7f572800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:45.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.571+0000 7f571bfff640 1 -- 172.21.15.3:0/1000180530 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5728017070 con 0x7f573415e0b0 2024-05-09T06:36:45.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.571+0000 7f571bfff640 1 -- 172.21.15.3:0/1000180530 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57280035f0 con 0x7f573415e0b0 2024-05-09T06:36:45.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.571+0000 7f571bfff640 1 -- 172.21.15.3:0/1000180530 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5728003a20 con 0x7f573415e0b0 2024-05-09T06:36:45.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.571+0000 7f573a093640 1 -- 172.21.15.3:0/1000180530 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57341e8730 con 0x7f573415e0b0 2024-05-09T06:36:45.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.571+0000 7f573a093640 1 -- 172.21.15.3:0/1000180530 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57341e8cf0 con 0x7f573415e0b0 2024-05-09T06:36:45.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.572+0000 7f573a093640 1 -- 172.21.15.3:0/1000180530 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5734161e80 con 0x7f573415e0b0 2024-05-09T06:36:45.554 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.572+0000 7f571bfff640 1 -- 172.21.15.3:0/1000180530 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f572801a070 con 0x7f573415e0b0 2024-05-09T06:36:45.554 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.573+0000 7f571bfff640 1 --2- 172.21.15.3:0/1000180530 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f572006be30 0x7f572006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:45.554 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.573+0000 7f571bfff640 1 -- 172.21.15.3:0/1000180530 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f572808a310 con 0x7f573415e0b0 2024-05-09T06:36:45.554 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.573+0000 7f57337fe640 1 --2- 172.21.15.3:0/1000180530 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f572006be30 0x7f572006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:45.554 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.574+0000 7f57337fe640 1 --2- 172.21.15.3:0/1000180530 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f572006be30 0x7f572006e2f0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f5724034000 tx=0x7f572402e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:45.557 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.577+0000 7f571bfff640 1 -- 172.21.15.3:0/1000180530 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f5734161e80 con 0x7f573415e0b0 2024-05-09T06:36:45.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.759+0000 7f573a093640 1 -- 172.21.15.3:0/1000180530 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f5734069550 con 0x7f573415e0b0 2024-05-09T06:36:45.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.759+0000 7f571bfff640 1 -- 172.21.15.3:0/1000180530 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7f5728058cc0 con 0x7f573415e0b0 2024-05-09T06:36:45.739 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 1 2024-05-09T06:36:45.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.761+0000 7f573a093640 1 -- 172.21.15.3:0/1000180530 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f572006be30 msgr2=0x7f572006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:45.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.761+0000 7f573a093640 1 --2- 172.21.15.3:0/1000180530 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f572006be30 0x7f572006e2f0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f5724034000 tx=0x7f572402e000 comp rx=0 tx=0).stop 2024-05-09T06:36:45.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.761+0000 7f573a093640 1 -- 172.21.15.3:0/1000180530 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f573415e0b0 msgr2=0x7f57341e7b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:45.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.761+0000 7f573a093640 1 --2- 172.21.15.3:0/1000180530 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f573415e0b0 0x7f57341e7b00 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f572800c410 tx=0x7f572800c8d0 comp rx=0 tx=0).stop 2024-05-09T06:36:45.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.761+0000 7f573a093640 1 -- 172.21.15.3:0/1000180530 shutdown_connections 2024-05-09T06:36:45.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.761+0000 7f573a093640 1 --2- 172.21.15.3:0/1000180530 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f572006be30 0x7f572006e2f0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:45.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.761+0000 7f573a093640 1 --2- 172.21.15.3:0/1000180530 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f573415e490 0x7f573415c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:45.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.761+0000 7f573a093640 1 --2- 172.21.15.3:0/1000180530 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f573415e0b0 0x7f57341e7b00 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:45.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.761+0000 7f573a093640 1 --2- 172.21.15.3:0/1000180530 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f573406b250 0x7f5734144960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:45.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.761+0000 7f573a093640 1 -- 172.21.15.3:0/1000180530 >> 172.21.15.3:0/1000180530 conn(0x7f57340d07b0 msgr2=0x7f573406f430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:45.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.761+0000 7f573a093640 1 -- 172.21.15.3:0/1000180530 shutdown_connections 2024-05-09T06:36:45.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.761+0000 7f573a093640 1 -- 172.21.15.3:0/1000180530 wait complete. 2024-05-09T06:36:45.752 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-09T06:36:45.859 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.880+0000 7f20fe23f640 1 Processor -- start 2024-05-09T06:36:45.860 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.880+0000 7f20fe23f640 1 -- start start 2024-05-09T06:36:45.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.880+0000 7f20fe23f640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f20f815dff0 0x7f20f80ae600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:45.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.880+0000 7f20fe23f640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f20f8161df0 0x7f20f806e600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:45.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.881+0000 7f20fe23f640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f20f8162280 con 0x7f20f80b4520 2024-05-09T06:36:45.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.881+0000 7f20fe23f640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f20f8162560 con 0x7f20f815dff0 2024-05-09T06:36:45.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.881+0000 7f20fe23f640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f20f8162840 con 0x7f20f8161df0 2024-05-09T06:36:45.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.881+0000 7f20f77fe640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f20f815dff0 0x7f20f80ae600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:45.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.881+0000 7f20f77fe640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f20f815dff0 0x7f20f80ae600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46690/0 (socket says 172.21.15.3:46690) 2024-05-09T06:36:45.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.881+0000 7f20f77fe640 1 -- 172.21.15.3:0/2711139459 learned_addr learned my addr 172.21.15.3:0/2711139459 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:36:45.862 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.881+0000 7f20f6ffd640 1 --2- 172.21.15.3:0/2711139459 >> v2:172.21.15.190:3300/0 conn(0x7f20f8161df0 0x7f20f806e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:45.862 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.881+0000 7f20f77fe640 1 -- 172.21.15.3:0/2711139459 >> v2:172.21.15.190:3300/0 conn(0x7f20f8161df0 msgr2=0x7f20f806e600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:45.862 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.881+0000 7f20f77fe640 1 --2- 172.21.15.3:0/2711139459 >> v2:172.21.15.190:3300/0 conn(0x7f20f8161df0 0x7f20f806e600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:45.862 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.881+0000 7f20f77fe640 1 -- 172.21.15.3:0/2711139459 >> v1:172.21.15.190:6789/0 conn(0x7f20f80b4520 legacy=0x7f20f80728f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:45.862 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.881+0000 7f20f77fe640 1 -- 172.21.15.3:0/2711139459 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20f8162b20 con 0x7f20f815dff0 2024-05-09T06:36:45.862 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.881+0000 7f20f6ffd640 1 --2- 172.21.15.3:0/2711139459 >> v2:172.21.15.190:3300/0 conn(0x7f20f8161df0 0x7f20f806e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:36:45.862 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.882+0000 7f20f77fe640 1 --2- 172.21.15.3:0/2711139459 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f20f815dff0 0x7f20f80ae600 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f20f806f2d0 tx=0x7f20e002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6157b2e6041d307 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:45.863 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.882+0000 7f20f67fc640 1 -- 172.21.15.3:0/2711139459 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20e0037070 con 0x7f20f815dff0 2024-05-09T06:36:45.863 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.882+0000 7f20f67fc640 1 -- 172.21.15.3:0/2711139459 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f20e0032440 con 0x7f20f815dff0 2024-05-09T06:36:45.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.882+0000 7f20f67fc640 1 -- 172.21.15.3:0/2711139459 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20e0032810 con 0x7f20f815dff0 2024-05-09T06:36:45.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.882+0000 7f20fe23f640 1 -- 172.21.15.3:0/2711139459 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f20f815dff0 msgr2=0x7f20f80ae600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:45.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.882+0000 7f20fe23f640 1 --2- 172.21.15.3:0/2711139459 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f20f815dff0 0x7f20f80ae600 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f20f806f2d0 tx=0x7f20e002a1a0 comp rx=0 tx=0).stop 2024-05-09T06:36:45.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.882+0000 7f20fe23f640 1 -- 172.21.15.3:0/2711139459 shutdown_connections 2024-05-09T06:36:45.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.882+0000 7f20fe23f640 1 --2- 172.21.15.3:0/2711139459 >> v2:172.21.15.190:3300/0 conn(0x7f20f8161df0 0x7f20f806e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:45.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.882+0000 7f20fe23f640 1 --2- 172.21.15.3:0/2711139459 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f20f815dff0 0x7f20f80ae600 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:45.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.882+0000 7f20fe23f640 1 -- 172.21.15.3:0/2711139459 >> 172.21.15.3:0/2711139459 conn(0x7f20f80b6530 msgr2=0x7f20f80b6910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:45.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.883+0000 7f20fe23f640 1 -- 172.21.15.3:0/2711139459 shutdown_connections 2024-05-09T06:36:45.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.883+0000 7f20fe23f640 1 -- 172.21.15.3:0/2711139459 wait complete. 2024-05-09T06:36:45.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.884+0000 7f20fe23f640 1 Processor -- start 2024-05-09T06:36:45.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.884+0000 7f20fe23f640 1 -- start start 2024-05-09T06:36:45.866 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.885+0000 7f20fe23f640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f20f80b4520 0x7f20f8150eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:45.866 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.885+0000 7f20fe23f640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f20f815dff0 0x7f20f814eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:45.866 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.885+0000 7f20fe23f640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f20f8152870 0x7f20f8152e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:45.866 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.885+0000 7f20fe23f640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f20f80b4b90 con 0x7f20f815dff0 2024-05-09T06:36:45.866 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.885+0000 7f20fe23f640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f20f815e620 con 0x7f20f8152870 2024-05-09T06:36:45.866 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.885+0000 7f20fe23f640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f20f80b4d10 con 0x7f20f80b4520 2024-05-09T06:36:45.866 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.885+0000 7f20f6ffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f20f815dff0 0x7f20f814eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:45.867 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.885+0000 7f20f6ffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f20f815dff0 0x7f20f814eb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46692/0 (socket says 172.21.15.3:46692) 2024-05-09T06:36:45.867 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.885+0000 7f20f6ffd640 1 -- 172.21.15.3:0/2360692794 learned_addr learned my addr 172.21.15.3:0/2360692794 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:36:45.867 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.885+0000 7f20f77fe640 1 --2- 172.21.15.3:0/2360692794 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f20f80b4520 0x7f20f8150eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:45.867 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.885+0000 7f20f7fff640 1 --2- 172.21.15.3:0/2360692794 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f20f8152870 0x7f20f8152e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:45.867 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.885+0000 7f20f6ffd640 1 -- 172.21.15.3:0/2360692794 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f20f80b4520 msgr2=0x7f20f8150eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:45.867 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.885+0000 7f20f6ffd640 1 --2- 172.21.15.3:0/2360692794 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f20f80b4520 0x7f20f8150eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:45.867 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.885+0000 7f20f6ffd640 1 -- 172.21.15.3:0/2360692794 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f20f8152870 msgr2=0x7f20f8152e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:45.867 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.885+0000 7f20f6ffd640 1 --2- 172.21.15.3:0/2360692794 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f20f8152870 0x7f20f8152e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:45.867 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.885+0000 7f20f6ffd640 1 -- 172.21.15.3:0/2360692794 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20f81517b0 con 0x7f20f815dff0 2024-05-09T06:36:45.867 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.885+0000 7f20f6ffd640 1 --2- 172.21.15.3:0/2360692794 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f20f815dff0 0x7f20f814eb40 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f20e800c7f0 tx=0x7f20e800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:45.867 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.885+0000 7f20effff640 1 -- 172.21.15.3:0/2360692794 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20e8015070 con 0x7f20f815dff0 2024-05-09T06:36:45.867 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.885+0000 7f20effff640 1 -- 172.21.15.3:0/2360692794 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f20e8007a30 con 0x7f20f815dff0 2024-05-09T06:36:45.867 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.885+0000 7f20fe23f640 1 -- 172.21.15.3:0/2360692794 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20f8151a90 con 0x7f20f815dff0 2024-05-09T06:36:45.867 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.885+0000 7f20effff640 1 -- 172.21.15.3:0/2360692794 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20e8007e00 con 0x7f20f815dff0 2024-05-09T06:36:45.868 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.885+0000 7f20fe23f640 1 -- 172.21.15.3:0/2360692794 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20f81f4320 con 0x7f20f815dff0 2024-05-09T06:36:45.868 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.886+0000 7f20fe23f640 1 -- 172.21.15.3:0/2360692794 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20f815fa20 con 0x7f20f815dff0 2024-05-09T06:36:45.868 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.887+0000 7f20effff640 1 -- 172.21.15.3:0/2360692794 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f20e8018070 con 0x7f20f815dff0 2024-05-09T06:36:45.868 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.888+0000 7f20effff640 1 --2- 172.21.15.3:0/2360692794 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f20d806be30 0x7f20d806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:45.869 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.888+0000 7f20effff640 1 -- 172.21.15.3:0/2360692794 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f20e808ad10 con 0x7f20f815dff0 2024-05-09T06:36:45.869 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.888+0000 7f20f77fe640 1 --2- 172.21.15.3:0/2360692794 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f20d806be30 0x7f20d806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:45.869 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.888+0000 7f20f77fe640 1 --2- 172.21.15.3:0/2360692794 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f20d806be30 0x7f20d806e2f0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f20e0034000 tx=0x7f20e002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:45.871 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:45.892+0000 7f20effff640 1 -- 172.21.15.3:0/2360692794 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f20e8059100 con 0x7f20f815dff0 2024-05-09T06:36:46.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.074+0000 7f20fe23f640 1 -- 172.21.15.3:0/2360692794 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f20f8069550 con 0x7f20f815dff0 2024-05-09T06:36:46.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.074+0000 7f20effff640 1 -- 172.21.15.3:0/2360692794 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7f20e805d400 con 0x7f20f815dff0 2024-05-09T06:36:46.054 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 32 2024-05-09T06:36:46.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.076+0000 7f20fe23f640 1 -- 172.21.15.3:0/2360692794 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f20d806be30 msgr2=0x7f20d806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:46.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.076+0000 7f20fe23f640 1 --2- 172.21.15.3:0/2360692794 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f20d806be30 0x7f20d806e2f0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f20e0034000 tx=0x7f20e002e000 comp rx=0 tx=0).stop 2024-05-09T06:36:46.057 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.076+0000 7f20fe23f640 1 -- 172.21.15.3:0/2360692794 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f20f815dff0 msgr2=0x7f20f814eb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:46.057 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.076+0000 7f20fe23f640 1 --2- 172.21.15.3:0/2360692794 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f20f815dff0 0x7f20f814eb40 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f20e800c7f0 tx=0x7f20e800cbf0 comp rx=0 tx=0).stop 2024-05-09T06:36:46.057 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.076+0000 7f20fe23f640 1 -- 172.21.15.3:0/2360692794 shutdown_connections 2024-05-09T06:36:46.057 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.076+0000 7f20fe23f640 1 --2- 172.21.15.3:0/2360692794 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f20d806be30 0x7f20d806e2f0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:46.057 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.076+0000 7f20fe23f640 1 --2- 172.21.15.3:0/2360692794 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f20f8152870 0x7f20f8152e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:46.057 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.076+0000 7f20fe23f640 1 --2- 172.21.15.3:0/2360692794 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f20f815dff0 0x7f20f814eb40 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:46.057 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.076+0000 7f20fe23f640 1 --2- 172.21.15.3:0/2360692794 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f20f80b4520 0x7f20f8150eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:46.057 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.076+0000 7f20fe23f640 1 -- 172.21.15.3:0/2360692794 >> 172.21.15.3:0/2360692794 conn(0x7f20f80b6530 msgr2=0x7f20f80b6910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:46.057 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.077+0000 7f20fe23f640 1 -- 172.21.15.3:0/2360692794 shutdown_connections 2024-05-09T06:36:46.057 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.077+0000 7f20fe23f640 1 -- 172.21.15.3:0/2360692794 wait complete. 2024-05-09T06:36:46.067 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-09T06:36:46.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.195+0000 7f24620e1640 1 Processor -- start 2024-05-09T06:36:46.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.195+0000 7f24620e1640 1 -- start start 2024-05-09T06:36:46.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.196+0000 7f24620e1640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f245c06cd70 0x7f245c162a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:46.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.196+0000 7f24620e1640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f245c0d02b0 0x7f245c06d150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:46.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.196+0000 7f24620e1640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f245c09aa30 0x7f245c09aec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:46.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.196+0000 7f24620e1640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f245c161e40 con 0x7f245c06cd70 2024-05-09T06:36:46.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.196+0000 7f24620e1640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f245c162fa0 con 0x7f245c0d02b0 2024-05-09T06:36:46.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.196+0000 7f24620e1640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f245c06d690 con 0x7f245c09aa30 2024-05-09T06:36:46.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.196+0000 7f245b7fe640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f245c06cd70 0x7f245c162a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:46.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.196+0000 7f245b7fe640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f245c06cd70 0x7f245c162a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46694/0 (socket says 172.21.15.3:46694) 2024-05-09T06:36:46.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.196+0000 7f245b7fe640 1 -- 172.21.15.3:0/4213551833 learned_addr learned my addr 172.21.15.3:0/4213551833 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:36:46.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.196+0000 7f245bfff640 1 --2- 172.21.15.3:0/4213551833 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f245c09aa30 0x7f245c09aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:46.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.196+0000 7f245affd640 1 --2- 172.21.15.3:0/4213551833 >> v2:172.21.15.190:3300/0 conn(0x7f245c0d02b0 0x7f245c06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:46.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.196+0000 7f245b7fe640 1 -- 172.21.15.3:0/4213551833 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f245c09aa30 msgr2=0x7f245c09aec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:46.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.196+0000 7f245b7fe640 1 --2- 172.21.15.3:0/4213551833 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f245c09aa30 0x7f245c09aec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:46.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.196+0000 7f245b7fe640 1 -- 172.21.15.3:0/4213551833 >> v2:172.21.15.190:3300/0 conn(0x7f245c0d02b0 msgr2=0x7f245c06d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:46.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.196+0000 7f245b7fe640 1 --2- 172.21.15.3:0/4213551833 >> v2:172.21.15.190:3300/0 conn(0x7f245c0d02b0 0x7f245c06d150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:46.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.196+0000 7f245b7fe640 1 -- 172.21.15.3:0/4213551833 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f245c0d0870 con 0x7f245c06cd70 2024-05-09T06:36:46.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.197+0000 7f245b7fe640 1 --2- 172.21.15.3:0/4213551833 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f245c06cd70 0x7f245c162a60 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f245c0724b0 tx=0x7f244c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=342b564cc1b11212 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:46.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.197+0000 7f245a7fc640 1 -- 172.21.15.3:0/4213551833 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f244c037070 con 0x7f245c06cd70 2024-05-09T06:36:46.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.197+0000 7f245a7fc640 1 -- 172.21.15.3:0/4213551833 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f244c032440 con 0x7f245c06cd70 2024-05-09T06:36:46.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.197+0000 7f245a7fc640 1 -- 172.21.15.3:0/4213551833 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f244c032810 con 0x7f245c06cd70 2024-05-09T06:36:46.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.198+0000 7f24620e1640 1 -- 172.21.15.3:0/4213551833 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f245c06cd70 msgr2=0x7f245c162a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:46.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.198+0000 7f24620e1640 1 --2- 172.21.15.3:0/4213551833 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f245c06cd70 0x7f245c162a60 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f245c0724b0 tx=0x7f244c02a1e0 comp rx=0 tx=0).stop 2024-05-09T06:36:46.185 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.198+0000 7f24620e1640 1 -- 172.21.15.3:0/4213551833 shutdown_connections 2024-05-09T06:36:46.185 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.198+0000 7f24620e1640 1 --2- 172.21.15.3:0/4213551833 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f245c09aa30 0x7f245c09aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:46.185 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.198+0000 7f24620e1640 1 --2- 172.21.15.3:0/4213551833 >> v2:172.21.15.190:3300/0 conn(0x7f245c0d02b0 0x7f245c06d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:46.185 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.198+0000 7f24620e1640 1 --2- 172.21.15.3:0/4213551833 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f245c06cd70 0x7f245c162a60 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:46.185 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.198+0000 7f24620e1640 1 -- 172.21.15.3:0/4213551833 >> 172.21.15.3:0/4213551833 conn(0x7f245c0b3af0 msgr2=0x7f245c0b3f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:46.185 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.198+0000 7f24620e1640 1 -- 172.21.15.3:0/4213551833 shutdown_connections 2024-05-09T06:36:46.185 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.198+0000 7f24620e1640 1 -- 172.21.15.3:0/4213551833 wait complete. 2024-05-09T06:36:46.185 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.199+0000 7f24620e1640 1 Processor -- start 2024-05-09T06:36:46.185 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.199+0000 7f24620e1640 1 -- start start 2024-05-09T06:36:46.185 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.200+0000 7f24620e1640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f245c06cd70 0x7f245c0c7760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:46.185 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.200+0000 7f24620e1640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f245c09aa30 0x7f245c155460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:46.185 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.200+0000 7f24620e1640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f245c0d02b0 0x7f245c0c9340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:46.185 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.200+0000 7f24620e1640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f245c0d1110 con 0x7f245c06cd70 2024-05-09T06:36:46.185 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.200+0000 7f24620e1640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f245c0d0e30 con 0x7f245c0d02b0 2024-05-09T06:36:46.185 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.200+0000 7f24620e1640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f245c0d13f0 con 0x7f245c09aa30 2024-05-09T06:36:46.186 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.200+0000 7f245b7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f245c06cd70 0x7f245c0c7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:46.186 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.200+0000 7f245b7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f245c06cd70 0x7f245c0c7760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46708/0 (socket says 172.21.15.3:46708) 2024-05-09T06:36:46.186 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.200+0000 7f245b7fe640 1 -- 172.21.15.3:0/675588186 learned_addr learned my addr 172.21.15.3:0/675588186 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:36:46.186 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.200+0000 7f245affd640 1 --2- 172.21.15.3:0/675588186 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f245c09aa30 0x7f245c155460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:46.186 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.200+0000 7f245bfff640 1 --2- 172.21.15.3:0/675588186 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f245c0d02b0 0x7f245c0c9340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:46.186 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.201+0000 7f245b7fe640 1 -- 172.21.15.3:0/675588186 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f245c09aa30 msgr2=0x7f245c155460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:46.186 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.201+0000 7f245b7fe640 1 --2- 172.21.15.3:0/675588186 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f245c09aa30 0x7f245c155460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:46.186 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.201+0000 7f245b7fe640 1 -- 172.21.15.3:0/675588186 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f245c0d02b0 msgr2=0x7f245c0c9340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:46.186 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.201+0000 7f245b7fe640 1 --2- 172.21.15.3:0/675588186 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f245c0d02b0 0x7f245c0c9340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:46.186 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.201+0000 7f245b7fe640 1 -- 172.21.15.3:0/675588186 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f245c0b6ab0 con 0x7f245c06cd70 2024-05-09T06:36:46.186 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.201+0000 7f245bfff640 1 --2- 172.21.15.3:0/675588186 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f245c0d02b0 0x7f245c0c9340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:36:46.186 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.201+0000 7f245b7fe640 1 --2- 172.21.15.3:0/675588186 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f245c06cd70 0x7f245c0c7760 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f244c000c00 tx=0x7f244c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:46.186 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.201+0000 7f2443fff640 1 -- 172.21.15.3:0/675588186 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f244c041070 con 0x7f245c06cd70 2024-05-09T06:36:46.186 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.201+0000 7f2443fff640 1 -- 172.21.15.3:0/675588186 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f244c03c870 con 0x7f245c06cd70 2024-05-09T06:36:46.186 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.201+0000 7f24620e1640 1 -- 172.21.15.3:0/675588186 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f245c0c66c0 con 0x7f245c06cd70 2024-05-09T06:36:46.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.201+0000 7f2443fff640 1 -- 172.21.15.3:0/675588186 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f244c037040 con 0x7f245c06cd70 2024-05-09T06:36:46.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.201+0000 7f24620e1640 1 -- 172.21.15.3:0/675588186 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f245c0c6c80 con 0x7f245c06cd70 2024-05-09T06:36:46.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.202+0000 7f24620e1640 1 -- 172.21.15.3:0/675588186 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f245c069550 con 0x7f245c06cd70 2024-05-09T06:36:46.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.203+0000 7f2443fff640 1 -- 172.21.15.3:0/675588186 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f244c044070 con 0x7f245c06cd70 2024-05-09T06:36:46.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.203+0000 7f2443fff640 1 --2- 172.21.15.3:0/675588186 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f244806be30 0x7f244806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:46.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.203+0000 7f2443fff640 1 -- 172.21.15.3:0/675588186 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f244c0ab930 con 0x7f245c06cd70 2024-05-09T06:36:46.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.203+0000 7f245affd640 1 --2- 172.21.15.3:0/675588186 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f244806be30 0x7f244806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:46.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.204+0000 7f245affd640 1 --2- 172.21.15.3:0/675588186 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f244806be30 0x7f244806e2f0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f245c15ff00 tx=0x7f2450008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:46.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.208+0000 7f2443fff640 1 -- 172.21.15.3:0/675588186 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f244c079e40 con 0x7f245c06cd70 2024-05-09T06:36:46.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.393+0000 7f24620e1640 1 -- 172.21.15.3:0/675588186 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f245c06cb50 con 0x7f245c06cd70 2024-05-09T06:36:46.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.393+0000 7f2443fff640 1 -- 172.21.15.3:0/675588186 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7f244c07e140 con 0x7f245c06cd70 2024-05-09T06:36:46.373 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 64 2024-05-09T06:36:46.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.395+0000 7f24620e1640 1 -- 172.21.15.3:0/675588186 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f244806be30 msgr2=0x7f244806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:46.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.395+0000 7f24620e1640 1 --2- 172.21.15.3:0/675588186 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f244806be30 0x7f244806e2f0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f245c15ff00 tx=0x7f2450008040 comp rx=0 tx=0).stop 2024-05-09T06:36:46.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.395+0000 7f24620e1640 1 -- 172.21.15.3:0/675588186 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f245c06cd70 msgr2=0x7f245c0c7760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:46.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.395+0000 7f24620e1640 1 --2- 172.21.15.3:0/675588186 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f245c06cd70 0x7f245c0c7760 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f244c000c00 tx=0x7f244c0046c0 comp rx=0 tx=0).stop 2024-05-09T06:36:46.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.396+0000 7f24620e1640 1 -- 172.21.15.3:0/675588186 shutdown_connections 2024-05-09T06:36:46.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.396+0000 7f24620e1640 1 --2- 172.21.15.3:0/675588186 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f244806be30 0x7f244806e2f0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:46.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.396+0000 7f24620e1640 1 --2- 172.21.15.3:0/675588186 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f245c0d02b0 0x7f245c0c9340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:46.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.396+0000 7f24620e1640 1 --2- 172.21.15.3:0/675588186 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f245c09aa30 0x7f245c155460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:46.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.396+0000 7f24620e1640 1 --2- 172.21.15.3:0/675588186 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f245c06cd70 0x7f245c0c7760 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:46.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.396+0000 7f24620e1640 1 -- 172.21.15.3:0/675588186 >> 172.21.15.3:0/675588186 conn(0x7f245c0b3af0 msgr2=0x7f245c070790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:46.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.396+0000 7f24620e1640 1 -- 172.21.15.3:0/675588186 shutdown_connections 2024-05-09T06:36:46.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.396+0000 7f24620e1640 1 -- 172.21.15.3:0/675588186 wait complete. 2024-05-09T06:36:46.387 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T06:36:46.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.512+0000 7f6759712640 1 Processor -- start 2024-05-09T06:36:46.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.512+0000 7f6759712640 1 -- start start 2024-05-09T06:36:46.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.513+0000 7f6759712640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f675415e480 0x7f67540d1dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:46.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.513+0000 7f6759712640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f67540d4e50 con 0x7f67540d2310 2024-05-09T06:36:46.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.513+0000 7f6759712640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f67540d0270 con 0x7f67540d2ed0 2024-05-09T06:36:46.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.513+0000 7f6759712640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f67540cf1d0 con 0x7f675415e480 2024-05-09T06:36:46.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.513+0000 7f67527fc640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f67540d2310 0x7f67540d27c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:51172/0 (socket says 172.21.15.3:51172) 2024-05-09T06:36:46.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.513+0000 7f67527fc640 1 -- 172.21.15.3:0/271499975 learned_addr learned my addr 172.21.15.3:0/271499975 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-05-09T06:36:46.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.514+0000 7f6752ffd640 1 --2- 172.21.15.3:0/271499975 >> v2:172.21.15.190:3300/0 conn(0x7f675415e480 0x7f67540d1dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:46.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.514+0000 7f6751ffb640 1 -- 172.21.15.3:0/271499975 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1220567484 0 0) 0x7f673c001110 con 0x7f67540d2310 2024-05-09T06:36:46.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.514+0000 7f6751ffb640 1 -- 172.21.15.3:0/271499975 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6740003520 con 0x7f67540d2310 2024-05-09T06:36:46.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.514+0000 7f6751ffb640 1 -- 172.21.15.3:0/271499975 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3713152020 0 0) 0x7f6740003520 con 0x7f67540d2310 2024-05-09T06:36:46.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.514+0000 7f6751ffb640 1 -- 172.21.15.3:0/271499975 >> v2:172.21.15.190:3300/0 conn(0x7f675415e480 msgr2=0x7f67540d1dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:46.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.514+0000 7f6751ffb640 1 --2- 172.21.15.3:0/271499975 >> v2:172.21.15.190:3300/0 conn(0x7f675415e480 0x7f67540d1dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:46.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.514+0000 7f6751ffb640 1 -- 172.21.15.3:0/271499975 >> v1:172.21.15.190:6789/0 conn(0x7f67540d2ed0 legacy=0x7f67541683a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:46.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.514+0000 7f6751ffb640 1 -- 172.21.15.3:0/271499975 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67540cf4b0 con 0x7f67540d2310 2024-05-09T06:36:46.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.514+0000 7f6751ffb640 1 -- 172.21.15.3:0/271499975 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 563312031 0 0) 0x7f673c001840 con 0x7f67540d2310 2024-05-09T06:36:46.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.514+0000 7f6751ffb640 1 -- 172.21.15.3:0/271499975 >> v1:172.21.15.3:6789/0 conn(0x7f67540d2310 legacy=0x7f67540d27c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:46.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.514+0000 7f6751ffb640 1 --2- 172.21.15.3:0/271499975 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f67400039b0 0x7f6740003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:46.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.514+0000 7f6751ffb640 1 --2- 172.21.15.3:0/271499975 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6740004800 0x7f6740004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:46.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.514+0000 7f6751ffb640 1 --2- 172.21.15.3:0/271499975 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f67400051c0 0x7f674000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:46.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.514+0000 7f6751ffb640 1 -- 172.21.15.3:0/271499975 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f67540cf1d0 con 0x7f6740004800 2024-05-09T06:36:46.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.514+0000 7f6751ffb640 1 -- 172.21.15.3:0/271499975 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f674000c060 con 0x7f67400051c0 2024-05-09T06:36:46.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.514+0000 7f6751ffb640 1 -- 172.21.15.3:0/271499975 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f674000c260 con 0x7f67400039b0 2024-05-09T06:36:46.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.514+0000 7f67527fc640 1 --2- 172.21.15.3:0/271499975 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6740004800 0x7f6740004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:46.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.514+0000 7f6752ffd640 1 --2- 172.21.15.3:0/271499975 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f67400039b0 0x7f6740003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:46.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.514+0000 7f67537fe640 1 --2- 172.21.15.3:0/271499975 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f67400051c0 0x7f674000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:46.498 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.514+0000 7f67527fc640 1 -- 172.21.15.3:0/271499975 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f67400039b0 msgr2=0x7f6740003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:46.498 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.514+0000 7f67527fc640 1 --2- 172.21.15.3:0/271499975 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f67400039b0 0x7f6740003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:46.498 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.514+0000 7f67527fc640 1 -- 172.21.15.3:0/271499975 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f67400051c0 msgr2=0x7f674000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:46.498 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.514+0000 7f67527fc640 1 --2- 172.21.15.3:0/271499975 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f67400051c0 0x7f674000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:46.498 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.514+0000 7f67527fc640 1 -- 172.21.15.3:0/271499975 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6740003520 con 0x7f6740004800 2024-05-09T06:36:46.498 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.514+0000 7f67537fe640 1 -- 172.21.15.3:0/271499975 reap_dead start 2024-05-09T06:36:46.498 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.515+0000 7f67527fc640 1 --2- 172.21.15.3:0/271499975 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6740004800 0x7f6740004c80 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f673c00c490 tx=0x7f6740005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6974db0ac7df64f3 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:46.498 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.515+0000 7f6751ffb640 1 -- 172.21.15.3:0/271499975 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f673c00c0e0 con 0x7f6740004800 2024-05-09T06:36:46.498 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.515+0000 7f6751ffb640 1 -- 172.21.15.3:0/271499975 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f673c034070 con 0x7f6740004800 2024-05-09T06:36:46.498 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.515+0000 7f6751ffb640 1 -- 172.21.15.3:0/271499975 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f673c005d70 con 0x7f6740004800 2024-05-09T06:36:46.498 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.516+0000 7f6759712640 1 -- 172.21.15.3:0/271499975 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6740004800 msgr2=0x7f6740004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:46.498 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.516+0000 7f6759712640 1 --2- 172.21.15.3:0/271499975 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6740004800 0x7f6740004c80 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f673c00c490 tx=0x7f6740005830 comp rx=0 tx=0).stop 2024-05-09T06:36:46.498 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.516+0000 7f6759712640 1 -- 172.21.15.3:0/271499975 shutdown_connections 2024-05-09T06:36:46.498 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.516+0000 7f6759712640 1 --2- 172.21.15.3:0/271499975 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6740004800 0x7f6740004c80 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:46.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.516+0000 7f6759712640 1 -- 172.21.15.3:0/271499975 >> 172.21.15.3:0/271499975 conn(0x7f675409d4d0 msgr2=0x7f67540d4200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:46.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.516+0000 7f6759712640 1 -- 172.21.15.3:0/271499975 shutdown_connections 2024-05-09T06:36:46.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.516+0000 7f6759712640 1 -- 172.21.15.3:0/271499975 wait complete. 2024-05-09T06:36:46.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.517+0000 7f6759712640 1 Processor -- start 2024-05-09T06:36:46.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.517+0000 7f6759712640 1 -- start start 2024-05-09T06:36:46.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.518+0000 7f6759712640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6740004800 0x7f675409fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:46.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.518+0000 7f6759712640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f67541506f0 0x7f67541f4000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:46.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.518+0000 7f6759712640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f67540cce70 0x7f67540cd2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:46.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.518+0000 7f6759712640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f67540cfd50 con 0x7f67541506f0 2024-05-09T06:36:46.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.518+0000 7f6759712640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f67540cfa70 con 0x7f67540cce70 2024-05-09T06:36:46.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.518+0000 7f6759712640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f67540ceec0 con 0x7f6740004800 2024-05-09T06:36:46.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.518+0000 7f67527fc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f67541506f0 0x7f67541f4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:46.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.518+0000 7f67527fc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f67541506f0 0x7f67541f4000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46718/0 (socket says 172.21.15.3:46718) 2024-05-09T06:36:46.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.518+0000 7f67527fc640 1 -- 172.21.15.3:0/709127336 learned_addr learned my addr 172.21.15.3:0/709127336 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:36:46.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.518+0000 7f6752ffd640 1 --2- 172.21.15.3:0/709127336 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6740004800 0x7f675409fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:46.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.518+0000 7f67537fe640 1 --2- 172.21.15.3:0/709127336 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f67540cce70 0x7f67540cd2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:46.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.518+0000 7f67527fc640 1 -- 172.21.15.3:0/709127336 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6740004800 msgr2=0x7f675409fa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:46.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.518+0000 7f67527fc640 1 --2- 172.21.15.3:0/709127336 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6740004800 0x7f675409fa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:46.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.519+0000 7f67527fc640 1 -- 172.21.15.3:0/709127336 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f67540cce70 msgr2=0x7f67540cd2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:46.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.519+0000 7f67527fc640 1 --2- 172.21.15.3:0/709127336 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f67540cce70 0x7f67540cd2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:46.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.519+0000 7f67527fc640 1 -- 172.21.15.3:0/709127336 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6754150b70 con 0x7f67541506f0 2024-05-09T06:36:46.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.519+0000 7f6752ffd640 1 --2- 172.21.15.3:0/709127336 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6740004800 0x7f675409fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:36:46.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.519+0000 7f67527fc640 1 --2- 172.21.15.3:0/709127336 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f67541506f0 0x7f67541f4000 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f673c02afb0 tx=0x7f67540cf680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:46.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.519+0000 7f674b7fe640 1 -- 172.21.15.3:0/709127336 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f673c004c70 con 0x7f67541506f0 2024-05-09T06:36:46.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.519+0000 7f674b7fe640 1 -- 172.21.15.3:0/709127336 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f673c03f070 con 0x7f67541506f0 2024-05-09T06:36:46.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.519+0000 7f6759712640 1 -- 172.21.15.3:0/709127336 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67541f4870 con 0x7f67541506f0 2024-05-09T06:36:46.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.519+0000 7f674b7fe640 1 -- 172.21.15.3:0/709127336 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f673c007270 con 0x7f67541506f0 2024-05-09T06:36:46.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.519+0000 7f6759712640 1 -- 172.21.15.3:0/709127336 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67541f4e30 con 0x7f67541506f0 2024-05-09T06:36:46.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.520+0000 7f6759712640 1 -- 172.21.15.3:0/709127336 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f675404f660 con 0x7f67541506f0 2024-05-09T06:36:46.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.520+0000 7f674b7fe640 1 -- 172.21.15.3:0/709127336 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f673c007410 con 0x7f67541506f0 2024-05-09T06:36:46.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.521+0000 7f674b7fe640 1 --2- 172.21.15.3:0/709127336 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f673406be30 0x7f673406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:46.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.521+0000 7f674b7fe640 1 -- 172.21.15.3:0/709127336 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f673c0abc00 con 0x7f67541506f0 2024-05-09T06:36:46.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.521+0000 7f6752ffd640 1 --2- 172.21.15.3:0/709127336 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f673406be30 0x7f673406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:46.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.522+0000 7f6752ffd640 1 --2- 172.21.15.3:0/709127336 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f673406be30 0x7f673406e2f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f6744007440 tx=0x7f674400b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:46.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.525+0000 7f674b7fe640 1 -- 172.21.15.3:0/709127336 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f673c0b2080 con 0x7f67541506f0 2024-05-09T06:36:46.726 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.745+0000 7f6759712640 1 -- 172.21.15.3:0/709127336 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f67540d18f0 con 0x7f67541506f0 2024-05-09T06:36:46.726 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.746+0000 7f674b7fe640 1 -- 172.21.15.3:0/709127336 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5172 (secure 0 0 0) 0x7f673c048060 con 0x7f67541506f0 2024-05-09T06:36:46.726 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-09T06:36:46.727 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4272,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6834/126273055","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":126273055},{"type":"v1","addr":"172.21.15.3:6834","nonce":126273055}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4273,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6836/3885692063","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":3885692063},{"type":"v1","addr":"172.21.15.190:6836","nonce":3885692063}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4275,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/3461228756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6833","nonce":3461228756},{"type":"v1","addr":"172.21.15.3:6835","nonce":3461228756}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-09T06:36:07.221126+0000","modified":"2024-05-09T06:36:10.231051+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4276},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4276":{"gid":4276,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.190:6837/4111472052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6835","nonce":4111472052},{"type":"v1","addr":"172.21.15.190:6837","nonce":4111472052}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4276,"qdb_cluster":[4276]},"id":1}]} 2024-05-09T06:36:46.728 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 8 2024-05-09T06:36:46.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.748+0000 7f6759712640 1 -- 172.21.15.3:0/709127336 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f673406be30 msgr2=0x7f673406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:46.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.748+0000 7f6759712640 1 --2- 172.21.15.3:0/709127336 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f673406be30 0x7f673406e2f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f6744007440 tx=0x7f674400b040 comp rx=0 tx=0).stop 2024-05-09T06:36:46.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.748+0000 7f6759712640 1 -- 172.21.15.3:0/709127336 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f67541506f0 msgr2=0x7f67541f4000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:46.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.748+0000 7f6759712640 1 --2- 172.21.15.3:0/709127336 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f67541506f0 0x7f67541f4000 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f673c02afb0 tx=0x7f67540cf680 comp rx=0 tx=0).stop 2024-05-09T06:36:46.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.748+0000 7f6759712640 1 -- 172.21.15.3:0/709127336 shutdown_connections 2024-05-09T06:36:46.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.748+0000 7f6759712640 1 --2- 172.21.15.3:0/709127336 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f673406be30 0x7f673406e2f0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:46.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.748+0000 7f6759712640 1 --2- 172.21.15.3:0/709127336 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f67540cce70 0x7f67540cd2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:46.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.748+0000 7f6759712640 1 --2- 172.21.15.3:0/709127336 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f67541506f0 0x7f67541f4000 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:46.731 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.748+0000 7f6759712640 1 --2- 172.21.15.3:0/709127336 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6740004800 0x7f675409fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:46.731 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.748+0000 7f6759712640 1 -- 172.21.15.3:0/709127336 >> 172.21.15.3:0/709127336 conn(0x7f675409d4d0 msgr2=0x7f675406cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:46.731 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.748+0000 7f6759712640 1 -- 172.21.15.3:0/709127336 shutdown_connections 2024-05-09T06:36:46.731 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.748+0000 7f6759712640 1 -- 172.21.15.3:0/709127336 wait complete. 2024-05-09T06:36:46.739 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T06:36:46.844 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.864+0000 7f6a9f309640 1 Processor -- start 2024-05-09T06:36:46.844 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.865+0000 7f6a9f309640 1 -- start start 2024-05-09T06:36:46.846 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.866+0000 7f6a9f309640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f6a9815e480 0x7f6a980d1dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:46.846 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.866+0000 7f6a9f309640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f6a980d2ed0 0x7f6a981683a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:46.846 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.866+0000 7f6a9f309640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6a980d4e50 con 0x7f6a980d2310 2024-05-09T06:36:46.846 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.866+0000 7f6a9f309640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f6a980d0270 con 0x7f6a9815e480 2024-05-09T06:36:46.846 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.866+0000 7f6a9f309640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f6a980cf1d0 con 0x7f6a980d2ed0 2024-05-09T06:36:46.846 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.866+0000 7f6a9d0a5640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f6a9815e480 0x7f6a980d1dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:46.846 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.866+0000 7f6a9d0a5640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f6a9815e480 0x7f6a980d1dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46720/0 (socket says 172.21.15.3:46720) 2024-05-09T06:36:46.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.866+0000 7f6a9c8a4640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f6a980d2310 0x7f6a980d27c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:51178/0 (socket says 172.21.15.3:51178) 2024-05-09T06:36:46.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.866+0000 7f6a9d0a5640 1 -- 172.21.15.3:0/3278819717 learned_addr learned my addr 172.21.15.3:0/3278819717 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:36:46.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.866+0000 7f6a9d8a6640 1 --2- 172.21.15.3:0/3278819717 >> v2:172.21.15.190:3300/0 conn(0x7f6a980d2ed0 0x7f6a981683a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:46.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.866+0000 7f6a9d0a5640 1 -- 172.21.15.3:0/3278819717 >> v2:172.21.15.190:3300/0 conn(0x7f6a980d2ed0 msgr2=0x7f6a981683a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:46.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.866+0000 7f6a9d0a5640 1 --2- 172.21.15.3:0/3278819717 >> v2:172.21.15.190:3300/0 conn(0x7f6a980d2ed0 0x7f6a981683a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:46.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.866+0000 7f6a9d0a5640 1 -- 172.21.15.3:0/3278819717 >> v1:172.21.15.3:6789/0 conn(0x7f6a980d2310 legacy=0x7f6a980d27c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:46.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.866+0000 7f6a9d0a5640 1 -- 172.21.15.3:0/3278819717 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a980cf4b0 con 0x7f6a9815e480 2024-05-09T06:36:46.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.866+0000 7f6a8ffff640 1 -- 172.21.15.3:0/3278819717 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3536858683 0 0) 0x7f6a88001110 con 0x7f6a980d2310 2024-05-09T06:36:46.848 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.867+0000 7f6a9d0a5640 1 --2- 172.21.15.3:0/3278819717 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6a9815e480 0x7f6a980d1dd0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f6a9808e050 tx=0x7f6a9402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=977d9cdc9196942c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:46.848 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.867+0000 7f6a8ffff640 1 -- 172.21.15.3:0/3278819717 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a94037070 con 0x7f6a9815e480 2024-05-09T06:36:46.848 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.867+0000 7f6a8ffff640 1 -- 172.21.15.3:0/3278819717 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a94032440 con 0x7f6a9815e480 2024-05-09T06:36:46.848 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.867+0000 7f6a8ffff640 1 -- 172.21.15.3:0/3278819717 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a94032810 con 0x7f6a9815e480 2024-05-09T06:36:46.848 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.868+0000 7f6a9f309640 1 -- 172.21.15.3:0/3278819717 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6a9815e480 msgr2=0x7f6a980d1dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:46.848 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.868+0000 7f6a9f309640 1 --2- 172.21.15.3:0/3278819717 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6a9815e480 0x7f6a980d1dd0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f6a9808e050 tx=0x7f6a9402a270 comp rx=0 tx=0).stop 2024-05-09T06:36:46.848 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.868+0000 7f6a9f309640 1 -- 172.21.15.3:0/3278819717 shutdown_connections 2024-05-09T06:36:46.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.868+0000 7f6a9f309640 1 --2- 172.21.15.3:0/3278819717 >> v2:172.21.15.190:3300/0 conn(0x7f6a980d2ed0 0x7f6a981683a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:46.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.868+0000 7f6a9f309640 1 --2- 172.21.15.3:0/3278819717 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6a9815e480 0x7f6a980d1dd0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:46.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.868+0000 7f6a9f309640 1 -- 172.21.15.3:0/3278819717 >> 172.21.15.3:0/3278819717 conn(0x7f6a9809d4d0 msgr2=0x7f6a980d4200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:46.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.868+0000 7f6a9f309640 1 -- 172.21.15.3:0/3278819717 shutdown_connections 2024-05-09T06:36:46.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.868+0000 7f6a9f309640 1 -- 172.21.15.3:0/3278819717 wait complete. 2024-05-09T06:36:46.851 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.869+0000 7f6a9f309640 1 Processor -- start 2024-05-09T06:36:46.851 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.869+0000 7f6a9f309640 1 -- start start 2024-05-09T06:36:46.851 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.870+0000 7f6a9f309640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6a980d2310 0x7f6a981f43e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:46.851 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.870+0000 7f6a9f309640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6a980d2ed0 0x7f6a980cdaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:46.851 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.870+0000 7f6a9f309640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6a9815e480 0x7f6a980b8e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:46.851 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.870+0000 7f6a9f309640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f6a980cfd50 con 0x7f6a980d2ed0 2024-05-09T06:36:46.851 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.870+0000 7f6a9f309640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f6a980cfa70 con 0x7f6a9815e480 2024-05-09T06:36:46.851 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.870+0000 7f6a9f309640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f6a980b5ab0 con 0x7f6a980d2310 2024-05-09T06:36:46.852 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.870+0000 7f6a9c8a4640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6a980d2ed0 0x7f6a980cdaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:46.852 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.870+0000 7f6a9c8a4640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6a980d2ed0 0x7f6a980cdaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46722/0 (socket says 172.21.15.3:46722) 2024-05-09T06:36:46.852 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.870+0000 7f6a9c8a4640 1 -- 172.21.15.3:0/1585274342 learned_addr learned my addr 172.21.15.3:0/1585274342 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:36:46.852 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.870+0000 7f6a9d8a6640 1 --2- 172.21.15.3:0/1585274342 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6a9815e480 0x7f6a980b8e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:46.852 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.870+0000 7f6a9d0a5640 1 --2- 172.21.15.3:0/1585274342 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6a980d2310 0x7f6a981f43e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:46.852 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.870+0000 7f6a9c8a4640 1 -- 172.21.15.3:0/1585274342 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6a980d2310 msgr2=0x7f6a981f43e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:46.852 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.870+0000 7f6a9c8a4640 1 --2- 172.21.15.3:0/1585274342 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6a980d2310 0x7f6a981f43e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:46.852 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.870+0000 7f6a9c8a4640 1 -- 172.21.15.3:0/1585274342 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6a9815e480 msgr2=0x7f6a980b8e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:46.852 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.870+0000 7f6a9c8a4640 1 --2- 172.21.15.3:0/1585274342 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6a9815e480 0x7f6a980b8e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:46.852 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.870+0000 7f6a9c8a4640 1 -- 172.21.15.3:0/1585274342 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a980cdfe0 con 0x7f6a980d2ed0 2024-05-09T06:36:46.852 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.870+0000 7f6a9d8a6640 1 --2- 172.21.15.3:0/1585274342 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6a9815e480 0x7f6a980b8e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:36:46.852 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.870+0000 7f6a9c8a4640 1 --2- 172.21.15.3:0/1585274342 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6a980d2ed0 0x7f6a980cdaa0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f6a8800c410 tx=0x7f6a8800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:46.852 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.871+0000 7f6a8dffb640 1 -- 172.21.15.3:0/1585274342 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a880074e0 con 0x7f6a980d2ed0 2024-05-09T06:36:46.852 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.871+0000 7f6a8dffb640 1 -- 172.21.15.3:0/1585274342 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a880047e0 con 0x7f6a980d2ed0 2024-05-09T06:36:46.852 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.871+0000 7f6a9f309640 1 -- 172.21.15.3:0/1585274342 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a980cc2a0 con 0x7f6a980d2ed0 2024-05-09T06:36:46.853 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.871+0000 7f6a8dffb640 1 -- 172.21.15.3:0/1585274342 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a88004bd0 con 0x7f6a980d2ed0 2024-05-09T06:36:46.853 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.871+0000 7f6a9f309640 1 -- 172.21.15.3:0/1585274342 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a980cc860 con 0x7f6a980d2ed0 2024-05-09T06:36:46.853 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.872+0000 7f6a9f309640 1 -- 172.21.15.3:0/1585274342 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a9808e1d0 con 0x7f6a980d2ed0 2024-05-09T06:36:46.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.872+0000 7f6a8dffb640 1 -- 172.21.15.3:0/1585274342 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6a88011070 con 0x7f6a980d2ed0 2024-05-09T06:36:46.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.873+0000 7f6a8dffb640 1 --2- 172.21.15.3:0/1585274342 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f6a7c06be60 0x7f6a7c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:46.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.873+0000 7f6a8dffb640 1 -- 172.21.15.3:0/1585274342 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f6a8808aa60 con 0x7f6a980d2ed0 2024-05-09T06:36:46.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.873+0000 7f6a9d0a5640 1 --2- 172.21.15.3:0/1585274342 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f6a7c06be60 0x7f6a7c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:46.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.874+0000 7f6a9d0a5640 1 --2- 172.21.15.3:0/1585274342 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f6a7c06be60 0x7f6a7c06e320 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f6a94034000 tx=0x7f6a9402e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:46.856 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:46.877+0000 7f6a8dffb640 1 -- 172.21.15.3:0/1585274342 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f6a9808e1d0 con 0x7f6a980d2ed0 2024-05-09T06:36:47.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:47.064+0000 7f6a9f309640 1 -- 172.21.15.3:0/1585274342 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6a980ceec0 con 0x7f6a980d2ed0 2024-05-09T06:36:47.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:47.065+0000 7f6a8dffb640 1 -- 172.21.15.3:0/1585274342 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16771 (secure 0 0 0) 0x7f6a980ceec0 con 0x7f6a980d2ed0 2024-05-09T06:36:47.045 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-09T06:36:47.045 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":21,"fsid":"16ed737d-2665-431a-8ac6-ef242be3975a","created":"2024-05-09T06:35:42.541368+0000","modified":"2024-05-09T06:36:47.016655+0000","last_up_change":"2024-05-09T06:35:53.778274+0000","last_in_change":"2024-05-09T06:35:44.780586+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T06:35:54.819999+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T06:36:04.931314+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T06:36:06.200433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fae3a8db-4826-4f3c-b33a-12c09bb5df6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":1897849827},{"type":"v1","addr":"172.21.15.3:6817","nonce":1897849827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":1897849827},{"type":"v1","addr":"172.21.15.3:6819","nonce":1897849827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":1897849827},{"type":"v1","addr":"172.21.15.3:6823","nonce":1897849827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":1897849827},{"type":"v1","addr":"172.21.15.3:6821","nonce":1897849827}]},"public_addr":"172.21.15.3:6817/1897849827","cluster_addr":"172.21.15.3:6819/1897849827","heartbeat_back_addr":"172.21.15.3:6823/1897849827","heartbeat_front_addr":"172.21.15.3:6821/1897849827","state":["exists","up"]},{"osd":1,"uuid":"646e882d-4ce4-4720-8c84-f71df563a35e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":926805498},{"type":"v1","addr":"172.21.15.3:6801","nonce":926805498}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":926805498},{"type":"v1","addr":"172.21.15.3:6803","nonce":926805498}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":926805498},{"type":"v1","addr":"172.21.15.3:6807","nonce":926805498}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":926805498},{"type":"v1","addr":"172.21.15.3:6805","nonce":926805498}]},"public_addr":"172.21.15.3:6801/926805498","cluster_addr":"172.21.15.3:6803/926805498","heartbeat_back_addr":"172.21.15.3:6807/926805498","heartbeat_front_addr":"172.21.15.3:6805/926805498","state":["exists","up"]},{"osd":2,"uuid":"1abbadb5-a570-4359-9643-acff3ede35a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":801295451},{"type":"v1","addr":"172.21.15.3:6809","nonce":801295451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":801295451},{"type":"v1","addr":"172.21.15.3:6811","nonce":801295451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":801295451},{"type":"v1","addr":"172.21.15.3:6815","nonce":801295451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":801295451},{"type":"v1","addr":"172.21.15.3:6813","nonce":801295451}]},"public_addr":"172.21.15.3:6809/801295451","cluster_addr":"172.21.15.3:6811/801295451","heartbeat_back_addr":"172.21.15.3:6815/801295451","heartbeat_front_addr":"172.21.15.3:6813/801295451","state":["exists","up"]},{"osd":3,"uuid":"62ac4dc1-2371-4dd2-acd3-e2dbb11aedf5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":1903389857},{"type":"v1","addr":"172.21.15.3:6825","nonce":1903389857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":1903389857},{"type":"v1","addr":"172.21.15.3:6827","nonce":1903389857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":1903389857},{"type":"v1","addr":"172.21.15.3:6831","nonce":1903389857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":1903389857},{"type":"v1","addr":"172.21.15.3:6829","nonce":1903389857}]},"public_addr":"172.21.15.3:6825/1903389857","cluster_addr":"172.21.15.3:6827/1903389857","heartbeat_back_addr":"172.21.15.3:6831/1903389857","heartbeat_front_addr":"172.21.15.3:6829/1903389857","state":["exists","up"]},{"osd":4,"uuid":"ae808398-8326-4d7c-9319-a2a0eafffb59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":2143624576},{"type":"v1","addr":"172.21.15.190:6801","nonce":2143624576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":2143624576},{"type":"v1","addr":"172.21.15.190:6803","nonce":2143624576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":2143624576},{"type":"v1","addr":"172.21.15.190:6807","nonce":2143624576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":2143624576},{"type":"v1","addr":"172.21.15.190:6805","nonce":2143624576}]},"public_addr":"172.21.15.190:6801/2143624576","cluster_addr":"172.21.15.190:6803/2143624576","heartbeat_back_addr":"172.21.15.190:6807/2143624576","heartbeat_front_addr":"172.21.15.190:6805/2143624576","state":["exists","up"]},{"osd":5,"uuid":"918adf50-c440-48ac-9a8d-4837562d7275","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":1766923126},{"type":"v1","addr":"172.21.15.190:6814","nonce":1766923126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":1766923126},{"type":"v1","addr":"172.21.15.190:6818","nonce":1766923126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":1766923126},{"type":"v1","addr":"172.21.15.190:6823","nonce":1766923126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":1766923126},{"type":"v1","addr":"172.21.15.190:6821","nonce":1766923126}]},"public_addr":"172.21.15.190:6814/1766923126","cluster_addr":"172.21.15.190:6818/1766923126","heartbeat_back_addr":"172.21.15.190:6823/1766923126","heartbeat_front_addr":"172.21.15.190:6821/1766923126","state":["exists","up"]},{"osd":6,"uuid":"ac2882bc-2a7c-463c-8f6c-76ff49828adc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":342414248},{"type":"v1","addr":"172.21.15.190:6825","nonce":342414248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":342414248},{"type":"v1","addr":"172.21.15.190:6827","nonce":342414248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":342414248},{"type":"v1","addr":"172.21.15.190:6831","nonce":342414248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":342414248},{"type":"v1","addr":"172.21.15.190:6829","nonce":342414248}]},"public_addr":"172.21.15.190:6825/342414248","cluster_addr":"172.21.15.190:6827/342414248","heartbeat_back_addr":"172.21.15.190:6831/342414248","heartbeat_front_addr":"172.21.15.190:6829/342414248","state":["exists","up"]},{"osd":7,"uuid":"dd4e767a-4730-4c70-8bd5-3f088140a5a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":537849891},{"type":"v1","addr":"172.21.15.190:6809","nonce":537849891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":537849891},{"type":"v1","addr":"172.21.15.190:6811","nonce":537849891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6817","nonce":537849891},{"type":"v1","addr":"172.21.15.190:6819","nonce":537849891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6813","nonce":537849891},{"type":"v1","addr":"172.21.15.190:6815","nonce":537849891}]},"public_addr":"172.21.15.190:6809/537849891","cluster_addr":"172.21.15.190:6811/537849891","heartbeat_back_addr":"172.21.15.190:6819/537849891","heartbeat_front_addr":"172.21.15.190:6815/537849891","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.889784+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.974002+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.973549+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.955396+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.928471+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.902418+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.958696+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.926474+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.27","mappings":[{"from":3,"to":1}]},{"pgid":"3.2a","mappings":[{"from":5,"to":2}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"3.36","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T06:36:47.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:47.067+0000 7f6a9f309640 1 -- 172.21.15.3:0/1585274342 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f6a7c06be60 msgr2=0x7f6a7c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:47.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:47.067+0000 7f6a9f309640 1 --2- 172.21.15.3:0/1585274342 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f6a7c06be60 0x7f6a7c06e320 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f6a94034000 tx=0x7f6a9402e000 comp rx=0 tx=0).stop 2024-05-09T06:36:47.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:47.067+0000 7f6a9f309640 1 -- 172.21.15.3:0/1585274342 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6a980d2ed0 msgr2=0x7f6a980cdaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:47.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:47.067+0000 7f6a9f309640 1 --2- 172.21.15.3:0/1585274342 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6a980d2ed0 0x7f6a980cdaa0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f6a8800c410 tx=0x7f6a8800c8d0 comp rx=0 tx=0).stop 2024-05-09T06:36:47.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:47.067+0000 7f6a9f309640 1 -- 172.21.15.3:0/1585274342 shutdown_connections 2024-05-09T06:36:47.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:47.067+0000 7f6a9f309640 1 --2- 172.21.15.3:0/1585274342 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f6a7c06be60 0x7f6a7c06e320 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:47.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:47.067+0000 7f6a9f309640 1 --2- 172.21.15.3:0/1585274342 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6a9815e480 0x7f6a980b8e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:47.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:47.067+0000 7f6a9f309640 1 --2- 172.21.15.3:0/1585274342 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6a980d2ed0 0x7f6a980cdaa0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:47.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:47.067+0000 7f6a9f309640 1 --2- 172.21.15.3:0/1585274342 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6a980d2310 0x7f6a981f43e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:47.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:47.067+0000 7f6a9f309640 1 -- 172.21.15.3:0/1585274342 >> 172.21.15.3:0/1585274342 conn(0x7f6a9809d4d0 msgr2=0x7f6a9815d6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:47.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:47.068+0000 7f6a9f309640 1 -- 172.21.15.3:0/1585274342 shutdown_connections 2024-05-09T06:36:47.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:47.068+0000 7f6a9f309640 1 -- 172.21.15.3:0/1585274342 wait complete. 2024-05-09T06:36:47.058 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-05-09T06:36:47.059 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T06:36:47.163 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:47.183+0000 7f5d4829a640 1 Processor -- start 2024-05-09T06:36:47.163 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:47.183+0000 7f5d4829a640 1 -- start start 2024-05-09T06:36:47.164 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:47.184+0000 7f5d4829a640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f5d4013e0c0 0x7f5d400cf720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:47.164 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:47.184+0000 7f5d4829a640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5d400cc870 0x7f5d400cccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:47.165 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:47.184+0000 7f5d4829a640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f5d401408f0 0x7f5d40140d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:47.165 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:47.184+0000 7f5d4829a640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f5d4013e4a0 con 0x7f5d401408f0 2024-05-09T06:36:47.165 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:47.184+0000 7f5d4829a640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f5d400cd210 con 0x7f5d4013e0c0 2024-05-09T06:36:47.165 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:47.184+0000 7f5d4829a640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f5d400ce960 con 0x7f5d400cc870 2024-05-09T06:36:47.165 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:47.184+0000 7f5d46837640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f5d401408f0 0x7f5d40140d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:47.165 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:47.184+0000 7f5d46837640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f5d401408f0 0x7f5d40140d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46724/0 (socket says 172.21.15.3:46724) 2024-05-09T06:36:47.165 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:47.184+0000 7f5d46837640 1 -- 172.21.15.3:0/1369550915 learned_addr learned my addr 172.21.15.3:0/1369550915 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:36:47.165 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:47.184+0000 7f5d46036640 1 --2- 172.21.15.3:0/1369550915 >> v2:172.21.15.190:3300/0 conn(0x7f5d4013e0c0 0x7f5d400cf720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:47.165 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:47.184+0000 7f5d45835640 1 --2- 172.21.15.3:0/1369550915 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5d400cc870 0x7f5d400cccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:47.165 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:47.185+0000 7f5d46837640 1 -- 172.21.15.3:0/1369550915 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5d400cc870 msgr2=0x7f5d400cccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:47.165 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:47.185+0000 7f5d46837640 1 --2- 172.21.15.3:0/1369550915 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5d400cc870 0x7f5d400cccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:47.165 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:47.185+0000 7f5d46837640 1 -- 172.21.15.3:0/1369550915 >> v2:172.21.15.190:3300/0 conn(0x7f5d4013e0c0 msgr2=0x7f5d400cf720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:47.165 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:47.185+0000 7f5d46837640 1 --2- 172.21.15.3:0/1369550915 >> v2:172.21.15.190:3300/0 conn(0x7f5d4013e0c0 0x7f5d400cf720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:47.165 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:47.185+0000 7f5d46837640 1 -- 172.21.15.3:0/1369550915 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d400cec40 con 0x7f5d401408f0 2024-05-09T06:36:47.167 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:47.185+0000 7f5d46837640 1 --2- 172.21.15.3:0/1369550915 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5d401408f0 0x7f5d40140d40 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f5d4006f010 tx=0x7f5d3802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea25bd23feb5dab8 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:47.167 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:47.186+0000 7f5d45034640 1 -- 172.21.15.3:0/1369550915 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d38037070 con 0x7f5d401408f0 2024-05-09T06:36:47.167 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:47.186+0000 7f5d45034640 1 -- 172.21.15.3:0/1369550915 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d38032440 con 0x7f5d401408f0 2024-05-09T06:36:47.167 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:47.186+0000 7f5d45034640 1 -- 172.21.15.3:0/1369550915 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d38032810 con 0x7f5d401408f0 2024-05-09T06:36:47.167 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:47.186+0000 7f5d4829a640 1 -- 172.21.15.3:0/1369550915 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5d401408f0 msgr2=0x7f5d40140d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:47.167 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:47.186+0000 7f5d4829a640 1 --2- 172.21.15.3:0/1369550915 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5d401408f0 0x7f5d40140d40 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f5d4006f010 tx=0x7f5d3802a1e0 comp rx=0 tx=0).stop 2024-05-09T06:36:47.167 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:47.186+0000 7f5d4829a640 1 -- 172.21.15.3:0/1369550915 shutdown_connections 2024-05-09T06:36:47.167 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:47.186+0000 7f5d4829a640 1 --2- 172.21.15.3:0/1369550915 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5d401408f0 0x7f5d40140d40 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:47.167 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:47.186+0000 7f5d4829a640 1 --2- 172.21.15.3:0/1369550915 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5d400cc870 0x7f5d400cccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:47.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:47.186+0000 7f5d4829a640 1 --2- 172.21.15.3:0/1369550915 >> v2:172.21.15.190:3300/0 conn(0x7f5d4013e0c0 0x7f5d400cf720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:47.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:47.186+0000 7f5d4829a640 1 -- 172.21.15.3:0/1369550915 >> 172.21.15.3:0/1369550915 conn(0x7f5d4006b430 msgr2=0x7f5d4006b810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:47.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:47.186+0000 7f5d4829a640 1 -- 172.21.15.3:0/1369550915 shutdown_connections 2024-05-09T06:36:47.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:47.186+0000 7f5d4829a640 1 -- 172.21.15.3:0/1369550915 wait complete. 2024-05-09T06:36:47.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:47.187+0000 7f5d4829a640 1 Processor -- start 2024-05-09T06:36:47.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:47.187+0000 7f5d4829a640 1 -- start start 2024-05-09T06:36:47.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:47.188+0000 7f5d4829a640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5d400cc870 0x7f5d4013d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:47.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:47.188+0000 7f5d4829a640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5d4013e0c0 0x7f5d400b4b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:47.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:47.188+0000 7f5d4829a640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5d401408f0 0x7f5d400b5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:47.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:47.188+0000 7f5d4829a640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f5d40141660 con 0x7f5d400cc870 2024-05-09T06:36:47.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:47.188+0000 7f5d4829a640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f5d400cf200 con 0x7f5d401408f0 2024-05-09T06:36:47.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:47.188+0000 7f5d4829a640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f5d40141280 con 0x7f5d4013e0c0 2024-05-09T06:36:47.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:47.188+0000 7f5d46036640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5d400cc870 0x7f5d4013d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:47.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:47.188+0000 7f5d46036640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5d400cc870 0x7f5d4013d730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46730/0 (socket says 172.21.15.3:46730) 2024-05-09T06:36:47.172 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:47.188+0000 7f5d46036640 1 -- 172.21.15.3:0/2366184056 learned_addr learned my addr 172.21.15.3:0/2366184056 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:36:47.172 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:47.189+0000 7f5d45835640 1 --2- 172.21.15.3:0/2366184056 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5d4013e0c0 0x7f5d400b4b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:47.172 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:47.189+0000 7f5d46837640 1 --2- 172.21.15.3:0/2366184056 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5d401408f0 0x7f5d400b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:47.172 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:47.189+0000 7f5d46036640 1 -- 172.21.15.3:0/2366184056 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5d4013e0c0 msgr2=0x7f5d400b4b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:47.172 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:47.189+0000 7f5d46036640 1 --2- 172.21.15.3:0/2366184056 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5d4013e0c0 0x7f5d400b4b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:47.172 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:47.189+0000 7f5d46036640 1 -- 172.21.15.3:0/2366184056 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5d401408f0 msgr2=0x7f5d400b5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:47.172 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:47.189+0000 7f5d46036640 1 --2- 172.21.15.3:0/2366184056 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5d401408f0 0x7f5d400b5050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:47.172 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:47.189+0000 7f5d46036640 1 -- 172.21.15.3:0/2366184056 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d4013dc70 con 0x7f5d400cc870 2024-05-09T06:36:47.172 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:47.189+0000 7f5d45835640 1 --2- 172.21.15.3:0/2366184056 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5d4013e0c0 0x7f5d400b4b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:36:47.172 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:47.189+0000 7f5d46036640 1 --2- 172.21.15.3:0/2366184056 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5d400cc870 0x7f5d4013d730 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f5d3000efc0 tx=0x7f5d3000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:47.172 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:47.189+0000 7f5d3effd640 1 -- 172.21.15.3:0/2366184056 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d30018020 con 0x7f5d400cc870 2024-05-09T06:36:47.172 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:47.189+0000 7f5d3effd640 1 -- 172.21.15.3:0/2366184056 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d30014480 con 0x7f5d400cc870 2024-05-09T06:36:47.173 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:47.189+0000 7f5d4829a640 1 -- 172.21.15.3:0/2366184056 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d401f7550 con 0x7f5d400cc870 2024-05-09T06:36:47.173 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:47.189+0000 7f5d3effd640 1 -- 172.21.15.3:0/2366184056 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d30014850 con 0x7f5d400cc870 2024-05-09T06:36:47.173 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:47.189+0000 7f5d4829a640 1 -- 172.21.15.3:0/2366184056 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d401f7b10 con 0x7f5d400cc870 2024-05-09T06:36:47.173 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:47.190+0000 7f5d4829a640 1 -- 172.21.15.3:0/2366184056 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d4004f660 con 0x7f5d400cc870 2024-05-09T06:36:47.173 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:47.190+0000 7f5d3effd640 1 -- 172.21.15.3:0/2366184056 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5d30005d10 con 0x7f5d400cc870 2024-05-09T06:36:47.173 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:47.191+0000 7f5d3effd640 1 --2- 172.21.15.3:0/2366184056 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f5d2c06be60 0x7f5d2c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:47.173 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:47.191+0000 7f5d3effd640 1 -- 172.21.15.3:0/2366184056 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5825+0+0 (secure 0 0 0) 0x7f5d3008bc30 con 0x7f5d400cc870 2024-05-09T06:36:47.173 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:47.191+0000 7f5d45835640 1 --2- 172.21.15.3:0/2366184056 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f5d2c06be60 0x7f5d2c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:47.173 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:47.192+0000 7f5d45835640 1 --2- 172.21.15.3:0/2366184056 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f5d2c06be60 0x7f5d2c06e320 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f5d401581b0 tx=0x7f5d34008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:47.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:47.195+0000 7f5d3effd640 1 -- 172.21.15.3:0/2366184056 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f5d30022030 con 0x7f5d400cc870 2024-05-09T06:36:47.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:47.417+0000 7f5d4829a640 1 -- 172.21.15.3:0/2366184056 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5d400730a0 con 0x7f5d400cc870 2024-05-09T06:36:47.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:47.418+0000 7f5d3effd640 1 -- 172.21.15.3:0/2366184056 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5172 (secure 0 0 0) 0x7f5d4013dc70 con 0x7f5d400cc870 2024-05-09T06:36:47.398 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-09T06:36:47.398 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4272,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6834/126273055","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":126273055},{"type":"v1","addr":"172.21.15.3:6834","nonce":126273055}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4273,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6836/3885692063","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":3885692063},{"type":"v1","addr":"172.21.15.190:6836","nonce":3885692063}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4275,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/3461228756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6833","nonce":3461228756},{"type":"v1","addr":"172.21.15.3:6835","nonce":3461228756}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-09T06:36:07.221126+0000","modified":"2024-05-09T06:36:10.231051+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4276},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4276":{"gid":4276,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.190:6837/4111472052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6835","nonce":4111472052},{"type":"v1","addr":"172.21.15.190:6837","nonce":4111472052}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4276,"qdb_cluster":[4276]},"id":1}]} 2024-05-09T06:36:47.398 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 8 2024-05-09T06:36:47.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:47.420+0000 7f5d4829a640 1 -- 172.21.15.3:0/2366184056 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f5d2c06be60 msgr2=0x7f5d2c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:47.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:47.420+0000 7f5d4829a640 1 --2- 172.21.15.3:0/2366184056 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f5d2c06be60 0x7f5d2c06e320 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f5d401581b0 tx=0x7f5d34008040 comp rx=0 tx=0).stop 2024-05-09T06:36:47.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:47.420+0000 7f5d4829a640 1 -- 172.21.15.3:0/2366184056 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5d400cc870 msgr2=0x7f5d4013d730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:47.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:47.420+0000 7f5d4829a640 1 --2- 172.21.15.3:0/2366184056 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5d400cc870 0x7f5d4013d730 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f5d3000efc0 tx=0x7f5d3000c490 comp rx=0 tx=0).stop 2024-05-09T06:36:47.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:47.420+0000 7f5d4829a640 1 -- 172.21.15.3:0/2366184056 shutdown_connections 2024-05-09T06:36:47.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:47.420+0000 7f5d4829a640 1 --2- 172.21.15.3:0/2366184056 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f5d2c06be60 0x7f5d2c06e320 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:47.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:47.420+0000 7f5d4829a640 1 --2- 172.21.15.3:0/2366184056 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5d401408f0 0x7f5d400b5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:47.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:47.420+0000 7f5d4829a640 1 --2- 172.21.15.3:0/2366184056 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5d4013e0c0 0x7f5d400b4b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:47.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:47.420+0000 7f5d4829a640 1 --2- 172.21.15.3:0/2366184056 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5d400cc870 0x7f5d4013d730 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:47.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:47.420+0000 7f5d4829a640 1 -- 172.21.15.3:0/2366184056 >> 172.21.15.3:0/2366184056 conn(0x7f5d4006b430 msgr2=0x7f5d4015a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:47.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:47.420+0000 7f5d4829a640 1 -- 172.21.15.3:0/2366184056 shutdown_connections 2024-05-09T06:36:47.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:47.420+0000 7f5d4829a640 1 -- 172.21.15.3:0/2366184056 wait complete. 2024-05-09T06:36:47.411 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T06:36:47.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:47.538+0000 7f12c2015640 1 Processor -- start 2024-05-09T06:36:47.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:47.538+0000 7f12c2015640 1 -- start start 2024-05-09T06:36:47.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:47.539+0000 7f12c2015640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f12bc15e480 0x7f12bc0d1dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:47.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:47.539+0000 7f12c2015640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f12bc0d2ed0 0x7f12bc1683a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:47.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:47.539+0000 7f12c2015640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f12bc0d4e50 con 0x7f12bc0d2310 2024-05-09T06:36:47.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:47.539+0000 7f12c2015640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f12bc0d0270 con 0x7f12bc15e480 2024-05-09T06:36:47.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:47.539+0000 7f12c2015640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f12bc0cf1d0 con 0x7f12bc0d2ed0 2024-05-09T06:36:47.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:47.539+0000 7f12bb7fe640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f12bc15e480 0x7f12bc0d1dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:47.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:47.539+0000 7f12b3fff640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f12bc0d2310 0x7f12bc0d27c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:51184/0 (socket says 172.21.15.3:51184) 2024-05-09T06:36:47.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:47.539+0000 7f12b3fff640 1 -- 172.21.15.3:0/2451690969 learned_addr learned my addr 172.21.15.3:0/2451690969 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-05-09T06:36:47.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:47.540+0000 7f12bbfff640 1 --2- 172.21.15.3:0/2451690969 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f12bc0d2ed0 0x7f12bc1683a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:47.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:47.540+0000 7f12bb7fe640 1 -- 172.21.15.3:0/2451690969 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f12bc0d2ed0 msgr2=0x7f12bc1683a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:47.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:47.540+0000 7f12bb7fe640 1 --2- 172.21.15.3:0/2451690969 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f12bc0d2ed0 0x7f12bc1683a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:47.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:47.540+0000 7f12bb7fe640 1 -- 172.21.15.3:0/2451690969 >> v1:172.21.15.3:6789/0 conn(0x7f12bc0d2310 legacy=0x7f12bc0d27c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:47.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:47.540+0000 7f12bb7fe640 1 -- 172.21.15.3:0/2451690969 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12bc0cf4b0 con 0x7f12bc15e480 2024-05-09T06:36:47.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:47.540+0000 7f12baffd640 1 -- 172.21.15.3:0/2451690969 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 457707096 0 0) 0x7f12a4001110 con 0x7f12bc0d2310 2024-05-09T06:36:47.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:47.540+0000 7f12bb7fe640 1 --2- 172.21.15.3:0/2451690969 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f12bc15e480 0x7f12bc0d1dd0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f12ac0097a0 tx=0x7f12ac02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aaf95b0205a91125 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:47.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:47.540+0000 7f12baffd640 1 -- 172.21.15.3:0/2451690969 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12ac037070 con 0x7f12bc15e480 2024-05-09T06:36:47.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:47.541+0000 7f12baffd640 1 -- 172.21.15.3:0/2451690969 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f12ac032440 con 0x7f12bc15e480 2024-05-09T06:36:47.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:47.541+0000 7f12baffd640 1 -- 172.21.15.3:0/2451690969 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12ac032810 con 0x7f12bc15e480 2024-05-09T06:36:47.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:47.541+0000 7f12c2015640 1 -- 172.21.15.3:0/2451690969 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f12bc15e480 msgr2=0x7f12bc0d1dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:47.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:47.541+0000 7f12c2015640 1 --2- 172.21.15.3:0/2451690969 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f12bc15e480 0x7f12bc0d1dd0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f12ac0097a0 tx=0x7f12ac02a180 comp rx=0 tx=0).stop 2024-05-09T06:36:47.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:47.541+0000 7f12c2015640 1 -- 172.21.15.3:0/2451690969 shutdown_connections 2024-05-09T06:36:47.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:47.541+0000 7f12c2015640 1 --2- 172.21.15.3:0/2451690969 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f12bc0d2ed0 0x7f12bc1683a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:47.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:47.541+0000 7f12c2015640 1 --2- 172.21.15.3:0/2451690969 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f12bc15e480 0x7f12bc0d1dd0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:47.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:47.541+0000 7f12c2015640 1 -- 172.21.15.3:0/2451690969 >> 172.21.15.3:0/2451690969 conn(0x7f12bc09d4d0 msgr2=0x7f12bc0d4200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:47.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:47.541+0000 7f12c2015640 1 -- 172.21.15.3:0/2451690969 shutdown_connections 2024-05-09T06:36:47.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:47.541+0000 7f12c2015640 1 -- 172.21.15.3:0/2451690969 wait complete. 2024-05-09T06:36:47.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:47.542+0000 7f12c2015640 1 Processor -- start 2024-05-09T06:36:47.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:47.543+0000 7f12c2015640 1 -- start start 2024-05-09T06:36:47.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:47.543+0000 7f12c2015640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f12bc0d2310 0x7f12bc1f51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:47.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:47.543+0000 7f12c2015640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f12bc0d2ed0 0x7f12bc06d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:47.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:47.543+0000 7f12c2015640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f12bc15e480 0x7f12bc0b6e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:47.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:47.543+0000 7f12c2015640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f12bc0cfd50 con 0x7f12bc0d2310 2024-05-09T06:36:47.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:47.543+0000 7f12c2015640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f12bc0cfa70 con 0x7f12bc0d2ed0 2024-05-09T06:36:47.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:47.543+0000 7f12c2015640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f12bc0b5ab0 con 0x7f12bc15e480 2024-05-09T06:36:47.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:47.543+0000 7f12bb7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f12bc0d2310 0x7f12bc1f51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:47.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:47.543+0000 7f12bb7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f12bc0d2310 0x7f12bc1f51b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46756/0 (socket says 172.21.15.3:46756) 2024-05-09T06:36:47.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:47.543+0000 7f12bb7fe640 1 -- 172.21.15.3:0/461326390 learned_addr learned my addr 172.21.15.3:0/461326390 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:36:47.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:47.543+0000 7f12bbfff640 1 --2- 172.21.15.3:0/461326390 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f12bc15e480 0x7f12bc0b6e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:47.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:47.543+0000 7f12b3fff640 1 --2- 172.21.15.3:0/461326390 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f12bc0d2ed0 0x7f12bc06d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:47.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:47.544+0000 7f12bb7fe640 1 -- 172.21.15.3:0/461326390 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f12bc15e480 msgr2=0x7f12bc0b6e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:47.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:47.544+0000 7f12bb7fe640 1 --2- 172.21.15.3:0/461326390 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f12bc15e480 0x7f12bc0b6e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:47.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:47.544+0000 7f12bb7fe640 1 -- 172.21.15.3:0/461326390 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f12bc0d2ed0 msgr2=0x7f12bc06d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:47.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:47.544+0000 7f12bb7fe640 1 --2- 172.21.15.3:0/461326390 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f12bc0d2ed0 0x7f12bc06d210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:47.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:47.544+0000 7f12bb7fe640 1 -- 172.21.15.3:0/461326390 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12bc0b7a30 con 0x7f12bc0d2310 2024-05-09T06:36:47.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:47.544+0000 7f12b3fff640 1 --2- 172.21.15.3:0/461326390 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f12bc0d2ed0 0x7f12bc06d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:36:47.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:47.544+0000 7f12bb7fe640 1 --2- 172.21.15.3:0/461326390 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f12bc0d2310 0x7f12bc1f51b0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f12ac000c00 tx=0x7f12ac0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:47.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:47.544+0000 7f12b8ff9640 1 -- 172.21.15.3:0/461326390 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12ac037040 con 0x7f12bc0d2310 2024-05-09T06:36:47.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:47.544+0000 7f12b8ff9640 1 -- 172.21.15.3:0/461326390 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f12ac042070 con 0x7f12bc0d2310 2024-05-09T06:36:47.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:47.544+0000 7f12b8ff9640 1 -- 172.21.15.3:0/461326390 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12ac030bb0 con 0x7f12bc0d2310 2024-05-09T06:36:47.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:47.544+0000 7f12c2015640 1 -- 172.21.15.3:0/461326390 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12bc0b7d10 con 0x7f12bc0d2310 2024-05-09T06:36:47.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:47.544+0000 7f12c2015640 1 -- 172.21.15.3:0/461326390 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12bc1f4c90 con 0x7f12bc0d2310 2024-05-09T06:36:47.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:47.545+0000 7f12c2015640 1 -- 172.21.15.3:0/461326390 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12bc0d3860 con 0x7f12bc0d2310 2024-05-09T06:36:47.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:47.546+0000 7f12b8ff9640 1 -- 172.21.15.3:0/461326390 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f12ac047460 con 0x7f12bc0d2310 2024-05-09T06:36:47.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:47.546+0000 7f12b8ff9640 1 --2- 172.21.15.3:0/461326390 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f129c06be30 0x7f129c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:47.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:47.546+0000 7f12b8ff9640 1 -- 172.21.15.3:0/461326390 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5825+0+0 (secure 0 0 0) 0x7f12ac0abb00 con 0x7f12bc0d2310 2024-05-09T06:36:47.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:47.546+0000 7f12b3fff640 1 --2- 172.21.15.3:0/461326390 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f129c06be30 0x7f129c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:47.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:47.547+0000 7f12b3fff640 1 --2- 172.21.15.3:0/461326390 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f129c06be30 0x7f129c06e2f0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f12bc15be50 tx=0x7f12a4007ca0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:47.530 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:47.551+0000 7f12b8ff9640 1 -- 172.21.15.3:0/461326390 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f12bc0d3860 con 0x7f12bc0d2310 2024-05-09T06:36:47.751 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:47.771+0000 7f12c2015640 1 -- 172.21.15.3:0/461326390 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f12bc04f660 con 0x7f12bc0d2310 2024-05-09T06:36:47.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:47.772+0000 7f12b8ff9640 1 -- 172.21.15.3:0/461326390 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5172 (secure 0 0 0) 0x7f12ac032e20 con 0x7f12bc0d2310 2024-05-09T06:36:47.752 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-09T06:36:47.752 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4272,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6834/126273055","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":126273055},{"type":"v1","addr":"172.21.15.3:6834","nonce":126273055}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4273,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6836/3885692063","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":3885692063},{"type":"v1","addr":"172.21.15.190:6836","nonce":3885692063}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4275,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/3461228756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6833","nonce":3461228756},{"type":"v1","addr":"172.21.15.3:6835","nonce":3461228756}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-09T06:36:07.221126+0000","modified":"2024-05-09T06:36:10.231051+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4276},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4276":{"gid":4276,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.190:6837/4111472052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6835","nonce":4111472052},{"type":"v1","addr":"172.21.15.190:6837","nonce":4111472052}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4276,"qdb_cluster":[4276]},"id":1}]} 2024-05-09T06:36:47.752 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 8 2024-05-09T06:36:47.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:47.774+0000 7f12c2015640 1 -- 172.21.15.3:0/461326390 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f129c06be30 msgr2=0x7f129c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:47.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:47.774+0000 7f12c2015640 1 --2- 172.21.15.3:0/461326390 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f129c06be30 0x7f129c06e2f0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f12bc15be50 tx=0x7f12a4007ca0 comp rx=0 tx=0).stop 2024-05-09T06:36:47.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:47.774+0000 7f12c2015640 1 -- 172.21.15.3:0/461326390 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f12bc0d2310 msgr2=0x7f12bc1f51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:47.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:47.774+0000 7f12c2015640 1 --2- 172.21.15.3:0/461326390 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f12bc0d2310 0x7f12bc1f51b0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f12ac000c00 tx=0x7f12ac0046c0 comp rx=0 tx=0).stop 2024-05-09T06:36:47.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:47.774+0000 7f12c2015640 1 -- 172.21.15.3:0/461326390 shutdown_connections 2024-05-09T06:36:47.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:47.774+0000 7f12c2015640 1 --2- 172.21.15.3:0/461326390 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f129c06be30 0x7f129c06e2f0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:47.755 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:47.774+0000 7f12c2015640 1 --2- 172.21.15.3:0/461326390 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f12bc15e480 0x7f12bc0b6e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:47.755 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:47.774+0000 7f12c2015640 1 --2- 172.21.15.3:0/461326390 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f12bc0d2ed0 0x7f12bc06d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:47.755 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:47.774+0000 7f12c2015640 1 --2- 172.21.15.3:0/461326390 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f12bc0d2310 0x7f12bc1f51b0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:47.755 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:47.774+0000 7f12c2015640 1 -- 172.21.15.3:0/461326390 >> 172.21.15.3:0/461326390 conn(0x7f12bc09d4d0 msgr2=0x7f12bc0b5050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:47.755 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:47.774+0000 7f12c2015640 1 -- 172.21.15.3:0/461326390 shutdown_connections 2024-05-09T06:36:47.755 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:47.774+0000 7f12c2015640 1 -- 172.21.15.3:0/461326390 wait complete. 2024-05-09T06:36:47.765 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T06:36:47.870 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:47.891+0000 7f09f73d5640 1 Processor -- start 2024-05-09T06:36:47.871 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:47.891+0000 7f09f73d5640 1 -- start start 2024-05-09T06:36:47.872 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:47.892+0000 7f09f73d5640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f09f013e0c0 0x7f09f00cf720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:47.872 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:47.892+0000 7f09f73d5640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f09f0141000 0x7f09f00ccd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:47.872 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:47.892+0000 7f09f73d5640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f09f013e4a0 con 0x7f09f00cc870 2024-05-09T06:36:47.872 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:47.892+0000 7f09f73d5640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f09f0141430 con 0x7f09f013e0c0 2024-05-09T06:36:47.872 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:47.892+0000 7f09f73d5640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f09f00cd240 con 0x7f09f0141000 2024-05-09T06:36:47.872 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:47.892+0000 7f09f5171640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f09f013e0c0 0x7f09f00cf720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:47.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:47.892+0000 7f09f4970640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f09f00cc870 0x7f09f01408f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:51188/0 (socket says 172.21.15.3:51188) 2024-05-09T06:36:47.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:47.892+0000 7f09f4970640 1 -- 172.21.15.3:0/611328204 learned_addr learned my addr 172.21.15.3:0/611328204 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-05-09T06:36:47.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:47.892+0000 7f09f5972640 1 --2- 172.21.15.3:0/611328204 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f09f0141000 0x7f09f00ccd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:47.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:47.892+0000 7f09f5171640 1 -- 172.21.15.3:0/611328204 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f09f0141000 msgr2=0x7f09f00ccd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:47.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:47.892+0000 7f09f5171640 1 --2- 172.21.15.3:0/611328204 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f09f0141000 0x7f09f00ccd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:47.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:47.892+0000 7f09f5171640 1 -- 172.21.15.3:0/611328204 >> v1:172.21.15.3:6789/0 conn(0x7f09f00cc870 legacy=0x7f09f01408f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:47.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:47.892+0000 7f09e7fff640 1 -- 172.21.15.3:0/611328204 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1844721695 0 0) 0x7f09e0001110 con 0x7f09f00cc870 2024-05-09T06:36:47.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:47.892+0000 7f09f5171640 1 -- 172.21.15.3:0/611328204 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09f00ce960 con 0x7f09f013e0c0 2024-05-09T06:36:47.875 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:47.893+0000 7f09f5171640 1 --2- 172.21.15.3:0/611328204 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f09f013e0c0 0x7f09f00cf720 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f09ec0097a0 tx=0x7f09ec02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bafae9ddf2370128 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:47.875 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:47.893+0000 7f09e7fff640 1 -- 172.21.15.3:0/611328204 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09ec037070 con 0x7f09f013e0c0 2024-05-09T06:36:47.875 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:47.893+0000 7f09e7fff640 1 -- 172.21.15.3:0/611328204 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f09ec032440 con 0x7f09f013e0c0 2024-05-09T06:36:47.875 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:47.893+0000 7f09e7fff640 1 -- 172.21.15.3:0/611328204 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09ec032810 con 0x7f09f013e0c0 2024-05-09T06:36:47.875 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:47.894+0000 7f09f73d5640 1 -- 172.21.15.3:0/611328204 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f09f013e0c0 msgr2=0x7f09f00cf720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:47.875 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:47.894+0000 7f09f73d5640 1 --2- 172.21.15.3:0/611328204 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f09f013e0c0 0x7f09f00cf720 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f09ec0097a0 tx=0x7f09ec02a180 comp rx=0 tx=0).stop 2024-05-09T06:36:47.875 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:47.894+0000 7f09f73d5640 1 -- 172.21.15.3:0/611328204 shutdown_connections 2024-05-09T06:36:47.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:47.894+0000 7f09f73d5640 1 --2- 172.21.15.3:0/611328204 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f09f0141000 0x7f09f00ccd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:47.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:47.894+0000 7f09f73d5640 1 --2- 172.21.15.3:0/611328204 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f09f013e0c0 0x7f09f00cf720 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:47.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:47.894+0000 7f09f73d5640 1 -- 172.21.15.3:0/611328204 >> 172.21.15.3:0/611328204 conn(0x7f09f006b430 msgr2=0x7f09f006b810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:47.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:47.894+0000 7f09f73d5640 1 -- 172.21.15.3:0/611328204 shutdown_connections 2024-05-09T06:36:47.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:47.894+0000 7f09f73d5640 1 -- 172.21.15.3:0/611328204 wait complete. 2024-05-09T06:36:47.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:47.895+0000 7f09f73d5640 1 Processor -- start 2024-05-09T06:36:47.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:47.895+0000 7f09f73d5640 1 -- start start 2024-05-09T06:36:47.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:47.896+0000 7f09f73d5640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f09f00cc870 0x7f09f01f0b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:47.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:47.896+0000 7f09f73d5640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f09f013e0c0 0x7f09f0150bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:47.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:47.896+0000 7f09f73d5640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f09f0141000 0x7f09f01510f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:47.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:47.896+0000 7f09f73d5640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f09f00cf200 con 0x7f09f013e0c0 2024-05-09T06:36:47.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:47.896+0000 7f09f73d5640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f09f00cef20 con 0x7f09f0141000 2024-05-09T06:36:47.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:47.896+0000 7f09f73d5640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f09f0141660 con 0x7f09f00cc870 2024-05-09T06:36:47.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:47.896+0000 7f09f4970640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f09f013e0c0 0x7f09f0150bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:47.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:47.896+0000 7f09f4970640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f09f013e0c0 0x7f09f0150bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46776/0 (socket says 172.21.15.3:46776) 2024-05-09T06:36:47.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:47.896+0000 7f09f4970640 1 -- 172.21.15.3:0/1434202027 learned_addr learned my addr 172.21.15.3:0/1434202027 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:36:47.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:47.896+0000 7f09f5171640 1 --2- 172.21.15.3:0/1434202027 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f09f00cc870 0x7f09f01f0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:47.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:47.896+0000 7f09f5972640 1 --2- 172.21.15.3:0/1434202027 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f09f0141000 0x7f09f01510f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:47.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:47.896+0000 7f09f4970640 1 -- 172.21.15.3:0/1434202027 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f09f00cc870 msgr2=0x7f09f01f0b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:47.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:47.896+0000 7f09f4970640 1 --2- 172.21.15.3:0/1434202027 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f09f00cc870 0x7f09f01f0b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:47.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:47.896+0000 7f09f4970640 1 -- 172.21.15.3:0/1434202027 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f09f0141000 msgr2=0x7f09f01510f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:47.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:47.896+0000 7f09f4970640 1 --2- 172.21.15.3:0/1434202027 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f09f0141000 0x7f09f01510f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:47.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:47.896+0000 7f09f4970640 1 -- 172.21.15.3:0/1434202027 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09f00b0270 con 0x7f09f013e0c0 2024-05-09T06:36:47.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:47.896+0000 7f09f5171640 1 --2- 172.21.15.3:0/1434202027 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f09f00cc870 0x7f09f01f0b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:36:47.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:47.896+0000 7f09f5972640 1 --2- 172.21.15.3:0/1434202027 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f09f0141000 0x7f09f01510f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:36:47.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:47.896+0000 7f09f4970640 1 --2- 172.21.15.3:0/1434202027 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f09f013e0c0 0x7f09f0150bb0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f09e000c780 tx=0x7f09e000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:47.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:47.896+0000 7f09e5ffb640 1 -- 172.21.15.3:0/1434202027 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09e0018070 con 0x7f09f013e0c0 2024-05-09T06:36:47.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:47.897+0000 7f09e5ffb640 1 -- 172.21.15.3:0/1434202027 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f09e00072d0 con 0x7f09f013e0c0 2024-05-09T06:36:47.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:47.897+0000 7f09e5ffb640 1 -- 172.21.15.3:0/1434202027 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09e00047e0 con 0x7f09f013e0c0 2024-05-09T06:36:47.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:47.897+0000 7f09f73d5640 1 -- 172.21.15.3:0/1434202027 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09f00b0550 con 0x7f09f013e0c0 2024-05-09T06:36:47.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:47.897+0000 7f09f73d5640 1 -- 172.21.15.3:0/1434202027 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09f00b0b10 con 0x7f09f013e0c0 2024-05-09T06:36:47.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:47.898+0000 7f09f73d5640 1 -- 172.21.15.3:0/1434202027 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09f00b0d10 con 0x7f09f013e0c0 2024-05-09T06:36:47.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:47.898+0000 7f09e5ffb640 1 -- 172.21.15.3:0/1434202027 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f09e001b070 con 0x7f09f013e0c0 2024-05-09T06:36:47.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:47.899+0000 7f09e5ffb640 1 --2- 172.21.15.3:0/1434202027 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f09d406be30 0x7f09d406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:47.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:47.899+0000 7f09e5ffb640 1 -- 172.21.15.3:0/1434202027 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5825+0+0 (secure 0 0 0) 0x7f09e008a210 con 0x7f09f013e0c0 2024-05-09T06:36:47.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:47.899+0000 7f09f5171640 1 --2- 172.21.15.3:0/1434202027 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f09d406be30 0x7f09d406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:47.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:47.899+0000 7f09f5171640 1 --2- 172.21.15.3:0/1434202027 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f09d406be30 0x7f09d406e2f0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f09ec034000 tx=0x7f09ec02e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:47.883 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:47.903+0000 7f09e5ffb640 1 -- 172.21.15.3:0/1434202027 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f09f00b0d10 con 0x7f09f013e0c0 2024-05-09T06:36:48.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:48.091+0000 7f09f73d5640 1 -- 172.21.15.3:0/1434202027 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f09f0159dc0 con 0x7f09f013e0c0 2024-05-09T06:36:48.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:48.092+0000 7f09e5ffb640 1 -- 172.21.15.3:0/1434202027 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16771 (secure 0 0 0) 0x7f09f0159dc0 con 0x7f09f013e0c0 2024-05-09T06:36:48.072 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-09T06:36:48.072 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":22,"fsid":"16ed737d-2665-431a-8ac6-ef242be3975a","created":"2024-05-09T06:35:42.541368+0000","modified":"2024-05-09T06:36:48.023005+0000","last_up_change":"2024-05-09T06:35:53.778274+0000","last_in_change":"2024-05-09T06:35:44.780586+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T06:35:54.819999+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T06:36:04.931314+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T06:36:06.200433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fae3a8db-4826-4f3c-b33a-12c09bb5df6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":1897849827},{"type":"v1","addr":"172.21.15.3:6817","nonce":1897849827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":1897849827},{"type":"v1","addr":"172.21.15.3:6819","nonce":1897849827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":1897849827},{"type":"v1","addr":"172.21.15.3:6823","nonce":1897849827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":1897849827},{"type":"v1","addr":"172.21.15.3:6821","nonce":1897849827}]},"public_addr":"172.21.15.3:6817/1897849827","cluster_addr":"172.21.15.3:6819/1897849827","heartbeat_back_addr":"172.21.15.3:6823/1897849827","heartbeat_front_addr":"172.21.15.3:6821/1897849827","state":["exists","up"]},{"osd":1,"uuid":"646e882d-4ce4-4720-8c84-f71df563a35e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":926805498},{"type":"v1","addr":"172.21.15.3:6801","nonce":926805498}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":926805498},{"type":"v1","addr":"172.21.15.3:6803","nonce":926805498}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":926805498},{"type":"v1","addr":"172.21.15.3:6807","nonce":926805498}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":926805498},{"type":"v1","addr":"172.21.15.3:6805","nonce":926805498}]},"public_addr":"172.21.15.3:6801/926805498","cluster_addr":"172.21.15.3:6803/926805498","heartbeat_back_addr":"172.21.15.3:6807/926805498","heartbeat_front_addr":"172.21.15.3:6805/926805498","state":["exists","up"]},{"osd":2,"uuid":"1abbadb5-a570-4359-9643-acff3ede35a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":801295451},{"type":"v1","addr":"172.21.15.3:6809","nonce":801295451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":801295451},{"type":"v1","addr":"172.21.15.3:6811","nonce":801295451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":801295451},{"type":"v1","addr":"172.21.15.3:6815","nonce":801295451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":801295451},{"type":"v1","addr":"172.21.15.3:6813","nonce":801295451}]},"public_addr":"172.21.15.3:6809/801295451","cluster_addr":"172.21.15.3:6811/801295451","heartbeat_back_addr":"172.21.15.3:6815/801295451","heartbeat_front_addr":"172.21.15.3:6813/801295451","state":["exists","up"]},{"osd":3,"uuid":"62ac4dc1-2371-4dd2-acd3-e2dbb11aedf5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":1903389857},{"type":"v1","addr":"172.21.15.3:6825","nonce":1903389857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":1903389857},{"type":"v1","addr":"172.21.15.3:6827","nonce":1903389857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":1903389857},{"type":"v1","addr":"172.21.15.3:6831","nonce":1903389857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":1903389857},{"type":"v1","addr":"172.21.15.3:6829","nonce":1903389857}]},"public_addr":"172.21.15.3:6825/1903389857","cluster_addr":"172.21.15.3:6827/1903389857","heartbeat_back_addr":"172.21.15.3:6831/1903389857","heartbeat_front_addr":"172.21.15.3:6829/1903389857","state":["exists","up"]},{"osd":4,"uuid":"ae808398-8326-4d7c-9319-a2a0eafffb59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":2143624576},{"type":"v1","addr":"172.21.15.190:6801","nonce":2143624576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":2143624576},{"type":"v1","addr":"172.21.15.190:6803","nonce":2143624576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":2143624576},{"type":"v1","addr":"172.21.15.190:6807","nonce":2143624576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":2143624576},{"type":"v1","addr":"172.21.15.190:6805","nonce":2143624576}]},"public_addr":"172.21.15.190:6801/2143624576","cluster_addr":"172.21.15.190:6803/2143624576","heartbeat_back_addr":"172.21.15.190:6807/2143624576","heartbeat_front_addr":"172.21.15.190:6805/2143624576","state":["exists","up"]},{"osd":5,"uuid":"918adf50-c440-48ac-9a8d-4837562d7275","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":1766923126},{"type":"v1","addr":"172.21.15.190:6814","nonce":1766923126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":1766923126},{"type":"v1","addr":"172.21.15.190:6818","nonce":1766923126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":1766923126},{"type":"v1","addr":"172.21.15.190:6823","nonce":1766923126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":1766923126},{"type":"v1","addr":"172.21.15.190:6821","nonce":1766923126}]},"public_addr":"172.21.15.190:6814/1766923126","cluster_addr":"172.21.15.190:6818/1766923126","heartbeat_back_addr":"172.21.15.190:6823/1766923126","heartbeat_front_addr":"172.21.15.190:6821/1766923126","state":["exists","up"]},{"osd":6,"uuid":"ac2882bc-2a7c-463c-8f6c-76ff49828adc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":342414248},{"type":"v1","addr":"172.21.15.190:6825","nonce":342414248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":342414248},{"type":"v1","addr":"172.21.15.190:6827","nonce":342414248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":342414248},{"type":"v1","addr":"172.21.15.190:6831","nonce":342414248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":342414248},{"type":"v1","addr":"172.21.15.190:6829","nonce":342414248}]},"public_addr":"172.21.15.190:6825/342414248","cluster_addr":"172.21.15.190:6827/342414248","heartbeat_back_addr":"172.21.15.190:6831/342414248","heartbeat_front_addr":"172.21.15.190:6829/342414248","state":["exists","up"]},{"osd":7,"uuid":"dd4e767a-4730-4c70-8bd5-3f088140a5a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":537849891},{"type":"v1","addr":"172.21.15.190:6809","nonce":537849891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":537849891},{"type":"v1","addr":"172.21.15.190:6811","nonce":537849891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6817","nonce":537849891},{"type":"v1","addr":"172.21.15.190:6819","nonce":537849891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6813","nonce":537849891},{"type":"v1","addr":"172.21.15.190:6815","nonce":537849891}]},"public_addr":"172.21.15.190:6809/537849891","cluster_addr":"172.21.15.190:6811/537849891","heartbeat_back_addr":"172.21.15.190:6819/537849891","heartbeat_front_addr":"172.21.15.190:6815/537849891","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.889784+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.974002+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.973549+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.955396+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.928471+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.902418+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.958696+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.926474+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.27","mappings":[{"from":3,"to":1}]},{"pgid":"3.2a","mappings":[{"from":5,"to":2}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"3.36","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T06:36:48.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:48.094+0000 7f09f73d5640 1 -- 172.21.15.3:0/1434202027 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f09d406be30 msgr2=0x7f09d406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:48.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:48.094+0000 7f09f73d5640 1 --2- 172.21.15.3:0/1434202027 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f09d406be30 0x7f09d406e2f0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f09ec034000 tx=0x7f09ec02e000 comp rx=0 tx=0).stop 2024-05-09T06:36:48.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:48.094+0000 7f09f73d5640 1 -- 172.21.15.3:0/1434202027 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f09f013e0c0 msgr2=0x7f09f0150bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:48.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:48.094+0000 7f09f73d5640 1 --2- 172.21.15.3:0/1434202027 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f09f013e0c0 0x7f09f0150bb0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f09e000c780 tx=0x7f09e000cc40 comp rx=0 tx=0).stop 2024-05-09T06:36:48.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:48.094+0000 7f09f73d5640 1 -- 172.21.15.3:0/1434202027 shutdown_connections 2024-05-09T06:36:48.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:48.094+0000 7f09f73d5640 1 --2- 172.21.15.3:0/1434202027 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f09d406be30 0x7f09d406e2f0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:48.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:48.094+0000 7f09f73d5640 1 --2- 172.21.15.3:0/1434202027 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f09f0141000 0x7f09f01510f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:48.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:48.094+0000 7f09f73d5640 1 --2- 172.21.15.3:0/1434202027 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f09f013e0c0 0x7f09f0150bb0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:48.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:48.094+0000 7f09f73d5640 1 --2- 172.21.15.3:0/1434202027 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f09f00cc870 0x7f09f01f0b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:48.075 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:48.094+0000 7f09f73d5640 1 -- 172.21.15.3:0/1434202027 >> 172.21.15.3:0/1434202027 conn(0x7f09f006b430 msgr2=0x7f09f015df10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:48.075 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:48.094+0000 7f09f73d5640 1 -- 172.21.15.3:0/1434202027 shutdown_connections 2024-05-09T06:36:48.075 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:48.094+0000 7f09f73d5640 1 -- 172.21.15.3:0/1434202027 wait complete. 2024-05-09T06:36:48.085 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-05-09T06:36:48.190 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:48.210+0000 7f86f0cd6640 1 Processor -- start 2024-05-09T06:36:48.190 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:48.210+0000 7f86f0cd6640 1 -- start start 2024-05-09T06:36:48.193 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:48.211+0000 7f86f0cd6640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f86ec06c260 0x7f86ec169210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:48.193 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:48.211+0000 7f86f0cd6640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f86ec162ca0 con 0x7f86ec15e0b0 2024-05-09T06:36:48.193 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:48.211+0000 7f86f0cd6640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f86ec06cad0 con 0x7f86ec15e490 2024-05-09T06:36:48.194 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:48.211+0000 7f86f0cd6640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f86ec0b3eb0 con 0x7f86ec06c260 2024-05-09T06:36:48.194 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:48.211+0000 7f86ea59c640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f86ec15e0b0 0x7f86ec161e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:51194/0 (socket says 172.21.15.3:51194) 2024-05-09T06:36:48.194 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:48.211+0000 7f86ea59c640 1 -- 172.21.15.3:0/1150390577 learned_addr learned my addr 172.21.15.3:0/1150390577 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-05-09T06:36:48.194 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:48.211+0000 7f86ead9d640 1 --2- 172.21.15.3:0/1150390577 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f86ec06c260 0x7f86ec169210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:48.194 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:48.211+0000 7f86e959a640 1 -- 172.21.15.3:0/1150390577 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4025402745 0 0) 0x7f86ec162ca0 con 0x7f86ec15e0b0 2024-05-09T06:36:48.194 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:48.211+0000 7f86e959a640 1 -- 172.21.15.3:0/1150390577 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f86d8003520 con 0x7f86ec15e0b0 2024-05-09T06:36:48.194 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:48.212+0000 7f86e959a640 1 -- 172.21.15.3:0/1150390577 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2312732008 0 0) 0x7f86ec06cad0 con 0x7f86ec15e490 2024-05-09T06:36:48.194 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:48.212+0000 7f86e959a640 1 -- 172.21.15.3:0/1150390577 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f86ec162ca0 con 0x7f86ec15e490 2024-05-09T06:36:48.194 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:48.212+0000 7f86e959a640 1 -- 172.21.15.3:0/1150390577 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 999895495 0 0) 0x7f86d8003520 con 0x7f86ec15e0b0 2024-05-09T06:36:48.194 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:48.212+0000 7f86e959a640 1 -- 172.21.15.3:0/1150390577 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f86ec06c260 msgr2=0x7f86ec169210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:48.194 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:48.212+0000 7f86e959a640 1 --2- 172.21.15.3:0/1150390577 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f86ec06c260 0x7f86ec169210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:48.194 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:48.212+0000 7f86e959a640 1 -- 172.21.15.3:0/1150390577 >> v1:172.21.15.190:6789/0 conn(0x7f86ec15e490 legacy=0x7f86ec162590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:48.194 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:48.212+0000 7f86e959a640 1 -- 172.21.15.3:0/1150390577 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86ec0b4190 con 0x7f86ec15e0b0 2024-05-09T06:36:48.194 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:48.212+0000 7f86e959a640 1 -- 172.21.15.3:0/1150390577 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 563312031 0 0) 0x7f86e40019f0 con 0x7f86ec15e0b0 2024-05-09T06:36:48.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:48.212+0000 7f86e959a640 1 -- 172.21.15.3:0/1150390577 >> v1:172.21.15.3:6789/0 conn(0x7f86ec15e0b0 legacy=0x7f86ec161e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:48.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:48.212+0000 7f86e959a640 1 --2- 172.21.15.3:0/1150390577 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f86d8003b70 0x7f86d8003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:48.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:48.212+0000 7f86e959a640 1 --2- 172.21.15.3:0/1150390577 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f86d8004800 0x7f86d8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:48.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:48.212+0000 7f86e959a640 1 --2- 172.21.15.3:0/1150390577 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f86ec06c260 0x7f86d80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:48.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:48.212+0000 7f86e959a640 1 -- 172.21.15.3:0/1150390577 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f86ec0b3eb0 con 0x7f86d8003b70 2024-05-09T06:36:48.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:48.212+0000 7f86e959a640 1 -- 172.21.15.3:0/1150390577 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f86d800ba70 con 0x7f86d8004800 2024-05-09T06:36:48.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:48.212+0000 7f86e959a640 1 -- 172.21.15.3:0/1150390577 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f86d800bc40 con 0x7f86ec06c260 2024-05-09T06:36:48.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:48.212+0000 7f86ea59c640 1 --2- 172.21.15.3:0/1150390577 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f86d8003b70 0x7f86d8003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:48.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:48.212+0000 7f86ead9d640 1 --2- 172.21.15.3:0/1150390577 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f86ec06c260 0x7f86d80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:48.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:48.212+0000 7f86e9d9b640 1 --2- 172.21.15.3:0/1150390577 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f86d8004800 0x7f86d8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:48.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:48.212+0000 7f86ea59c640 1 -- 172.21.15.3:0/1150390577 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f86ec06c260 msgr2=0x7f86d80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:48.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:48.212+0000 7f86ea59c640 1 --2- 172.21.15.3:0/1150390577 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f86ec06c260 0x7f86d80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:48.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:48.212+0000 7f86ea59c640 1 -- 172.21.15.3:0/1150390577 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f86d8004800 msgr2=0x7f86d8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:48.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:48.212+0000 7f86ea59c640 1 --2- 172.21.15.3:0/1150390577 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f86d8004800 0x7f86d8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:48.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:48.212+0000 7f86ea59c640 1 -- 172.21.15.3:0/1150390577 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86d8003520 con 0x7f86d8003b70 2024-05-09T06:36:48.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:48.213+0000 7f86ea59c640 1 --2- 172.21.15.3:0/1150390577 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f86d8003b70 0x7f86d8003f70 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f86e400a810 tx=0x7f86d8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f140279f90eefdc server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:48.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:48.213+0000 7f86e959a640 1 -- 172.21.15.3:0/1150390577 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86e400a430 con 0x7f86d8003b70 2024-05-09T06:36:48.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:48.213+0000 7f86e959a640 1 -- 172.21.15.3:0/1150390577 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f86e402f070 con 0x7f86d8003b70 2024-05-09T06:36:48.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:48.213+0000 7f86e959a640 1 -- 172.21.15.3:0/1150390577 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86e402bc10 con 0x7f86d8003b70 2024-05-09T06:36:48.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:48.213+0000 7f86f0cd6640 1 -- 172.21.15.3:0/1150390577 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f86d8003b70 msgr2=0x7f86d8003f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:48.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:48.214+0000 7f86f0cd6640 1 --2- 172.21.15.3:0/1150390577 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f86d8003b70 0x7f86d8003f70 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f86e400a810 tx=0x7f86d8005830 comp rx=0 tx=0).stop 2024-05-09T06:36:48.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:48.214+0000 7f86ead9d640 1 -- 172.21.15.3:0/1150390577 reap_dead start 2024-05-09T06:36:48.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:48.214+0000 7f86f0cd6640 1 -- 172.21.15.3:0/1150390577 shutdown_connections 2024-05-09T06:36:48.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:48.214+0000 7f86f0cd6640 1 -- 172.21.15.3:0/1150390577 >> 172.21.15.3:0/1150390577 conn(0x7f86ec0d07b0 msgr2=0x7f86ec0d0bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:48.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:48.214+0000 7f86f0cd6640 1 -- 172.21.15.3:0/1150390577 shutdown_connections 2024-05-09T06:36:48.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:48.214+0000 7f86f0cd6640 1 -- 172.21.15.3:0/1150390577 wait complete. 2024-05-09T06:36:48.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:48.215+0000 7f86f0cd6640 1 Processor -- start 2024-05-09T06:36:48.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:48.215+0000 7f86f0cd6640 1 -- start start 2024-05-09T06:36:48.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:48.216+0000 7f86f0cd6640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f86d8003b70 0x7f86ec1f7590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:48.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:48.216+0000 7f86f0cd6640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f86ec1f7ad0 0x7f86ec15f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:48.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:48.216+0000 7f86f0cd6640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f86ec15f7f0 0x7f86ec15c190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:48.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:48.216+0000 7f86f0cd6640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f86ec06c6d0 con 0x7f86ec1f7ad0 2024-05-09T06:36:48.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:48.216+0000 7f86f0cd6640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f86ec0b4750 con 0x7f86ec15f7f0 2024-05-09T06:36:48.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:48.216+0000 7f86f0cd6640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f86ec06c850 con 0x7f86d8003b70 2024-05-09T06:36:48.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:48.216+0000 7f86e9d9b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f86ec1f7ad0 0x7f86ec15f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:48.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:48.216+0000 7f86e9d9b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f86ec1f7ad0 0x7f86ec15f2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46798/0 (socket says 172.21.15.3:46798) 2024-05-09T06:36:48.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:48.216+0000 7f86e9d9b640 1 -- 172.21.15.3:0/1421146227 learned_addr learned my addr 172.21.15.3:0/1421146227 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:36:48.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:48.216+0000 7f86ea59c640 1 --2- 172.21.15.3:0/1421146227 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f86d8003b70 0x7f86ec1f7590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:48.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:48.216+0000 7f86ead9d640 1 --2- 172.21.15.3:0/1421146227 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f86ec15f7f0 0x7f86ec15c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:48.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:48.216+0000 7f86e9d9b640 1 -- 172.21.15.3:0/1421146227 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f86d8003b70 msgr2=0x7f86ec1f7590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:48.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:48.216+0000 7f86e9d9b640 1 --2- 172.21.15.3:0/1421146227 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f86d8003b70 0x7f86ec1f7590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:48.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:48.216+0000 7f86e9d9b640 1 -- 172.21.15.3:0/1421146227 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f86ec15f7f0 msgr2=0x7f86ec15c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:48.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:48.216+0000 7f86e9d9b640 1 --2- 172.21.15.3:0/1421146227 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f86ec15f7f0 0x7f86ec15c190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:48.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:48.216+0000 7f86e9d9b640 1 -- 172.21.15.3:0/1421146227 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86ec15ffa0 con 0x7f86ec1f7ad0 2024-05-09T06:36:48.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:48.216+0000 7f86e9d9b640 1 --2- 172.21.15.3:0/1421146227 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f86ec1f7ad0 0x7f86ec15f2b0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f86d400e7b0 tx=0x7f86d400ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:48.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:48.217+0000 7f86e2ffd640 1 -- 172.21.15.3:0/1421146227 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86d400ee60 con 0x7f86ec1f7ad0 2024-05-09T06:36:48.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:48.217+0000 7f86e2ffd640 1 -- 172.21.15.3:0/1421146227 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f86d4018070 con 0x7f86ec1f7ad0 2024-05-09T06:36:48.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:48.217+0000 7f86e2ffd640 1 -- 172.21.15.3:0/1421146227 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86d400c6c0 con 0x7f86ec1f7ad0 2024-05-09T06:36:48.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:48.217+0000 7f86f0cd6640 1 -- 172.21.15.3:0/1421146227 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86ec140120 con 0x7f86ec1f7ad0 2024-05-09T06:36:48.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:48.217+0000 7f86f0cd6640 1 -- 172.21.15.3:0/1421146227 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86ec15fc70 con 0x7f86ec1f7ad0 2024-05-09T06:36:48.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:48.218+0000 7f86f0cd6640 1 -- 172.21.15.3:0/1421146227 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86ec069550 con 0x7f86ec1f7ad0 2024-05-09T06:36:48.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:48.218+0000 7f86e2ffd640 1 -- 172.21.15.3:0/1421146227 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f86d400c860 con 0x7f86ec1f7ad0 2024-05-09T06:36:48.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:48.219+0000 7f86e2ffd640 1 --2- 172.21.15.3:0/1421146227 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f86cc06be60 0x7f86cc06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:48.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:48.219+0000 7f86e2ffd640 1 -- 172.21.15.3:0/1421146227 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f86d408ab10 con 0x7f86ec1f7ad0 2024-05-09T06:36:48.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:48.219+0000 7f86ea59c640 1 --2- 172.21.15.3:0/1421146227 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f86cc06be60 0x7f86cc06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:48.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:48.219+0000 7f86ea59c640 1 --2- 172.21.15.3:0/1421146227 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f86cc06be60 0x7f86cc06e320 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f86e4033000 tx=0x7f86e4007000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:48.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:48.223+0000 7f86e2ffd640 1 -- 172.21.15.3:0/1421146227 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f86d4058e80 con 0x7f86ec1f7ad0 2024-05-09T06:36:48.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:48.449+0000 7f86f0cd6640 1 -- 172.21.15.3:0/1421146227 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f86ec0b3c90 con 0x7f86ec1f7ad0 2024-05-09T06:36:48.429 INFO:tasks.ceph.mon.a.smithi003.stderr:2024-05-09T06:36:48.449+0000 7f5fb0020640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-09T06:36:48.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:48.451+0000 7f86e2ffd640 1 -- 172.21.15.3:0/1421146227 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v9) ==== 161+0+0 (secure 0 0 0) 0x7f86d405d180 con 0x7f86ec1f7ad0 2024-05-09T06:36:48.430 INFO:teuthology.orchestra.run.smithi003.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-09T06:36:48.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:48.453+0000 7f86f0cd6640 1 -- 172.21.15.3:0/1421146227 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f86cc06be60 msgr2=0x7f86cc06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:48.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:48.453+0000 7f86f0cd6640 1 --2- 172.21.15.3:0/1421146227 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f86cc06be60 0x7f86cc06e320 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f86e4033000 tx=0x7f86e4007000 comp rx=0 tx=0).stop 2024-05-09T06:36:48.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:48.453+0000 7f86f0cd6640 1 -- 172.21.15.3:0/1421146227 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f86ec1f7ad0 msgr2=0x7f86ec15f2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:48.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:48.453+0000 7f86f0cd6640 1 --2- 172.21.15.3:0/1421146227 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f86ec1f7ad0 0x7f86ec15f2b0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f86d400e7b0 tx=0x7f86d400ec70 comp rx=0 tx=0).stop 2024-05-09T06:36:48.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:48.453+0000 7f86f0cd6640 1 -- 172.21.15.3:0/1421146227 shutdown_connections 2024-05-09T06:36:48.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:48.453+0000 7f86f0cd6640 1 --2- 172.21.15.3:0/1421146227 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f86cc06be60 0x7f86cc06e320 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:48.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:48.453+0000 7f86f0cd6640 1 --2- 172.21.15.3:0/1421146227 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f86ec15f7f0 0x7f86ec15c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:48.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:48.453+0000 7f86f0cd6640 1 --2- 172.21.15.3:0/1421146227 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f86ec1f7ad0 0x7f86ec15f2b0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:48.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:48.453+0000 7f86f0cd6640 1 --2- 172.21.15.3:0/1421146227 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f86d8003b70 0x7f86ec1f7590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:48.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:48.453+0000 7f86f0cd6640 1 -- 172.21.15.3:0/1421146227 >> 172.21.15.3:0/1421146227 conn(0x7f86ec0d07b0 msgr2=0x7f86ec06b250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:48.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:48.453+0000 7f86f0cd6640 1 -- 172.21.15.3:0/1421146227 shutdown_connections 2024-05-09T06:36:48.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:48.453+0000 7f86f0cd6640 1 -- 172.21.15.3:0/1421146227 wait complete. 2024-05-09T06:36:48.438 INFO:tasks.ceph.mds.d.smithi190.stderr: -7490> 2024-05-09T06:36:11.376+0000 7f33d478a640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-09T06:36:48.445 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-05-09T06:36:48.472 INFO:tasks.ceph.mds.d.smithi190.stdout:starting mds.d at 2024-05-09T06:36:48.551 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:48.571+0000 7fb3fcc31640 1 Processor -- start 2024-05-09T06:36:48.551 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:48.571+0000 7fb3fcc31640 1 -- start start 2024-05-09T06:36:48.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:48.572+0000 7fb3fcc31640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fb3f815e5c0 0x7fb3f80d2800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:48.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:48.572+0000 7fb3fcc31640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb3f80d1300 0x7fb3f80b47e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:48.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:48.572+0000 7fb3fcc31640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb3f815e9a0 con 0x7fb3f80d4c40 2024-05-09T06:36:48.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:48.572+0000 7fb3fcc31640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7fb3f80d50d0 con 0x7fb3f815e5c0 2024-05-09T06:36:48.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:48.572+0000 7fb3fcc31640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fb3f80d31c0 con 0x7fb3f80d1300 2024-05-09T06:36:48.554 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:48.573+0000 7fb3f6d9d640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb3f80d1300 0x7fb3f80b47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:48.554 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:48.573+0000 7fb3f659c640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fb3f815e5c0 0x7fb3f80d2800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:48.554 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:48.573+0000 7fb3f659c640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fb3f815e5c0 0x7fb3f80d2800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.3:32884/0 (socket says 172.21.15.3:32884) 2024-05-09T06:36:48.554 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:48.573+0000 7fb3f6d9d640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb3f80d1300 0x7fb3f80b47e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.3:45724/0 (socket says 172.21.15.3:45724) 2024-05-09T06:36:48.554 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:48.573+0000 7fb3f659c640 1 -- 172.21.15.3:0/2197067869 learned_addr learned my addr 172.21.15.3:0/2197067869 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:36:48.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:48.573+0000 7fb3f5d9b640 1 --1- 172.21.15.3:0/2197067869 >> v1:172.21.15.190:6789/0 conn(0x7fb3f80d4c40 0x7fb3f80d0bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.3:58318/0 (socket says 172.21.15.3:58318) 2024-05-09T06:36:48.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:48.573+0000 7fb3f6d9d640 1 -- 172.21.15.3:0/2197067869 >> v2:172.21.15.190:3300/0 conn(0x7fb3f815e5c0 msgr2=0x7fb3f80d2800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:48.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:48.573+0000 7fb3f6d9d640 1 --2- 172.21.15.3:0/2197067869 >> v2:172.21.15.190:3300/0 conn(0x7fb3f815e5c0 0x7fb3f80d2800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:48.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:48.573+0000 7fb3f6d9d640 1 -- 172.21.15.3:0/2197067869 >> v1:172.21.15.190:6789/0 conn(0x7fb3f80d4c40 legacy=0x7fb3f80d0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:48.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:48.573+0000 7fb3f6d9d640 1 -- 172.21.15.3:0/2197067869 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3f80d2ee0 con 0x7fb3f80d1300 2024-05-09T06:36:48.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:48.574+0000 7fb3f6d9d640 1 --2- 172.21.15.3:0/2197067869 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb3f80d1300 0x7fb3f80b47e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fb3e80097a0 tx=0x7fb3e802a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9286a62041f108bd server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:48.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:48.574+0000 7fb3f559a640 1 -- 172.21.15.3:0/2197067869 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3e8037070 con 0x7fb3f80d1300 2024-05-09T06:36:48.556 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:48.574+0000 7fb3f559a640 1 -- 172.21.15.3:0/2197067869 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb3e8032440 con 0x7fb3f80d1300 2024-05-09T06:36:48.556 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:48.574+0000 7fb3f559a640 1 -- 172.21.15.3:0/2197067869 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3e8032810 con 0x7fb3f80d1300 2024-05-09T06:36:48.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:48.575+0000 7fb3fcc31640 1 -- 172.21.15.3:0/2197067869 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb3f80d1300 msgr2=0x7fb3f80b47e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:48.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:48.575+0000 7fb3fcc31640 1 --2- 172.21.15.3:0/2197067869 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb3f80d1300 0x7fb3f80b47e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fb3e80097a0 tx=0x7fb3e802a180 comp rx=0 tx=0).stop 2024-05-09T06:36:48.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:48.575+0000 7fb3fcc31640 1 -- 172.21.15.3:0/2197067869 shutdown_connections 2024-05-09T06:36:48.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:48.575+0000 7fb3fcc31640 1 --2- 172.21.15.3:0/2197067869 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb3f80d1300 0x7fb3f80b47e0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:48.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:48.575+0000 7fb3fcc31640 1 --2- 172.21.15.3:0/2197067869 >> v2:172.21.15.190:3300/0 conn(0x7fb3f815e5c0 0x7fb3f80d2800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:48.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:48.575+0000 7fb3fcc31640 1 -- 172.21.15.3:0/2197067869 >> 172.21.15.3:0/2197067869 conn(0x7fb3f809d500 msgr2=0x7fb3f80d4220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:48.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:48.575+0000 7fb3fcc31640 1 -- 172.21.15.3:0/2197067869 shutdown_connections 2024-05-09T06:36:48.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:48.575+0000 7fb3fcc31640 1 -- 172.21.15.3:0/2197067869 wait complete. 2024-05-09T06:36:48.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:48.576+0000 7fb3fcc31640 1 Processor -- start 2024-05-09T06:36:48.564 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:48.576+0000 7fb3fcc31640 1 -- start start 2024-05-09T06:36:48.564 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:48.577+0000 7fb3fcc31640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb3f80d1300 0x7fb3f80d0210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:48.564 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:48.577+0000 7fb3fcc31640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb3f80d4c40 0x7fb3f80cc730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:48.564 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:48.577+0000 7fb3fcc31640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb3f815e5c0 0x7fb3f80ccc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:48.564 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:48.577+0000 7fb3fcc31640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fb3f80b42c0 con 0x7fb3f815e5c0 2024-05-09T06:36:48.564 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:48.577+0000 7fb3fcc31640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fb3f80b3fe0 con 0x7fb3f80d4c40 2024-05-09T06:36:48.564 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:48.577+0000 7fb3fcc31640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fb3f80b4d20 con 0x7fb3f80d1300 2024-05-09T06:36:48.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:48.577+0000 7fb3f6d9d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb3f815e5c0 0x7fb3f80ccc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:48.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:48.577+0000 7fb3f6d9d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb3f815e5c0 0x7fb3f80ccc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46800/0 (socket says 172.21.15.3:46800) 2024-05-09T06:36:48.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:48.577+0000 7fb3f6d9d640 1 -- 172.21.15.3:0/3759949237 learned_addr learned my addr 172.21.15.3:0/3759949237 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:36:48.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:48.577+0000 7fb3f5d9b640 1 --2- 172.21.15.3:0/3759949237 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb3f80d4c40 0x7fb3f80cc730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:48.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:48.577+0000 7fb3f659c640 1 --2- 172.21.15.3:0/3759949237 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb3f80d1300 0x7fb3f80d0210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:48.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:48.577+0000 7fb3f6d9d640 1 -- 172.21.15.3:0/3759949237 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb3f80d1300 msgr2=0x7fb3f80d0210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:48.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:48.577+0000 7fb3f6d9d640 1 --2- 172.21.15.3:0/3759949237 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb3f80d1300 0x7fb3f80d0210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:48.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:48.577+0000 7fb3f6d9d640 1 -- 172.21.15.3:0/3759949237 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb3f80d4c40 msgr2=0x7fb3f80cc730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:48.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:48.577+0000 7fb3f6d9d640 1 --2- 172.21.15.3:0/3759949237 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb3f80d4c40 0x7fb3f80cc730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:48.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:48.577+0000 7fb3f6d9d640 1 -- 172.21.15.3:0/3759949237 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3f80cb540 con 0x7fb3f815e5c0 2024-05-09T06:36:48.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:48.577+0000 7fb3f5d9b640 1 --2- 172.21.15.3:0/3759949237 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb3f80d4c40 0x7fb3f80cc730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:36:48.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:48.578+0000 7fb3f6d9d640 1 --2- 172.21.15.3:0/3759949237 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb3f815e5c0 0x7fb3f80ccc70 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fb3e80098c0 tx=0x7fb3e80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:48.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:48.578+0000 7fb3eeffd640 1 -- 172.21.15.3:0/3759949237 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3e8037040 con 0x7fb3f815e5c0 2024-05-09T06:36:48.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:48.578+0000 7fb3eeffd640 1 -- 172.21.15.3:0/3759949237 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb3e8042070 con 0x7fb3f815e5c0 2024-05-09T06:36:48.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:48.578+0000 7fb3fcc31640 1 -- 172.21.15.3:0/3759949237 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3f80cb820 con 0x7fb3f815e5c0 2024-05-09T06:36:48.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:48.578+0000 7fb3eeffd640 1 -- 172.21.15.3:0/3759949237 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3e80308a0 con 0x7fb3f815e5c0 2024-05-09T06:36:48.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:48.578+0000 7fb3fcc31640 1 -- 172.21.15.3:0/3759949237 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3f80cbde0 con 0x7fb3f815e5c0 2024-05-09T06:36:48.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:48.579+0000 7fb3fcc31640 1 -- 172.21.15.3:0/3759949237 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3f8069550 con 0x7fb3f815e5c0 2024-05-09T06:36:48.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:48.579+0000 7fb3eeffd640 1 -- 172.21.15.3:0/3759949237 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb3e8030a40 con 0x7fb3f815e5c0 2024-05-09T06:36:48.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:48.580+0000 7fb3eeffd640 1 --2- 172.21.15.3:0/3759949237 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fb3d806be30 0x7fb3d806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:48.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:48.580+0000 7fb3eeffd640 1 -- 172.21.15.3:0/3759949237 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5911+0+0 (secure 0 0 0) 0x7fb3e80ab960 con 0x7fb3f815e5c0 2024-05-09T06:36:48.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:48.580+0000 7fb3f659c640 1 --2- 172.21.15.3:0/3759949237 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fb3d806be30 0x7fb3d806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:48.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:48.581+0000 7fb3f659c640 1 --2- 172.21.15.3:0/3759949237 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fb3d806be30 0x7fb3d806e2f0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fb3f0009ce0 tx=0x7fb3f000b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:48.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:48.584+0000 7fb3eeffd640 1 -- 172.21.15.3:0/3759949237 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fb3e807a110 con 0x7fb3f815e5c0 2024-05-09T06:36:48.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:48.804+0000 7fb3fcc31640 1 -- 172.21.15.3:0/3759949237 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fb3f815dda0 con 0x7fb3f815e5c0 2024-05-09T06:36:49.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:49.456+0000 7fb3eeffd640 1 -- 172.21.15.3:0/3759949237 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v10) ==== 104+0+0 (secure 0 0 0) 0x7fb3e8079a10 con 0x7fb3f815e5c0 2024-05-09T06:36:49.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:49.459+0000 7fb3fcc31640 1 -- 172.21.15.3:0/3759949237 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fb3d806be30 msgr2=0x7fb3d806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:49.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:49.459+0000 7fb3fcc31640 1 --2- 172.21.15.3:0/3759949237 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fb3d806be30 0x7fb3d806e2f0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fb3f0009ce0 tx=0x7fb3f000b040 comp rx=0 tx=0).stop 2024-05-09T06:36:49.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:49.459+0000 7fb3fcc31640 1 -- 172.21.15.3:0/3759949237 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb3f815e5c0 msgr2=0x7fb3f80ccc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:49.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:49.459+0000 7fb3fcc31640 1 --2- 172.21.15.3:0/3759949237 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb3f815e5c0 0x7fb3f80ccc70 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fb3e80098c0 tx=0x7fb3e80046c0 comp rx=0 tx=0).stop 2024-05-09T06:36:49.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:49.459+0000 7fb3fcc31640 1 -- 172.21.15.3:0/3759949237 shutdown_connections 2024-05-09T06:36:49.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:49.459+0000 7fb3fcc31640 1 --2- 172.21.15.3:0/3759949237 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fb3d806be30 0x7fb3d806e2f0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:49.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:49.459+0000 7fb3fcc31640 1 --2- 172.21.15.3:0/3759949237 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb3f815e5c0 0x7fb3f80ccc70 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:49.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:49.459+0000 7fb3fcc31640 1 --2- 172.21.15.3:0/3759949237 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb3f80d4c40 0x7fb3f80cc730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:49.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:49.459+0000 7fb3fcc31640 1 --2- 172.21.15.3:0/3759949237 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb3f80d1300 0x7fb3f80d0210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:49.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:49.459+0000 7fb3fcc31640 1 -- 172.21.15.3:0/3759949237 >> 172.21.15.3:0/3759949237 conn(0x7fb3f809d500 msgr2=0x7fb3f80b1ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:49.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:49.459+0000 7fb3fcc31640 1 -- 172.21.15.3:0/3759949237 shutdown_connections 2024-05-09T06:36:49.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:49.459+0000 7fb3fcc31640 1 -- 172.21.15.3:0/3759949237 wait complete. 2024-05-09T06:36:49.454 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-05-09T06:36:49.570 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:49.590+0000 7f803f19e640 1 Processor -- start 2024-05-09T06:36:49.570 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:49.590+0000 7f803f19e640 1 -- start start 2024-05-09T06:36:49.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:49.591+0000 7f803f19e640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f8038070370 0x7f803815fc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:49.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:49.591+0000 7f803f19e640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8038162410 con 0x7f803809b1f0 2024-05-09T06:36:49.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:49.591+0000 7f803f19e640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f80381626f0 con 0x7f8038161ea0 2024-05-09T06:36:49.572 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:49.591+0000 7f803f19e640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f80380d0350 con 0x7f8038070370 2024-05-09T06:36:49.572 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:49.591+0000 7f803e19c640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f803809b1f0 0x7f803806f410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:51196/0 (socket says 172.21.15.3:51196) 2024-05-09T06:36:49.572 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:49.591+0000 7f803e19c640 1 -- 172.21.15.3:0/3726720081 learned_addr learned my addr 172.21.15.3:0/3726720081 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-05-09T06:36:49.572 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:49.591+0000 7f803e99d640 1 --2- 172.21.15.3:0/3726720081 >> v2:172.21.15.190:3300/0 conn(0x7f8038070370 0x7f803815fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:49.572 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:49.591+0000 7f803d19a640 1 -- 172.21.15.3:0/3726720081 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1666734334 0 0) 0x7f8038162410 con 0x7f803809b1f0 2024-05-09T06:36:49.572 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:49.591+0000 7f803d19a640 1 -- 172.21.15.3:0/3726720081 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8024003520 con 0x7f803809b1f0 2024-05-09T06:36:49.572 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:49.591+0000 7f803d19a640 1 -- 172.21.15.3:0/3726720081 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2340640634 0 0) 0x7f80381626f0 con 0x7f8038161ea0 2024-05-09T06:36:49.572 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:49.591+0000 7f803d19a640 1 -- 172.21.15.3:0/3726720081 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8038162410 con 0x7f8038161ea0 2024-05-09T06:36:49.572 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:49.592+0000 7f803e99d640 1 -- 172.21.15.3:0/3726720081 >> v1:172.21.15.190:6789/0 conn(0x7f8038161ea0 legacy=0x7f803806fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:49.572 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:49.592+0000 7f803d19a640 1 -- 172.21.15.3:0/3726720081 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 952389456 0 0) 0x7f8024003520 con 0x7f803809b1f0 2024-05-09T06:36:49.572 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:49.592+0000 7f803e99d640 1 -- 172.21.15.3:0/3726720081 >> v1:172.21.15.3:6789/0 conn(0x7f803809b1f0 legacy=0x7f803806f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:49.572 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:49.592+0000 7f803e99d640 1 -- 172.21.15.3:0/3726720081 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80381629d0 con 0x7f8038070370 2024-05-09T06:36:49.573 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:49.592+0000 7f803e99d640 1 --2- 172.21.15.3:0/3726720081 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8038070370 0x7f803815fc80 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f80300097a0 tx=0x7f803002a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=29fe262130fa96e9 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:49.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:49.593+0000 7f803d19a640 1 -- 172.21.15.3:0/3726720081 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8030037070 con 0x7f8038070370 2024-05-09T06:36:49.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:49.593+0000 7f803d19a640 1 -- 172.21.15.3:0/3726720081 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f803002ac50 con 0x7f8038070370 2024-05-09T06:36:49.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:49.593+0000 7f803d19a640 1 -- 172.21.15.3:0/3726720081 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80340019f0 con 0x7f8038070370 2024-05-09T06:36:49.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:49.593+0000 7f803f19e640 1 -- 172.21.15.3:0/3726720081 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8038070370 msgr2=0x7f803815fc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:49.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:49.593+0000 7f803f19e640 1 --2- 172.21.15.3:0/3726720081 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8038070370 0x7f803815fc80 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f80300097a0 tx=0x7f803002a1c0 comp rx=0 tx=0).stop 2024-05-09T06:36:49.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:49.593+0000 7f803f19e640 1 -- 172.21.15.3:0/3726720081 shutdown_connections 2024-05-09T06:36:49.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:49.593+0000 7f803f19e640 1 --2- 172.21.15.3:0/3726720081 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8038070370 0x7f803815fc80 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:49.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:49.593+0000 7f803f19e640 1 -- 172.21.15.3:0/3726720081 >> 172.21.15.3:0/3726720081 conn(0x7f80380b39e0 msgr2=0x7f80380b3df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:49.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:49.593+0000 7f803f19e640 1 -- 172.21.15.3:0/3726720081 shutdown_connections 2024-05-09T06:36:49.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:49.593+0000 7f803f19e640 1 -- 172.21.15.3:0/3726720081 wait complete. 2024-05-09T06:36:49.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:49.595+0000 7f803f19e640 1 Processor -- start 2024-05-09T06:36:49.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:49.595+0000 7f803f19e640 1 -- start start 2024-05-09T06:36:49.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:49.595+0000 7f803f19e640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8038070370 0x7f80380cba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:49.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:49.595+0000 7f803f19e640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f803809b1f0 0x7f80381f7340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:49.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:49.595+0000 7f803f19e640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8038161ea0 0x7f80381f7880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:49.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:49.595+0000 7f803f19e640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f803815eb70 con 0x7f8038161ea0 2024-05-09T06:36:49.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:49.595+0000 7f803f19e640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f803809b5d0 con 0x7f8038070370 2024-05-09T06:36:49.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:49.595+0000 7f803f19e640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f803815ecf0 con 0x7f803809b1f0 2024-05-09T06:36:49.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:49.595+0000 7f803e99d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8038161ea0 0x7f80381f7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:49.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:49.595+0000 7f803e99d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8038161ea0 0x7f80381f7880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46806/0 (socket says 172.21.15.3:46806) 2024-05-09T06:36:49.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:49.595+0000 7f803e99d640 1 -- 172.21.15.3:0/2722771681 learned_addr learned my addr 172.21.15.3:0/2722771681 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:36:49.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:49.596+0000 7f803e19c640 1 --2- 172.21.15.3:0/2722771681 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8038070370 0x7f80380cba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:49.578 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:49.596+0000 7f803d99b640 1 --2- 172.21.15.3:0/2722771681 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f803809b1f0 0x7f80381f7340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:49.578 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:49.596+0000 7f803e99d640 1 -- 172.21.15.3:0/2722771681 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f803809b1f0 msgr2=0x7f80381f7340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:49.578 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:49.596+0000 7f803e99d640 1 --2- 172.21.15.3:0/2722771681 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f803809b1f0 0x7f80381f7340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:49.578 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:49.596+0000 7f803e99d640 1 -- 172.21.15.3:0/2722771681 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8038070370 msgr2=0x7f80380cba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:49.578 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:49.596+0000 7f803e99d640 1 --2- 172.21.15.3:0/2722771681 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8038070370 0x7f80380cba80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:49.578 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:49.596+0000 7f803e99d640 1 -- 172.21.15.3:0/2722771681 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80380b0380 con 0x7f8038161ea0 2024-05-09T06:36:49.578 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:49.596+0000 7f803e99d640 1 --2- 172.21.15.3:0/2722771681 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8038161ea0 0x7f80381f7880 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f8030035040 tx=0x7f80300046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:49.578 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:49.596+0000 7f802effd640 1 -- 172.21.15.3:0/2722771681 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8030037050 con 0x7f8038161ea0 2024-05-09T06:36:49.578 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:49.596+0000 7f802effd640 1 -- 172.21.15.3:0/2722771681 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8030038040 con 0x7f8038161ea0 2024-05-09T06:36:49.578 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:49.596+0000 7f802effd640 1 -- 172.21.15.3:0/2722771681 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8030006e10 con 0x7f8038161ea0 2024-05-09T06:36:49.578 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:49.596+0000 7f803f19e640 1 -- 172.21.15.3:0/2722771681 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80380b0660 con 0x7f8038161ea0 2024-05-09T06:36:49.579 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:49.596+0000 7f803f19e640 1 -- 172.21.15.3:0/2722771681 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80380af690 con 0x7f8038161ea0 2024-05-09T06:36:49.579 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:49.597+0000 7f803f19e640 1 -- 172.21.15.3:0/2722771681 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f803814a8a0 con 0x7f8038161ea0 2024-05-09T06:36:49.579 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:49.598+0000 7f802effd640 1 -- 172.21.15.3:0/2722771681 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8030004270 con 0x7f8038161ea0 2024-05-09T06:36:49.580 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:49.598+0000 7f802effd640 1 --2- 172.21.15.3:0/2722771681 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f801c06be30 0x7f801c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:49.580 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:49.598+0000 7f802effd640 1 -- 172.21.15.3:0/2722771681 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5911+0+0 (secure 0 0 0) 0x7f80300ab560 con 0x7f8038161ea0 2024-05-09T06:36:49.580 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:49.599+0000 7f803e19c640 1 --2- 172.21.15.3:0/2722771681 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f801c06be30 0x7f801c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:49.580 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:49.599+0000 7f803e19c640 1 --2- 172.21.15.3:0/2722771681 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f801c06be30 0x7f801c06e2f0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f8034001560 tx=0x7f8034009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:49.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:49.603+0000 7f802effd640 1 -- 172.21.15.3:0/2722771681 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f8030079ff0 con 0x7f8038161ea0 2024-05-09T06:36:49.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:49.786+0000 7f803f19e640 1 -- 172.21.15.3:0/2722771681 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f80380737c0 con 0x7f8038161ea0 2024-05-09T06:36:50.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:50.458+0000 7f802effd640 1 -- 172.21.15.3:0/2722771681 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v24) ==== 181+0+0 (secure 0 0 0) 0x7f80300798f0 con 0x7f8038161ea0 2024-05-09T06:36:50.438 INFO:teuthology.orchestra.run.smithi003.stderr:pool 'cephfs_metadata' removed 2024-05-09T06:36:50.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:50.461+0000 7f803f19e640 1 -- 172.21.15.3:0/2722771681 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f801c06be30 msgr2=0x7f801c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:50.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:50.461+0000 7f803f19e640 1 --2- 172.21.15.3:0/2722771681 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f801c06be30 0x7f801c06e2f0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f8034001560 tx=0x7f8034009040 comp rx=0 tx=0).stop 2024-05-09T06:36:50.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:50.461+0000 7f803f19e640 1 -- 172.21.15.3:0/2722771681 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8038161ea0 msgr2=0x7f80381f7880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:50.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:50.461+0000 7f803f19e640 1 --2- 172.21.15.3:0/2722771681 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8038161ea0 0x7f80381f7880 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f8030035040 tx=0x7f80300046c0 comp rx=0 tx=0).stop 2024-05-09T06:36:50.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:50.462+0000 7f803f19e640 1 -- 172.21.15.3:0/2722771681 shutdown_connections 2024-05-09T06:36:50.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:50.462+0000 7f803f19e640 1 --2- 172.21.15.3:0/2722771681 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f801c06be30 0x7f801c06e2f0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:50.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:50.462+0000 7f803f19e640 1 --2- 172.21.15.3:0/2722771681 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8038161ea0 0x7f80381f7880 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:50.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:50.462+0000 7f803f19e640 1 --2- 172.21.15.3:0/2722771681 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f803809b1f0 0x7f80381f7340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:50.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:50.462+0000 7f803f19e640 1 --2- 172.21.15.3:0/2722771681 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8038070370 0x7f80380cba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:50.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:50.462+0000 7f803f19e640 1 -- 172.21.15.3:0/2722771681 >> 172.21.15.3:0/2722771681 conn(0x7f80380b39e0 msgr2=0x7f80380d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:50.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:50.462+0000 7f803f19e640 1 -- 172.21.15.3:0/2722771681 shutdown_connections 2024-05-09T06:36:50.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:50.462+0000 7f803f19e640 1 -- 172.21.15.3:0/2722771681 wait complete. 2024-05-09T06:36:50.456 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-05-09T06:36:50.572 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:50.592+0000 7f29ae121640 1 Processor -- start 2024-05-09T06:36:50.572 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:50.592+0000 7f29ae121640 1 -- start start 2024-05-09T06:36:50.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:50.593+0000 7f29ae121640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f29a81458b0 0x7f29a815c690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:50.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:50.593+0000 7f29ae121640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f29a80b4410 0x7f29a80cd7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:50.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:50.593+0000 7f29ae121640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f29a80cdcf0 0x7f29a815fc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:50.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:50.593+0000 7f29ae121640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f29a80b48c0 con 0x7f29a81458b0 2024-05-09T06:36:50.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:50.593+0000 7f29ae121640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f29a8145c90 con 0x7f29a80cdcf0 2024-05-09T06:36:50.576 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:50.593+0000 7f29ae121640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f29a80b4b60 con 0x7f29a80b4410 2024-05-09T06:36:50.576 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:50.593+0000 7f29a77fe640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f29a81458b0 0x7f29a815c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:50.576 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:50.593+0000 7f29a77fe640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f29a81458b0 0x7f29a815c690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:56546/0 (socket says 172.21.15.3:56546) 2024-05-09T06:36:50.576 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:50.593+0000 7f29a77fe640 1 -- 172.21.15.3:0/587857544 learned_addr learned my addr 172.21.15.3:0/587857544 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:36:50.576 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:50.593+0000 7f29a7fff640 1 --2- 172.21.15.3:0/587857544 >> v2:172.21.15.190:3300/0 conn(0x7f29a80cdcf0 0x7f29a815fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:50.576 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:50.593+0000 7f29a77fe640 1 -- 172.21.15.3:0/587857544 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f29a80b4410 msgr2=0x7f29a80cd7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:50.576 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:50.593+0000 7f29a6ffd640 1 --2- 172.21.15.3:0/587857544 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f29a80b4410 0x7f29a80cd7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:50.576 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:50.593+0000 7f29a77fe640 1 --2- 172.21.15.3:0/587857544 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f29a80b4410 0x7f29a80cd7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:50.576 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:50.593+0000 7f29a77fe640 1 -- 172.21.15.3:0/587857544 >> v2:172.21.15.190:3300/0 conn(0x7f29a80cdcf0 msgr2=0x7f29a815fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:50.576 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:50.593+0000 7f29a77fe640 1 --2- 172.21.15.3:0/587857544 >> v2:172.21.15.190:3300/0 conn(0x7f29a80cdcf0 0x7f29a815fc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:50.576 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:50.593+0000 7f29a77fe640 1 -- 172.21.15.3:0/587857544 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29a80b4e40 con 0x7f29a81458b0 2024-05-09T06:36:50.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:50.593+0000 7f29a7fff640 1 --2- 172.21.15.3:0/587857544 >> v2:172.21.15.190:3300/0 conn(0x7f29a80cdcf0 0x7f29a815fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:36:50.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:50.594+0000 7f29a77fe640 1 --2- 172.21.15.3:0/587857544 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f29a81458b0 0x7f29a815c690 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f29a8156cf0 tx=0x7f299802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54dfc9d80cfb3e18 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:50.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:50.594+0000 7f29a67fc640 1 -- 172.21.15.3:0/587857544 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2998037070 con 0x7f29a81458b0 2024-05-09T06:36:50.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:50.594+0000 7f29a67fc640 1 -- 172.21.15.3:0/587857544 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2998032440 con 0x7f29a81458b0 2024-05-09T06:36:50.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:50.594+0000 7f29a67fc640 1 -- 172.21.15.3:0/587857544 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2998032810 con 0x7f29a81458b0 2024-05-09T06:36:50.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:50.595+0000 7f29ae121640 1 -- 172.21.15.3:0/587857544 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f29a81458b0 msgr2=0x7f29a815c690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:50.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:50.595+0000 7f29ae121640 1 --2- 172.21.15.3:0/587857544 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f29a81458b0 0x7f29a815c690 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f29a8156cf0 tx=0x7f299802a1e0 comp rx=0 tx=0).stop 2024-05-09T06:36:50.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:50.595+0000 7f29ae121640 1 -- 172.21.15.3:0/587857544 shutdown_connections 2024-05-09T06:36:50.578 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:50.595+0000 7f29ae121640 1 --2- 172.21.15.3:0/587857544 >> v2:172.21.15.190:3300/0 conn(0x7f29a80cdcf0 0x7f29a815fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:50.578 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:50.595+0000 7f29ae121640 1 --2- 172.21.15.3:0/587857544 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f29a80b4410 0x7f29a80cd7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:50.578 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:50.595+0000 7f29ae121640 1 --2- 172.21.15.3:0/587857544 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f29a81458b0 0x7f29a815c690 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:50.578 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:50.595+0000 7f29ae121640 1 -- 172.21.15.3:0/587857544 >> 172.21.15.3:0/587857544 conn(0x7f29a806b640 msgr2=0x7f29a806ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:50.578 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:50.595+0000 7f29ae121640 1 -- 172.21.15.3:0/587857544 shutdown_connections 2024-05-09T06:36:50.578 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:50.595+0000 7f29ae121640 1 -- 172.21.15.3:0/587857544 wait complete. 2024-05-09T06:36:50.581 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:50.596+0000 7f29ae121640 1 Processor -- start 2024-05-09T06:36:50.581 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:50.596+0000 7f29ae121640 1 -- start start 2024-05-09T06:36:50.581 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:50.597+0000 7f29ae121640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f29a80b4410 0x7f29a8071a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:50.581 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:50.597+0000 7f29ae121640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f29a80cdcf0 0x7f29a8072b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:50.581 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:50.597+0000 7f29ae121640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f29a81458b0 0x7f29a813d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:50.581 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:50.597+0000 7f29ae121640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f29a806d6b0 con 0x7f29a81458b0 2024-05-09T06:36:50.581 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:50.597+0000 7f29ae121640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f29a80b5400 con 0x7f29a80cdcf0 2024-05-09T06:36:50.581 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:50.597+0000 7f29ae121640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f29a81601c0 con 0x7f29a80b4410 2024-05-09T06:36:50.581 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:50.597+0000 7f29a7fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f29a81458b0 0x7f29a813d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:50.581 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:50.597+0000 7f29a7fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f29a81458b0 0x7f29a813d6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:56548/0 (socket says 172.21.15.3:56548) 2024-05-09T06:36:50.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:50.597+0000 7f29a7fff640 1 -- 172.21.15.3:0/2396659912 learned_addr learned my addr 172.21.15.3:0/2396659912 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:36:50.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:50.597+0000 7f29a6ffd640 1 --2- 172.21.15.3:0/2396659912 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f29a80cdcf0 0x7f29a8072b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:50.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:50.597+0000 7f29a77fe640 1 --2- 172.21.15.3:0/2396659912 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f29a80b4410 0x7f29a8071a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:50.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:50.597+0000 7f29a7fff640 1 -- 172.21.15.3:0/2396659912 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f29a80b4410 msgr2=0x7f29a8071a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:50.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:50.597+0000 7f29a7fff640 1 --2- 172.21.15.3:0/2396659912 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f29a80b4410 0x7f29a8071a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:50.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:50.598+0000 7f29a7fff640 1 -- 172.21.15.3:0/2396659912 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f29a80cdcf0 msgr2=0x7f29a8072b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:50.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:50.598+0000 7f29a7fff640 1 --2- 172.21.15.3:0/2396659912 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f29a80cdcf0 0x7f29a8072b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:50.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:50.598+0000 7f29a7fff640 1 -- 172.21.15.3:0/2396659912 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29a80730a0 con 0x7f29a81458b0 2024-05-09T06:36:50.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:50.598+0000 7f29a6ffd640 1 --2- 172.21.15.3:0/2396659912 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f29a80cdcf0 0x7f29a8072b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:36:50.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:50.598+0000 7f29a7fff640 1 --2- 172.21.15.3:0/2396659912 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f29a81458b0 0x7f29a813d6a0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f29a000cef0 tx=0x7f29a000a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:50.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:50.598+0000 7f298ffff640 1 -- 172.21.15.3:0/2396659912 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29a001a070 con 0x7f29a81458b0 2024-05-09T06:36:50.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:50.598+0000 7f298ffff640 1 -- 172.21.15.3:0/2396659912 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f29a000d860 con 0x7f29a81458b0 2024-05-09T06:36:50.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:50.598+0000 7f29ae121640 1 -- 172.21.15.3:0/2396659912 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29a813dbe0 con 0x7f29a81458b0 2024-05-09T06:36:50.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:50.598+0000 7f298ffff640 1 -- 172.21.15.3:0/2396659912 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29a000dc50 con 0x7f29a81458b0 2024-05-09T06:36:50.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:50.598+0000 7f29ae121640 1 -- 172.21.15.3:0/2396659912 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29a80723e0 con 0x7f29a81458b0 2024-05-09T06:36:50.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:50.599+0000 7f29ae121640 1 -- 172.21.15.3:0/2396659912 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29a8069550 con 0x7f29a81458b0 2024-05-09T06:36:50.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:50.600+0000 7f298ffff640 1 -- 172.21.15.3:0/2396659912 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f29a001d070 con 0x7f29a81458b0 2024-05-09T06:36:50.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:50.600+0000 7f298ffff640 1 --2- 172.21.15.3:0/2396659912 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f299406be30 0x7f299406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:50.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:50.600+0000 7f298ffff640 1 -- 172.21.15.3:0/2396659912 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5473+0+0 (secure 0 0 0) 0x7f29a008b520 con 0x7f29a81458b0 2024-05-09T06:36:50.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:50.601+0000 7f29a77fe640 1 --2- 172.21.15.3:0/2396659912 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f299406be30 0x7f299406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:50.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:50.601+0000 7f29a77fe640 1 --2- 172.21.15.3:0/2396659912 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f299406be30 0x7f299406e2f0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f2998034000 tx=0x7f299802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:50.584 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:50.605+0000 7f298ffff640 1 -- 172.21.15.3:0/2396659912 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f29a005a040 con 0x7f29a81458b0 2024-05-09T06:36:50.767 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:50.787+0000 7f29ae121640 1 -- 172.21.15.3:0/2396659912 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f29a815d250 con 0x7f29a81458b0 2024-05-09T06:36:51.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:51.458+0000 7f298ffff640 1 -- 172.21.15.3:0/2396659912 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v25) ==== 169+0+0 (secure 0 0 0) 0x7f29a0059940 con 0x7f29a81458b0 2024-05-09T06:36:51.439 INFO:teuthology.orchestra.run.smithi003.stderr:pool 'cephfs_data' removed 2024-05-09T06:36:51.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:51.462+0000 7f29ae121640 1 -- 172.21.15.3:0/2396659912 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f299406be30 msgr2=0x7f299406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:51.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:51.462+0000 7f29ae121640 1 --2- 172.21.15.3:0/2396659912 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f299406be30 0x7f299406e2f0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f2998034000 tx=0x7f299802e000 comp rx=0 tx=0).stop 2024-05-09T06:36:51.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:51.462+0000 7f29ae121640 1 -- 172.21.15.3:0/2396659912 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f29a81458b0 msgr2=0x7f29a813d6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:51.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:51.462+0000 7f29ae121640 1 --2- 172.21.15.3:0/2396659912 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f29a81458b0 0x7f29a813d6a0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f29a000cef0 tx=0x7f29a000a710 comp rx=0 tx=0).stop 2024-05-09T06:36:51.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:51.464+0000 7f29ae121640 1 -- 172.21.15.3:0/2396659912 shutdown_connections 2024-05-09T06:36:51.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:51.464+0000 7f29ae121640 1 --2- 172.21.15.3:0/2396659912 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f299406be30 0x7f299406e2f0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:51.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:51.464+0000 7f29ae121640 1 --2- 172.21.15.3:0/2396659912 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f29a81458b0 0x7f29a813d6a0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:51.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:51.464+0000 7f29ae121640 1 --2- 172.21.15.3:0/2396659912 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f29a80cdcf0 0x7f29a8072b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:51.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:51.464+0000 7f29ae121640 1 --2- 172.21.15.3:0/2396659912 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f29a80b4410 0x7f29a8071a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:51.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:51.464+0000 7f29ae121640 1 -- 172.21.15.3:0/2396659912 >> 172.21.15.3:0/2396659912 conn(0x7f29a806b640 msgr2=0x7f29a806ba20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:51.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:51.464+0000 7f29ae121640 1 -- 172.21.15.3:0/2396659912 shutdown_connections 2024-05-09T06:36:51.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:51.464+0000 7f29ae121640 1 -- 172.21.15.3:0/2396659912 wait complete. 2024-05-09T06:36:51.462 INFO:tasks.ceph.mds.a:Restarting daemon 2024-05-09T06:36:51.462 INFO:tasks.ceph.mds.a:Stopping old one... 2024-05-09T06:36:51.462 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-05-09T06:36:51.463 INFO:teuthology.orchestra.run:waiting for 300 2024-05-09T06:36:51.463 INFO:tasks.ceph.mds.c:Restarting daemon 2024-05-09T06:36:51.463 INFO:tasks.ceph.mds.c:Stopping old one... 2024-05-09T06:36:51.463 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-05-09T06:36:51.463 INFO:teuthology.orchestra.run:waiting for 300 2024-05-09T06:36:51.463 INFO:tasks.ceph.mds.b:Restarting daemon 2024-05-09T06:36:51.463 INFO:tasks.ceph.mds.b:Stopping old one... 2024-05-09T06:36:51.464 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-05-09T06:36:51.464 INFO:teuthology.orchestra.run:waiting for 300 2024-05-09T06:36:51.464 INFO:tasks.ceph.mds.d:Restarting daemon 2024-05-09T06:36:51.464 INFO:tasks.ceph.mds.d:Stopping old one... 2024-05-09T06:36:51.464 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-05-09T06:36:51.464 INFO:teuthology.orchestra.run:waiting for 300 2024-05-09T06:36:51.474 INFO:tasks.ceph.mds.a:Stopped 2024-05-09T06:36:51.475 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-05-09T06:36:51.476 INFO:tasks.ceph.mds.c:Stopped 2024-05-09T06:36:51.477 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-05-09T06:36:51.479 INFO:tasks.ceph.mds.d:Stopped 2024-05-09T06:36:51.479 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-05-09T06:36:51.481 INFO:tasks.ceph.mds.b:Stopped 2024-05-09T06:36:51.481 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-05-09T06:36:51.518 INFO:tasks.ceph.mds.a:Started 2024-05-09T06:36:51.518 INFO:tasks.ceph.mds.c:Started 2024-05-09T06:36:51.519 DEBUG:teuthology.parallel:result is None 2024-05-09T06:36:51.519 DEBUG:teuthology.parallel:result is None 2024-05-09T06:36:51.522 INFO:tasks.ceph.mds.d:Started 2024-05-09T06:36:51.523 INFO:tasks.ceph.mds.b:Started 2024-05-09T06:36:51.523 DEBUG:teuthology.parallel:result is None 2024-05-09T06:36:51.523 DEBUG:teuthology.parallel:result is None 2024-05-09T06:36:51.524 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-05-09T06:36:51.603 INFO:tasks.ceph.mds.b.smithi190.stdout:starting mds.b at 2024-05-09T06:36:51.603 INFO:tasks.ceph.mds.d.smithi190.stdout:starting mds.d at 2024-05-09T06:36:51.603 INFO:tasks.ceph.mds.a.smithi003.stdout:starting mds.a at 2024-05-09T06:36:51.604 INFO:tasks.ceph.mds.c.smithi003.stdout:starting mds.c at 2024-05-09T06:36:51.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:51.658+0000 7f0c6e8dd640 1 Processor -- start 2024-05-09T06:36:51.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:51.658+0000 7f0c6e8dd640 1 -- start start 2024-05-09T06:36:51.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:51.659+0000 7f0c6e8dd640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0c6815e5c0 0x7f0c680d2800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:51.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:51.659+0000 7f0c6e8dd640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0c6815e9a0 con 0x7f0c680d4c40 2024-05-09T06:36:51.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:51.659+0000 7f0c6e8dd640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0c680d31c0 con 0x7f0c680d1300 2024-05-09T06:36:51.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:51.659+0000 7f0c6e8dd640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f0c680d50a0 con 0x7f0c6815e5c0 2024-05-09T06:36:51.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:51.659+0000 7f0c677fe640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f0c680d4c40 0x7f0c680d0bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:53340/0 (socket says 172.21.15.3:53340) 2024-05-09T06:36:51.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:51.659+0000 7f0c677fe640 1 -- 172.21.15.3:0/1546121605 learned_addr learned my addr 172.21.15.3:0/1546121605 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-05-09T06:36:51.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:51.659+0000 7f0c67fff640 1 --2- 172.21.15.3:0/1546121605 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0c6815e5c0 0x7f0c680d2800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:51.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:51.660+0000 7f0c66ffd640 1 -- 172.21.15.3:0/1546121605 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2760636755 0 0) 0x7f0c6815e9a0 con 0x7f0c680d4c40 2024-05-09T06:36:51.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:51.660+0000 7f0c66ffd640 1 -- 172.21.15.3:0/1546121605 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0c50003520 con 0x7f0c680d4c40 2024-05-09T06:36:51.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:51.660+0000 7f0c66ffd640 1 -- 172.21.15.3:0/1546121605 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2551971508 0 0) 0x7f0c680d31c0 con 0x7f0c680d1300 2024-05-09T06:36:51.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:51.660+0000 7f0c66ffd640 1 -- 172.21.15.3:0/1546121605 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0c6815e9a0 con 0x7f0c680d1300 2024-05-09T06:36:51.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:51.660+0000 7f0c67fff640 1 -- 172.21.15.3:0/1546121605 >> v1:172.21.15.190:6789/0 conn(0x7f0c680d1300 legacy=0x7f0c680b47e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:51.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:51.660+0000 7f0c67fff640 1 -- 172.21.15.3:0/1546121605 >> v1:172.21.15.3:6789/0 conn(0x7f0c680d4c40 legacy=0x7f0c680d0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:51.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:51.660+0000 7f0c67fff640 1 -- 172.21.15.3:0/1546121605 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c680d2ee0 con 0x7f0c6815e5c0 2024-05-09T06:36:51.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:51.661+0000 7f0c67fff640 1 --2- 172.21.15.3:0/1546121605 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0c6815e5c0 0x7f0c680d2800 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f0c580097d0 tx=0x7f0c5802a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bed4bae185227c44 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:51.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:51.661+0000 7f0c66ffd640 1 -- 172.21.15.3:0/1546121605 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c58037070 con 0x7f0c6815e5c0 2024-05-09T06:36:51.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:51.661+0000 7f0c66ffd640 1 -- 172.21.15.3:0/1546121605 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c58032440 con 0x7f0c6815e5c0 2024-05-09T06:36:51.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:51.661+0000 7f0c66ffd640 1 -- 172.21.15.3:0/1546121605 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c58032810 con 0x7f0c6815e5c0 2024-05-09T06:36:51.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:51.661+0000 7f0c6e8dd640 1 -- 172.21.15.3:0/1546121605 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0c6815e5c0 msgr2=0x7f0c680d2800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:51.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:51.661+0000 7f0c6e8dd640 1 --2- 172.21.15.3:0/1546121605 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0c6815e5c0 0x7f0c680d2800 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f0c580097d0 tx=0x7f0c5802a210 comp rx=0 tx=0).stop 2024-05-09T06:36:51.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:51.661+0000 7f0c6e8dd640 1 -- 172.21.15.3:0/1546121605 shutdown_connections 2024-05-09T06:36:51.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:51.661+0000 7f0c6e8dd640 1 --2- 172.21.15.3:0/1546121605 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0c6815e5c0 0x7f0c680d2800 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:51.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:51.661+0000 7f0c6e8dd640 1 -- 172.21.15.3:0/1546121605 >> 172.21.15.3:0/1546121605 conn(0x7f0c6809d500 msgr2=0x7f0c680d4220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:51.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:51.661+0000 7f0c6e8dd640 1 -- 172.21.15.3:0/1546121605 shutdown_connections 2024-05-09T06:36:51.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:51.661+0000 7f0c6e8dd640 1 -- 172.21.15.3:0/1546121605 wait complete. 2024-05-09T06:36:51.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:51.663+0000 7f0c6e8dd640 1 Processor -- start 2024-05-09T06:36:51.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:51.663+0000 7f0c6e8dd640 1 -- start start 2024-05-09T06:36:51.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:51.663+0000 7f0c6e8dd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0c680d1300 0x7f0c680b6e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:51.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:51.663+0000 7f0c6e8dd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0c680d4c40 0x7f0c680b93b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:51.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:51.663+0000 7f0c6e8dd640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0c6815e5c0 0x7f0c681f3b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:51.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:51.663+0000 7f0c6e8dd640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f0c680b42c0 con 0x7f0c680d1300 2024-05-09T06:36:51.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:51.663+0000 7f0c6e8dd640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f0c680b3fe0 con 0x7f0c680d4c40 2024-05-09T06:36:51.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:51.663+0000 7f0c6e8dd640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f0c680b4ef0 con 0x7f0c6815e5c0 2024-05-09T06:36:51.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:51.664+0000 7f0c67fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0c680d1300 0x7f0c680b6e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:51.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:51.664+0000 7f0c67fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0c680d1300 0x7f0c680b6e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:56598/0 (socket says 172.21.15.3:56598) 2024-05-09T06:36:51.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:51.664+0000 7f0c67fff640 1 -- 172.21.15.3:0/1976226956 learned_addr learned my addr 172.21.15.3:0/1976226956 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:36:51.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:51.664+0000 7f0c6ce7a640 1 --2- 172.21.15.3:0/1976226956 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0c6815e5c0 0x7f0c681f3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:51.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:51.664+0000 7f0c677fe640 1 --2- 172.21.15.3:0/1976226956 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0c680d4c40 0x7f0c680b93b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:51.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:51.664+0000 7f0c67fff640 1 -- 172.21.15.3:0/1976226956 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0c6815e5c0 msgr2=0x7f0c681f3b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:51.648 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:51.664+0000 7f0c67fff640 1 --2- 172.21.15.3:0/1976226956 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0c6815e5c0 0x7f0c681f3b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:51.648 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:51.664+0000 7f0c67fff640 1 -- 172.21.15.3:0/1976226956 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0c680d4c40 msgr2=0x7f0c680b93b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:51.648 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:51.664+0000 7f0c67fff640 1 --2- 172.21.15.3:0/1976226956 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0c680d4c40 0x7f0c680b93b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:51.648 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:51.664+0000 7f0c67fff640 1 -- 172.21.15.3:0/1976226956 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c680b7520 con 0x7f0c680d1300 2024-05-09T06:36:51.648 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:51.664+0000 7f0c6ce7a640 1 --2- 172.21.15.3:0/1976226956 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0c6815e5c0 0x7f0c681f3b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:36:51.648 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:51.664+0000 7f0c677fe640 1 --2- 172.21.15.3:0/1976226956 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0c680d4c40 0x7f0c680b93b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:36:51.648 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:51.664+0000 7f0c67fff640 1 --2- 172.21.15.3:0/1976226956 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0c680d1300 0x7f0c680b6e40 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f0c5802a250 tx=0x7f0c580046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:51.648 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:51.664+0000 7f0c64ff9640 1 -- 172.21.15.3:0/1976226956 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c58037040 con 0x7f0c680d1300 2024-05-09T06:36:51.648 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:51.664+0000 7f0c64ff9640 1 -- 172.21.15.3:0/1976226956 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c58042070 con 0x7f0c680d1300 2024-05-09T06:36:51.648 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:51.664+0000 7f0c6e8dd640 1 -- 172.21.15.3:0/1976226956 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c680b7800 con 0x7f0c680d1300 2024-05-09T06:36:51.648 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:51.664+0000 7f0c64ff9640 1 -- 172.21.15.3:0/1976226956 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c580308c0 con 0x7f0c680d1300 2024-05-09T06:36:51.648 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:51.664+0000 7f0c6e8dd640 1 -- 172.21.15.3:0/1976226956 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c680b7dc0 con 0x7f0c680d1300 2024-05-09T06:36:51.648 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:51.665+0000 7f0c6e8dd640 1 -- 172.21.15.3:0/1976226956 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c68069550 con 0x7f0c680d1300 2024-05-09T06:36:51.648 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:51.666+0000 7f0c64ff9640 1 -- 172.21.15.3:0/1976226956 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0c58030a60 con 0x7f0c680d1300 2024-05-09T06:36:51.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:51.666+0000 7f0c64ff9640 1 --2- 172.21.15.3:0/1976226956 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f0c5406be30 0x7f0c5406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:51.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:51.666+0000 7f0c64ff9640 1 -- 172.21.15.3:0/1976226956 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 4943+0+0 (secure 0 0 0) 0x7f0c580ab450 con 0x7f0c680d1300 2024-05-09T06:36:51.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:51.667+0000 7f0c677fe640 1 --2- 172.21.15.3:0/1976226956 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f0c5406be30 0x7f0c5406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:51.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:51.667+0000 7f0c677fe640 1 --2- 172.21.15.3:0/1976226956 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f0c5406be30 0x7f0c5406e2f0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f0c6806cc70 tx=0x7f0c5c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:51.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:51.671+0000 7f0c64ff9640 1 -- 172.21.15.3:0/1976226956 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f0c58045360 con 0x7f0c680d1300 2024-05-09T06:36:51.848 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:51.867+0000 7f0c6e8dd640 1 -- 172.21.15.3:0/1976226956 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f0c680d43c0 con 0x7f0c680d1300 2024-05-09T06:36:52.455 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:52.474+0000 7f0c64ff9640 1 -- 172.21.15.3:0/1976226956 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v27) ==== 97+0+0 (secure 0 0 0) 0x7f0c5807a100 con 0x7f0c680d1300 2024-05-09T06:36:52.455 INFO:teuthology.orchestra.run.smithi003.stderr: removed all blocklist entries 2024-05-09T06:36:52.458 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:52.477+0000 7f0c6e8dd640 1 -- 172.21.15.3:0/1976226956 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f0c5406be30 msgr2=0x7f0c5406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:52.458 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:52.477+0000 7f0c6e8dd640 1 --2- 172.21.15.3:0/1976226956 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f0c5406be30 0x7f0c5406e2f0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f0c6806cc70 tx=0x7f0c5c009040 comp rx=0 tx=0).stop 2024-05-09T06:36:52.458 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:52.477+0000 7f0c6e8dd640 1 -- 172.21.15.3:0/1976226956 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0c680d1300 msgr2=0x7f0c680b6e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:52.458 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:52.477+0000 7f0c6e8dd640 1 --2- 172.21.15.3:0/1976226956 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0c680d1300 0x7f0c680b6e40 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f0c5802a250 tx=0x7f0c580046c0 comp rx=0 tx=0).stop 2024-05-09T06:36:52.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:52.477+0000 7f0c6e8dd640 1 -- 172.21.15.3:0/1976226956 shutdown_connections 2024-05-09T06:36:52.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:52.477+0000 7f0c6e8dd640 1 --2- 172.21.15.3:0/1976226956 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f0c5406be30 0x7f0c5406e2f0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:52.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:52.477+0000 7f0c6e8dd640 1 --2- 172.21.15.3:0/1976226956 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0c6815e5c0 0x7f0c681f3b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:52.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:52.477+0000 7f0c6e8dd640 1 --2- 172.21.15.3:0/1976226956 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0c680d4c40 0x7f0c680b93b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:52.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:52.477+0000 7f0c6e8dd640 1 --2- 172.21.15.3:0/1976226956 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0c680d1300 0x7f0c680b6e40 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:52.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:52.477+0000 7f0c6e8dd640 1 -- 172.21.15.3:0/1976226956 >> 172.21.15.3:0/1976226956 conn(0x7f0c6809d500 msgr2=0x7f0c680d0250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:52.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:52.477+0000 7f0c6e8dd640 1 -- 172.21.15.3:0/1976226956 shutdown_connections 2024-05-09T06:36:52.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:52.478+0000 7f0c6e8dd640 1 -- 172.21.15.3:0/1976226956 wait complete. 2024-05-09T06:36:52.472 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-05-09T06:36:52.580 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:52.599+0000 7f47e89a9640 1 Processor -- start 2024-05-09T06:36:52.580 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:52.599+0000 7f47e89a9640 1 -- start start 2024-05-09T06:36:52.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:52.600+0000 7f47e89a9640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f47e413e0c0 0x7f47e40cf720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:52.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:52.600+0000 7f47e89a9640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f47e40cc870 0x7f47e40cccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:52.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:52.600+0000 7f47e89a9640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f47e41408f0 0x7f47e4140d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:52.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:52.600+0000 7f47e89a9640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f47e413e4a0 con 0x7f47e40cc870 2024-05-09T06:36:52.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:52.600+0000 7f47e89a9640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f47e40cd210 con 0x7f47e41408f0 2024-05-09T06:36:52.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:52.600+0000 7f47e89a9640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f47e40ce960 con 0x7f47e413e0c0 2024-05-09T06:36:52.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:52.600+0000 7f47d9d9b640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f47e40cc870 0x7f47e40cccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:52.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:52.600+0000 7f47d9d9b640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f47e40cc870 0x7f47e40cccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:56606/0 (socket says 172.21.15.3:56606) 2024-05-09T06:36:52.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:52.600+0000 7f47d9d9b640 1 -- 172.21.15.3:0/794981162 learned_addr learned my addr 172.21.15.3:0/794981162 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:36:52.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:52.600+0000 7f47e259c640 1 --2- 172.21.15.3:0/794981162 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f47e413e0c0 0x7f47e40cf720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:52.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:52.600+0000 7f47e2d9d640 1 --2- 172.21.15.3:0/794981162 >> v2:172.21.15.190:3300/0 conn(0x7f47e41408f0 0x7f47e4140d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:52.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:52.600+0000 7f47d9d9b640 1 -- 172.21.15.3:0/794981162 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f47e413e0c0 msgr2=0x7f47e40cf720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:52.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:52.600+0000 7f47d9d9b640 1 --2- 172.21.15.3:0/794981162 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f47e413e0c0 0x7f47e40cf720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:52.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:52.600+0000 7f47d9d9b640 1 -- 172.21.15.3:0/794981162 >> v2:172.21.15.190:3300/0 conn(0x7f47e41408f0 msgr2=0x7f47e4140d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:52.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:52.600+0000 7f47d9d9b640 1 --2- 172.21.15.3:0/794981162 >> v2:172.21.15.190:3300/0 conn(0x7f47e41408f0 0x7f47e4140d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:52.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:52.600+0000 7f47d9d9b640 1 -- 172.21.15.3:0/794981162 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47e40cec40 con 0x7f47e40cc870 2024-05-09T06:36:52.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:52.600+0000 7f47e2d9d640 1 --2- 172.21.15.3:0/794981162 >> v2:172.21.15.190:3300/0 conn(0x7f47e41408f0 0x7f47e4140d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:36:52.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:52.601+0000 7f47d9d9b640 1 --2- 172.21.15.3:0/794981162 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f47e40cc870 0x7f47e40cccd0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f47e406dc80 tx=0x7f47cc02c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cec7ad9ff3ac8fd server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:52.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:52.601+0000 7f47e1d9b640 1 -- 172.21.15.3:0/794981162 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47cc032790 con 0x7f47e40cc870 2024-05-09T06:36:52.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:52.601+0000 7f47e1d9b640 1 -- 172.21.15.3:0/794981162 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f47cc038070 con 0x7f47e40cc870 2024-05-09T06:36:52.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:52.601+0000 7f47e1d9b640 1 -- 172.21.15.3:0/794981162 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47cc032b50 con 0x7f47e40cc870 2024-05-09T06:36:52.584 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:52.602+0000 7f47e89a9640 1 -- 172.21.15.3:0/794981162 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f47e40cc870 msgr2=0x7f47e40cccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:52.584 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:52.602+0000 7f47e89a9640 1 --2- 172.21.15.3:0/794981162 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f47e40cc870 0x7f47e40cccd0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f47e406dc80 tx=0x7f47cc02c890 comp rx=0 tx=0).stop 2024-05-09T06:36:52.584 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:52.602+0000 7f47e89a9640 1 -- 172.21.15.3:0/794981162 shutdown_connections 2024-05-09T06:36:52.584 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:52.602+0000 7f47e89a9640 1 --2- 172.21.15.3:0/794981162 >> v2:172.21.15.190:3300/0 conn(0x7f47e41408f0 0x7f47e4140d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:52.584 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:52.602+0000 7f47e89a9640 1 --2- 172.21.15.3:0/794981162 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f47e40cc870 0x7f47e40cccd0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:52.584 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:52.602+0000 7f47e89a9640 1 --2- 172.21.15.3:0/794981162 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f47e413e0c0 0x7f47e40cf720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:52.585 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:52.602+0000 7f47e89a9640 1 -- 172.21.15.3:0/794981162 >> 172.21.15.3:0/794981162 conn(0x7f47e406b430 msgr2=0x7f47e406b810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:52.585 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:52.602+0000 7f47e89a9640 1 -- 172.21.15.3:0/794981162 shutdown_connections 2024-05-09T06:36:52.585 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:52.602+0000 7f47e89a9640 1 -- 172.21.15.3:0/794981162 wait complete. 2024-05-09T06:36:52.585 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:52.603+0000 7f47e89a9640 1 Processor -- start 2024-05-09T06:36:52.585 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:52.603+0000 7f47e89a9640 1 -- start start 2024-05-09T06:36:52.587 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:52.604+0000 7f47e89a9640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f47e40cc870 0x7f47e40aebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:52.587 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:52.604+0000 7f47e89a9640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f47e413e0c0 0x7f47e413d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:52.587 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:52.604+0000 7f47e89a9640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f47e41408f0 0x7f47e41f4410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:52.587 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:52.604+0000 7f47e89a9640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f47e4141660 con 0x7f47e41408f0 2024-05-09T06:36:52.587 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:52.604+0000 7f47e89a9640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f47e40cf200 con 0x7f47e413e0c0 2024-05-09T06:36:52.587 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:52.604+0000 7f47e89a9640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f47e4141280 con 0x7f47e40cc870 2024-05-09T06:36:52.587 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:52.604+0000 7f47e2d9d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f47e41408f0 0x7f47e41f4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:52.587 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:52.604+0000 7f47e2d9d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f47e41408f0 0x7f47e41f4410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:56614/0 (socket says 172.21.15.3:56614) 2024-05-09T06:36:52.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:52.604+0000 7f47e2d9d640 1 -- 172.21.15.3:0/3833636144 learned_addr learned my addr 172.21.15.3:0/3833636144 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:36:52.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:52.604+0000 7f47e259c640 1 --2- 172.21.15.3:0/3833636144 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f47e40cc870 0x7f47e40aebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:52.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:52.604+0000 7f47d9d9b640 1 --2- 172.21.15.3:0/3833636144 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f47e413e0c0 0x7f47e413d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:52.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:52.604+0000 7f47e2d9d640 1 -- 172.21.15.3:0/3833636144 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f47e40cc870 msgr2=0x7f47e40aebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:52.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:52.604+0000 7f47e2d9d640 1 --2- 172.21.15.3:0/3833636144 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f47e40cc870 0x7f47e40aebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:52.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:52.604+0000 7f47e2d9d640 1 -- 172.21.15.3:0/3833636144 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f47e413e0c0 msgr2=0x7f47e413d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:52.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:52.604+0000 7f47e2d9d640 1 --2- 172.21.15.3:0/3833636144 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f47e413e0c0 0x7f47e413d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:52.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:52.604+0000 7f47e2d9d640 1 -- 172.21.15.3:0/3833636144 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47e40ad8a0 con 0x7f47e41408f0 2024-05-09T06:36:52.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:52.605+0000 7f47e2d9d640 1 --2- 172.21.15.3:0/3833636144 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f47e41408f0 0x7f47e41f4410 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f47dc009fa0 tx=0x7f47dc00d560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:52.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:52.605+0000 7f47db7fe640 1 -- 172.21.15.3:0/3833636144 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47dc01a070 con 0x7f47e41408f0 2024-05-09T06:36:52.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:52.605+0000 7f47db7fe640 1 -- 172.21.15.3:0/3833636144 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f47dc003bb0 con 0x7f47e41408f0 2024-05-09T06:36:52.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:52.605+0000 7f47e89a9640 1 -- 172.21.15.3:0/3833636144 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47e40adb40 con 0x7f47e41408f0 2024-05-09T06:36:52.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:52.605+0000 7f47db7fe640 1 -- 172.21.15.3:0/3833636144 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47dc004030 con 0x7f47e41408f0 2024-05-09T06:36:52.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:52.605+0000 7f47e89a9640 1 -- 172.21.15.3:0/3833636144 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47e40ae3e0 con 0x7f47e41408f0 2024-05-09T06:36:52.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:52.606+0000 7f47e89a9640 1 -- 172.21.15.3:0/3833636144 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47e40cccd0 con 0x7f47e41408f0 2024-05-09T06:36:52.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:52.606+0000 7f47db7fe640 1 -- 172.21.15.3:0/3833636144 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f47dc01d070 con 0x7f47e41408f0 2024-05-09T06:36:52.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:52.607+0000 7f47db7fe640 1 --2- 172.21.15.3:0/3833636144 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f47c406be30 0x7f47c406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:52.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:52.607+0000 7f47db7fe640 1 -- 172.21.15.3:0/3833636144 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7f47dc08adf0 con 0x7f47e41408f0 2024-05-09T06:36:52.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:52.607+0000 7f47e259c640 1 --2- 172.21.15.3:0/3833636144 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f47c406be30 0x7f47c406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:52.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:52.608+0000 7f47e259c640 1 --2- 172.21.15.3:0/3833636144 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f47c406be30 0x7f47c406e2f0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f47e40af4c0 tx=0x7f47d4009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:52.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:52.611+0000 7f47db7fe640 1 -- 172.21.15.3:0/3833636144 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f47e40cccd0 con 0x7f47e41408f0 2024-05-09T06:36:52.843 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:52.862+0000 7f47e89a9640 1 -- 172.21.15.3:0/3833636144 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f47e404f660 con 0x7f47e41408f0 2024-05-09T06:36:52.843 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:52.862+0000 7f47db7fe640 1 -- 172.21.15.3:0/3833636144 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v4) ==== 80+0+6645 (secure 0 0 0) 0x7f47dc010070 con 0x7f47e41408f0 2024-05-09T06:36:52.843 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-09T06:36:52.844 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-05-09T06:36:52.844 INFO:teuthology.orchestra.run.smithi003.stdout: "auth_dump": [ 2024-05-09T06:36:52.844 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-09T06:36:52.844 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "mds.a", 2024-05-09T06:36:52.844 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQCcbjxmIM8dLRAAn0rimccNJB0tY1uuOIpVWg==", 2024-05-09T06:36:52.844 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-05-09T06:36:52.844 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow", 2024-05-09T06:36:52.844 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow *", 2024-05-09T06:36:52.844 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow *", 2024-05-09T06:36:52.844 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2024-05-09T06:36:52.844 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-09T06:36:52.844 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:36:52.844 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-09T06:36:52.844 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "mds.b", 2024-05-09T06:36:52.845 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQCcbjxm5jUhORAAuT9nVpIYSt7HW1Bdn2Yy1g==", 2024-05-09T06:36:52.845 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-05-09T06:36:52.845 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow", 2024-05-09T06:36:52.845 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow *", 2024-05-09T06:36:52.845 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow *", 2024-05-09T06:36:52.845 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2024-05-09T06:36:52.845 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-09T06:36:52.845 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:36:52.845 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-09T06:36:52.845 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "mds.c", 2024-05-09T06:36:52.845 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQCcbjxmyJ1SNBAA9tBFwVRrtU2iascCqj9azg==", 2024-05-09T06:36:52.845 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-05-09T06:36:52.845 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow", 2024-05-09T06:36:52.845 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow *", 2024-05-09T06:36:52.845 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow *", 2024-05-09T06:36:52.846 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2024-05-09T06:36:52.846 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-09T06:36:52.846 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:36:52.846 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-09T06:36:52.846 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "mds.d", 2024-05-09T06:36:52.846 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQCdbjxm49uZBBAAgu9Zg8TxJP+hHWvgRCl90Q==", 2024-05-09T06:36:52.846 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-05-09T06:36:52.846 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow", 2024-05-09T06:36:52.846 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow *", 2024-05-09T06:36:52.846 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow *", 2024-05-09T06:36:52.846 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2024-05-09T06:36:52.846 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-09T06:36:52.846 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:36:52.846 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-09T06:36:52.847 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "osd.0", 2024-05-09T06:36:52.847 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQCgbjxmpTR+JhAAdhWCazu+ZuBtc+wZa8OgIw==", 2024-05-09T06:36:52.847 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-05-09T06:36:52.847 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow profile osd", 2024-05-09T06:36:52.847 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile osd", 2024-05-09T06:36:52.847 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2024-05-09T06:36:52.847 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-09T06:36:52.847 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:36:52.847 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-09T06:36:52.847 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "osd.1", 2024-05-09T06:36:52.847 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQCjbjxmc4w0JhAA45zp+/amPniB68hpBkaysQ==", 2024-05-09T06:36:52.847 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-05-09T06:36:52.847 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow profile osd", 2024-05-09T06:36:52.848 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile osd", 2024-05-09T06:36:52.848 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2024-05-09T06:36:52.848 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-09T06:36:52.848 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:36:52.848 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-09T06:36:52.848 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "osd.2", 2024-05-09T06:36:52.848 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQCmbjxmvvBQJRAAfDrk7T9dUnlypbAmTVp5yQ==", 2024-05-09T06:36:52.848 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-05-09T06:36:52.848 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow profile osd", 2024-05-09T06:36:52.848 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile osd", 2024-05-09T06:36:52.848 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2024-05-09T06:36:52.848 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-09T06:36:52.848 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:36:52.848 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-09T06:36:52.848 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "osd.3", 2024-05-09T06:36:52.849 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQCpbjxmEo2MJBAAS+oDw1c9dBSn5dTkfia6Kg==", 2024-05-09T06:36:52.849 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-05-09T06:36:52.849 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow profile osd", 2024-05-09T06:36:52.849 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile osd", 2024-05-09T06:36:52.849 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2024-05-09T06:36:52.849 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-09T06:36:52.849 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:36:52.849 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-09T06:36:52.849 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "osd.4", 2024-05-09T06:36:52.849 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQCvbjxm7sImKxAAT6K/yI3d38j33Kzx62v6cA==", 2024-05-09T06:36:52.849 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-05-09T06:36:52.849 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow profile osd", 2024-05-09T06:36:52.849 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile osd", 2024-05-09T06:36:52.849 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2024-05-09T06:36:52.849 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-09T06:36:52.850 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:36:52.850 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-09T06:36:52.850 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "osd.5", 2024-05-09T06:36:52.850 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQCybjxmIq2UKRAAsSo6S3rfKxiFMYJUtOGFRw==", 2024-05-09T06:36:52.850 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-05-09T06:36:52.850 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow profile osd", 2024-05-09T06:36:52.850 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile osd", 2024-05-09T06:36:52.850 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2024-05-09T06:36:52.850 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-09T06:36:52.850 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:36:52.850 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-09T06:36:52.850 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "osd.6", 2024-05-09T06:36:52.850 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQC1bjxmIJWOKBAAwvyw8lN+6rck9nRcFVV2ZA==", 2024-05-09T06:36:52.850 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-05-09T06:36:52.850 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow profile osd", 2024-05-09T06:36:52.851 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile osd", 2024-05-09T06:36:52.851 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2024-05-09T06:36:52.851 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-09T06:36:52.851 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:36:52.851 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-09T06:36:52.851 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "osd.7", 2024-05-09T06:36:52.851 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQC4bjxmc367JhAADo/hC2C8Mr3HxcxbKga10Q==", 2024-05-09T06:36:52.851 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-05-09T06:36:52.851 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow profile osd", 2024-05-09T06:36:52.851 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile osd", 2024-05-09T06:36:52.851 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2024-05-09T06:36:52.851 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-09T06:36:52.851 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:36:52.851 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-09T06:36:52.851 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "client.0", 2024-05-09T06:36:52.852 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQCdbjxmhD/JCBAAcZ8c4sZXw8rR2/s1Lr++wQ==", 2024-05-09T06:36:52.852 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-05-09T06:36:52.852 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow", 2024-05-09T06:36:52.852 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow r", 2024-05-09T06:36:52.852 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow rw", 2024-05-09T06:36:52.852 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow rwx" 2024-05-09T06:36:52.852 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-09T06:36:52.852 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:36:52.852 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-09T06:36:52.852 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "client.1", 2024-05-09T06:36:52.852 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQCdbjxmTm+qERAACTtDZrYaqM/GfaXdtuzksw==", 2024-05-09T06:36:52.852 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-05-09T06:36:52.852 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow", 2024-05-09T06:36:52.852 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow r", 2024-05-09T06:36:52.852 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow rw", 2024-05-09T06:36:52.853 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow rwx" 2024-05-09T06:36:52.853 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-09T06:36:52.853 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:36:52.853 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-09T06:36:52.853 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "client.2", 2024-05-09T06:36:52.853 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQCdbjxmSqR/DhAA3AJMR+k+5sFEe+XJ8zJrSg==", 2024-05-09T06:36:52.853 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-05-09T06:36:52.853 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow", 2024-05-09T06:36:52.853 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow r", 2024-05-09T06:36:52.853 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow rw", 2024-05-09T06:36:52.853 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow rwx" 2024-05-09T06:36:52.853 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-09T06:36:52.853 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:36:52.853 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-09T06:36:52.853 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "client.3", 2024-05-09T06:36:52.853 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQCdbjxm9ktOFxAA5520o6cXlTpMtyhm05SLfw==", 2024-05-09T06:36:52.854 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-05-09T06:36:52.854 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow", 2024-05-09T06:36:52.854 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow r", 2024-05-09T06:36:52.854 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow rw", 2024-05-09T06:36:52.854 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow rwx" 2024-05-09T06:36:52.854 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-09T06:36:52.854 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:36:52.854 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-09T06:36:52.854 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "client.admin", 2024-05-09T06:36:52.854 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQCcbjxmuj4fEBAAgLtMi8tQSm+ftWpFoXAwdw==", 2024-05-09T06:36:52.854 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-05-09T06:36:52.854 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow *", 2024-05-09T06:36:52.854 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow *", 2024-05-09T06:36:52.854 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow *", 2024-05-09T06:36:52.855 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2024-05-09T06:36:52.855 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-09T06:36:52.855 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:36:52.855 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-09T06:36:52.855 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "client.bootstrap-mds", 2024-05-09T06:36:52.855 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQC+bjxmuU5LIBAA2DwqkuDdmPOOsmTub7TYkg==", 2024-05-09T06:36:52.855 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-05-09T06:36:52.855 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile bootstrap-mds" 2024-05-09T06:36:52.855 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-09T06:36:52.855 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:36:52.855 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-09T06:36:52.855 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "client.bootstrap-mgr", 2024-05-09T06:36:52.855 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQC+bjxmylZLIBAAg5/lCZ2X6mMDjdhwoU1z2g==", 2024-05-09T06:36:52.855 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-05-09T06:36:52.855 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile bootstrap-mgr" 2024-05-09T06:36:52.856 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-09T06:36:52.856 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:36:52.856 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-09T06:36:52.856 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "client.bootstrap-osd", 2024-05-09T06:36:52.856 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQC+bjxm6V1LIBAAexhNffgXGDjFN6tPPmZRLw==", 2024-05-09T06:36:52.856 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-05-09T06:36:52.856 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile bootstrap-osd" 2024-05-09T06:36:52.856 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-09T06:36:52.856 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:36:52.856 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-09T06:36:52.856 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "client.bootstrap-rbd", 2024-05-09T06:36:52.856 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQC+bjxm+GRLIBAAYARBd5QVLDPkhsYwJQAahQ==", 2024-05-09T06:36:52.856 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-05-09T06:36:52.856 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile bootstrap-rbd" 2024-05-09T06:36:52.857 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-09T06:36:52.857 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:36:52.857 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-09T06:36:52.857 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "client.bootstrap-rbd-mirror", 2024-05-09T06:36:52.857 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQC+bjxmWWxLIBAAiM/6X1P+jb8hIcVsXYKP/Q==", 2024-05-09T06:36:52.857 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-05-09T06:36:52.857 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2024-05-09T06:36:52.857 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-09T06:36:52.857 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:36:52.857 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-09T06:36:52.857 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "client.bootstrap-rgw", 2024-05-09T06:36:52.857 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQC+bjxmxHNLIBAAn2Xeg5rlVztDbA35vagVuw==", 2024-05-09T06:36:52.857 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-05-09T06:36:52.857 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile bootstrap-rgw" 2024-05-09T06:36:52.858 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-09T06:36:52.858 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:36:52.858 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-09T06:36:52.858 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "mgr.x", 2024-05-09T06:36:52.858 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQCcbjxmCYDJKRAAY+qhWSKa8EqdbiguI2d5KA==", 2024-05-09T06:36:52.858 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-05-09T06:36:52.858 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow *", 2024-05-09T06:36:52.858 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile mgr", 2024-05-09T06:36:52.858 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2024-05-09T06:36:52.858 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-09T06:36:52.858 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:36:52.858 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-09T06:36:52.858 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "mgr.y", 2024-05-09T06:36:52.858 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQCcbjxmMadJJRAALIKKxlTHKn4WuluZ6SqzjQ==", 2024-05-09T06:36:52.858 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-05-09T06:36:52.859 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow *", 2024-05-09T06:36:52.859 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile mgr", 2024-05-09T06:36:52.859 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2024-05-09T06:36:52.859 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-09T06:36:52.859 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-09T06:36:52.859 INFO:teuthology.orchestra.run.smithi003.stdout: ] 2024-05-09T06:36:52.859 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-05-09T06:36:52.860 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:52.864+0000 7f47e89a9640 1 -- 172.21.15.3:0/3833636144 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f47c406be30 msgr2=0x7f47c406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:52.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:52.864+0000 7f47e89a9640 1 --2- 172.21.15.3:0/3833636144 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f47c406be30 0x7f47c406e2f0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f47e40af4c0 tx=0x7f47d4009190 comp rx=0 tx=0).stop 2024-05-09T06:36:52.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:52.864+0000 7f47e89a9640 1 -- 172.21.15.3:0/3833636144 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f47e41408f0 msgr2=0x7f47e41f4410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:52.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:52.864+0000 7f47e89a9640 1 --2- 172.21.15.3:0/3833636144 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f47e41408f0 0x7f47e41f4410 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f47dc009fa0 tx=0x7f47dc00d560 comp rx=0 tx=0).stop 2024-05-09T06:36:52.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:52.864+0000 7f47e89a9640 1 -- 172.21.15.3:0/3833636144 shutdown_connections 2024-05-09T06:36:52.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:52.864+0000 7f47e89a9640 1 --2- 172.21.15.3:0/3833636144 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f47c406be30 0x7f47c406e2f0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:52.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:52.864+0000 7f47e89a9640 1 --2- 172.21.15.3:0/3833636144 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f47e41408f0 0x7f47e41f4410 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:52.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:52.864+0000 7f47e89a9640 1 --2- 172.21.15.3:0/3833636144 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f47e413e0c0 0x7f47e413d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:52.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:52.864+0000 7f47e89a9640 1 --2- 172.21.15.3:0/3833636144 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f47e40cc870 0x7f47e40aebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:52.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:52.864+0000 7f47e89a9640 1 -- 172.21.15.3:0/3833636144 >> 172.21.15.3:0/3833636144 conn(0x7f47e406b430 msgr2=0x7f47e4157a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:52.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:52.865+0000 7f47e89a9640 1 -- 172.21.15.3:0/3833636144 shutdown_connections 2024-05-09T06:36:52.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:52.865+0000 7f47e89a9640 1 -- 172.21.15.3:0/3833636144 wait complete. 2024-05-09T06:36:52.862 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2024-05-09T06:36:52.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:52.986+0000 7fa7599ed640 1 Processor -- start 2024-05-09T06:36:52.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:52.986+0000 7fa7599ed640 1 -- start start 2024-05-09T06:36:52.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:52.987+0000 7fa7599ed640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa75415e420 0x7fa7540d1750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:52.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:52.987+0000 7fa7599ed640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa7540d3f40 0x7fa7540d43f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:52.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:52.987+0000 7fa7599ed640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa7540d4930 con 0x7fa7540cfdd0 2024-05-09T06:36:52.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:52.987+0000 7fa7599ed640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fa7540cede0 con 0x7fa7540d3f40 2024-05-09T06:36:52.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:52.987+0000 7fa7599ed640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fa7540d28f0 con 0x7fa75415e420 2024-05-09T06:36:52.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:52.987+0000 7fa7537fe640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa7540d3f40 0x7fa7540d43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:52.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:52.987+0000 7fa7537fe640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa7540d3f40 0x7fa7540d43f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:56620/0 (socket says 172.21.15.3:56620) 2024-05-09T06:36:52.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:52.987+0000 7fa7537fe640 1 -- 172.21.15.3:0/321637953 learned_addr learned my addr 172.21.15.3:0/321637953 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:36:52.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:52.988+0000 7fa752ffd640 1 --2- 172.21.15.3:0/321637953 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa75415e420 0x7fa7540d1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:52.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:52.988+0000 7fa7537fe640 1 -- 172.21.15.3:0/321637953 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa75415e420 msgr2=0x7fa7540d1750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:52.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:52.988+0000 7fa7537fe640 1 --2- 172.21.15.3:0/321637953 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa75415e420 0x7fa7540d1750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:52.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:52.988+0000 7fa7537fe640 1 -- 172.21.15.3:0/321637953 >> v1:172.21.15.190:6789/0 conn(0x7fa7540cfdd0 legacy=0x7fa7540d2110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:52.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:52.988+0000 7fa7537fe640 1 -- 172.21.15.3:0/321637953 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7540cf010 con 0x7fa7540d3f40 2024-05-09T06:36:52.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:52.988+0000 7fa752ffd640 1 --2- 172.21.15.3:0/321637953 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa75415e420 0x7fa7540d1750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:36:52.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:52.988+0000 7fa7537fe640 1 --2- 172.21.15.3:0/321637953 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa7540d3f40 0x7fa7540d43f0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fa74c009870 tx=0x7fa74c02a200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41163b96b0ec865d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:52.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:52.989+0000 7fa751ffb640 1 -- 172.21.15.3:0/321637953 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa74c037070 con 0x7fa7540d3f40 2024-05-09T06:36:52.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:52.989+0000 7fa751ffb640 1 -- 172.21.15.3:0/321637953 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa74c032440 con 0x7fa7540d3f40 2024-05-09T06:36:52.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:52.989+0000 7fa751ffb640 1 -- 172.21.15.3:0/321637953 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa74c032810 con 0x7fa7540d3f40 2024-05-09T06:36:52.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:52.989+0000 7fa7599ed640 1 -- 172.21.15.3:0/321637953 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa7540d3f40 msgr2=0x7fa7540d43f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:52.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:52.989+0000 7fa7599ed640 1 --2- 172.21.15.3:0/321637953 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa7540d3f40 0x7fa7540d43f0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fa74c009870 tx=0x7fa74c02a200 comp rx=0 tx=0).stop 2024-05-09T06:36:52.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:52.989+0000 7fa7599ed640 1 -- 172.21.15.3:0/321637953 shutdown_connections 2024-05-09T06:36:52.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:52.989+0000 7fa7599ed640 1 --2- 172.21.15.3:0/321637953 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa7540d3f40 0x7fa7540d43f0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:52.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:52.989+0000 7fa7599ed640 1 --2- 172.21.15.3:0/321637953 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa75415e420 0x7fa7540d1750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:52.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:52.989+0000 7fa7599ed640 1 -- 172.21.15.3:0/321637953 >> 172.21.15.3:0/321637953 conn(0x7fa75408dbc0 msgr2=0x7fa75409d4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:52.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:52.989+0000 7fa7599ed640 1 -- 172.21.15.3:0/321637953 shutdown_connections 2024-05-09T06:36:52.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:52.989+0000 7fa7599ed640 1 -- 172.21.15.3:0/321637953 wait complete. 2024-05-09T06:36:52.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:52.990+0000 7fa7599ed640 1 Processor -- start 2024-05-09T06:36:52.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:52.991+0000 7fa7599ed640 1 -- start start 2024-05-09T06:36:52.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:52.991+0000 7fa7599ed640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa7540cfdd0 0x7fa754154dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:52.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:52.991+0000 7fa7599ed640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa7540d3f40 0x7fa754155640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:52.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:52.991+0000 7fa7599ed640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa75415e420 0x7fa7540cc5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:52.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:52.991+0000 7fa7599ed640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fa7540cf8b0 con 0x7fa7540d3f40 2024-05-09T06:36:52.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:52.991+0000 7fa7599ed640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fa7540cf5d0 con 0x7fa75415e420 2024-05-09T06:36:52.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:52.991+0000 7fa7599ed640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fa75415e800 con 0x7fa7540cfdd0 2024-05-09T06:36:52.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:52.992+0000 7fa7527fc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa7540d3f40 0x7fa754155640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:52.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:52.992+0000 7fa7527fc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa7540d3f40 0x7fa754155640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:56634/0 (socket says 172.21.15.3:56634) 2024-05-09T06:36:52.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:52.992+0000 7fa7527fc640 1 -- 172.21.15.3:0/1134389867 learned_addr learned my addr 172.21.15.3:0/1134389867 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:36:52.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:52.992+0000 7fa752ffd640 1 --2- 172.21.15.3:0/1134389867 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa7540cfdd0 0x7fa754154dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:52.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:52.992+0000 7fa7537fe640 1 --2- 172.21.15.3:0/1134389867 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa75415e420 0x7fa7540cc5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:52.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:52.992+0000 7fa7527fc640 1 -- 172.21.15.3:0/1134389867 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa7540cfdd0 msgr2=0x7fa754154dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:52.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:52.992+0000 7fa7527fc640 1 --2- 172.21.15.3:0/1134389867 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa7540cfdd0 0x7fa754154dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:52.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:52.992+0000 7fa7527fc640 1 -- 172.21.15.3:0/1134389867 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa75415e420 msgr2=0x7fa7540cc5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:52.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:52.992+0000 7fa7527fc640 1 --2- 172.21.15.3:0/1134389867 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa75415e420 0x7fa7540cc5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:52.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:52.992+0000 7fa7527fc640 1 -- 172.21.15.3:0/1134389867 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7540cb530 con 0x7fa7540d3f40 2024-05-09T06:36:52.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:52.992+0000 7fa7527fc640 1 --2- 172.21.15.3:0/1134389867 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa7540d3f40 0x7fa754155640 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fa73c00c400 tx=0x7fa73c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:52.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:52.992+0000 7fa74bfff640 1 -- 172.21.15.3:0/1134389867 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa73c011790 con 0x7fa7540d3f40 2024-05-09T06:36:52.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:52.992+0000 7fa74bfff640 1 -- 172.21.15.3:0/1134389867 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa73c017070 con 0x7fa7540d3f40 2024-05-09T06:36:52.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:52.992+0000 7fa7599ed640 1 -- 172.21.15.3:0/1134389867 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7540cb810 con 0x7fa7540d3f40 2024-05-09T06:36:52.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:52.992+0000 7fa74bfff640 1 -- 172.21.15.3:0/1134389867 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa73c011bb0 con 0x7fa7540d3f40 2024-05-09T06:36:52.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:52.992+0000 7fa7599ed640 1 -- 172.21.15.3:0/1134389867 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7540cbdd0 con 0x7fa7540d3f40 2024-05-09T06:36:52.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:52.993+0000 7fa7599ed640 1 -- 172.21.15.3:0/1134389867 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa754069550 con 0x7fa7540d3f40 2024-05-09T06:36:52.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:52.994+0000 7fa74bfff640 1 -- 172.21.15.3:0/1134389867 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa73c01c460 con 0x7fa7540d3f40 2024-05-09T06:36:52.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:52.994+0000 7fa74bfff640 1 --2- 172.21.15.3:0/1134389867 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fa73406be30 0x7fa73406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:52.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:52.994+0000 7fa74bfff640 1 -- 172.21.15.3:0/1134389867 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7fa73c08a850 con 0x7fa7540d3f40 2024-05-09T06:36:52.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:52.995+0000 7fa752ffd640 1 --2- 172.21.15.3:0/1134389867 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fa73406be30 0x7fa73406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:52.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:52.995+0000 7fa752ffd640 1 --2- 172.21.15.3:0/1134389867 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fa73406be30 0x7fa73406e2f0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fa74400f500 tx=0x7fa744007770 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:52.979 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:52.999+0000 7fa74bfff640 1 -- 172.21.15.3:0/1134389867 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fa73c058ef0 con 0x7fa7540d3f40 2024-05-09T06:36:53.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:53.253+0000 7fa7599ed640 1 -- 172.21.15.3:0/1134389867 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) -- 0x7fa7540b5060 con 0x7fa7540d3f40 2024-05-09T06:36:53.235 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:53.255+0000 7fa74bfff640 1 -- 172.21.15.3:0/1134389867 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v5) ==== 90+0+0 (secure 0 0 0) 0x7fa73c05d1f0 con 0x7fa7540d3f40 2024-05-09T06:36:53.238 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:53.257+0000 7fa7599ed640 1 -- 172.21.15.3:0/1134389867 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fa73406be30 msgr2=0x7fa73406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:53.238 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:53.257+0000 7fa7599ed640 1 --2- 172.21.15.3:0/1134389867 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fa73406be30 0x7fa73406e2f0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fa74400f500 tx=0x7fa744007770 comp rx=0 tx=0).stop 2024-05-09T06:36:53.238 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:53.257+0000 7fa7599ed640 1 -- 172.21.15.3:0/1134389867 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa7540d3f40 msgr2=0x7fa754155640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:53.238 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:53.257+0000 7fa7599ed640 1 --2- 172.21.15.3:0/1134389867 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa7540d3f40 0x7fa754155640 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fa73c00c400 tx=0x7fa73c00c8c0 comp rx=0 tx=0).stop 2024-05-09T06:36:53.239 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:53.257+0000 7fa7599ed640 1 -- 172.21.15.3:0/1134389867 shutdown_connections 2024-05-09T06:36:53.239 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:53.257+0000 7fa7599ed640 1 --2- 172.21.15.3:0/1134389867 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fa73406be30 0x7fa73406e2f0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:53.239 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:53.257+0000 7fa7599ed640 1 --2- 172.21.15.3:0/1134389867 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa75415e420 0x7fa7540cc5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:53.239 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:53.257+0000 7fa7599ed640 1 --2- 172.21.15.3:0/1134389867 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa7540d3f40 0x7fa754155640 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:53.239 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:53.257+0000 7fa7599ed640 1 --2- 172.21.15.3:0/1134389867 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa7540cfdd0 0x7fa754154dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:53.239 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:53.257+0000 7fa7599ed640 1 -- 172.21.15.3:0/1134389867 >> 172.21.15.3:0/1134389867 conn(0x7fa75408dbc0 msgr2=0x7fa7540d2110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:53.239 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:53.257+0000 7fa7599ed640 1 -- 172.21.15.3:0/1134389867 shutdown_connections 2024-05-09T06:36:53.239 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:53.257+0000 7fa7599ed640 1 -- 172.21.15.3:0/1134389867 wait complete. 2024-05-09T06:36:53.249 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2024-05-09T06:36:53.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:53.373+0000 7fbeba142640 1 Processor -- start 2024-05-09T06:36:53.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:53.373+0000 7fbeba142640 1 -- start start 2024-05-09T06:36:53.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:53.374+0000 7fbeba142640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fbeb415c0f0 0x7fbeb4160730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:53.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:53.374+0000 7fbeba142640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbeb40b4e30 0x7fbeb406e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:53.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:53.374+0000 7fbeba142640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbeb4160c70 con 0x7fbeb406e790 2024-05-09T06:36:53.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:53.374+0000 7fbeba142640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7fbeb406ec00 con 0x7fbeb415c0f0 2024-05-09T06:36:53.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:53.374+0000 7fbeba142640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fbeb40b40f0 con 0x7fbeb40b4e30 2024-05-09T06:36:53.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:53.374+0000 7fbeb3fff640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fbeb406e790 0x7fbeb40cd5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:53350/0 (socket says 172.21.15.3:53350) 2024-05-09T06:36:53.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:53.374+0000 7fbeb3fff640 1 -- 172.21.15.3:0/4210525831 learned_addr learned my addr 172.21.15.3:0/4210525831 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-05-09T06:36:53.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:53.375+0000 7fbeb2ffd640 1 --2- 172.21.15.3:0/4210525831 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbeb40b4e30 0x7fbeb406e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:53.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:53.375+0000 7fbeb37fe640 1 --2- 172.21.15.3:0/4210525831 >> v2:172.21.15.190:3300/0 conn(0x7fbeb415c0f0 0x7fbeb4160730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:53.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:53.375+0000 7fbeb27fc640 1 -- 172.21.15.3:0/4210525831 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 885725891 0 0) 0x7fbeac001490 con 0x7fbeb406e790 2024-05-09T06:36:53.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:53.375+0000 7fbeb27fc640 1 -- 172.21.15.3:0/4210525831 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbe9c003520 con 0x7fbeb406e790 2024-05-09T06:36:53.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:53.375+0000 7fbeb27fc640 1 -- 172.21.15.3:0/4210525831 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 634946930 0 0) 0x7fbe9c003520 con 0x7fbeb406e790 2024-05-09T06:36:53.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:53.375+0000 7fbeb27fc640 1 -- 172.21.15.3:0/4210525831 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbeb40b4e30 msgr2=0x7fbeb406e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:53.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:53.375+0000 7fbeb27fc640 1 --2- 172.21.15.3:0/4210525831 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbeb40b4e30 0x7fbeb406e250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:53.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:53.375+0000 7fbeb27fc640 1 -- 172.21.15.3:0/4210525831 >> v2:172.21.15.190:3300/0 conn(0x7fbeb415c0f0 msgr2=0x7fbeb4160730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:53.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:53.375+0000 7fbeb27fc640 1 --2- 172.21.15.3:0/4210525831 >> v2:172.21.15.190:3300/0 conn(0x7fbeb415c0f0 0x7fbeb4160730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:53.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:53.375+0000 7fbeb27fc640 1 -- 172.21.15.3:0/4210525831 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbeb40b3050 con 0x7fbeb406e790 2024-05-09T06:36:53.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:53.375+0000 7fbeb27fc640 1 -- 172.21.15.3:0/4210525831 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 563312031 0 0) 0x7fbeac001bc0 con 0x7fbeb406e790 2024-05-09T06:36:53.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:53.375+0000 7fbeb27fc640 1 -- 172.21.15.3:0/4210525831 >> v1:172.21.15.3:6789/0 conn(0x7fbeb406e790 legacy=0x7fbeb40cd5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:53.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:53.375+0000 7fbeb27fc640 1 --2- 172.21.15.3:0/4210525831 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbe9c0039b0 0x7fbe9c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:53.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:53.375+0000 7fbeb27fc640 1 --2- 172.21.15.3:0/4210525831 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbe9c004800 0x7fbe9c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:53.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:53.375+0000 7fbeb27fc640 1 --2- 172.21.15.3:0/4210525831 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbeb40b4e30 0x7fbe9c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:53.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:53.375+0000 7fbeb27fc640 1 -- 172.21.15.3:0/4210525831 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fbeb406ec00 con 0x7fbe9c004800 2024-05-09T06:36:53.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:53.375+0000 7fbeb27fc640 1 -- 172.21.15.3:0/4210525831 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fbeb40b40f0 con 0x7fbe9c0039b0 2024-05-09T06:36:53.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:53.375+0000 7fbeb27fc640 1 -- 172.21.15.3:0/4210525831 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fbe9c00b9c0 con 0x7fbeb40b4e30 2024-05-09T06:36:53.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:53.375+0000 7fbeb2ffd640 1 --2- 172.21.15.3:0/4210525831 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbe9c004800 0x7fbe9c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:53.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:53.375+0000 7fbeb37fe640 1 --2- 172.21.15.3:0/4210525831 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbe9c0039b0 0x7fbe9c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:53.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:53.375+0000 7fbeb3fff640 1 --2- 172.21.15.3:0/4210525831 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbeb40b4e30 0x7fbe9c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:53.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:53.375+0000 7fbeb2ffd640 1 -- 172.21.15.3:0/4210525831 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbeb40b4e30 msgr2=0x7fbe9c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:53.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:53.375+0000 7fbeb2ffd640 1 --2- 172.21.15.3:0/4210525831 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbeb40b4e30 0x7fbe9c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:53.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:53.375+0000 7fbeb2ffd640 1 -- 172.21.15.3:0/4210525831 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbe9c0039b0 msgr2=0x7fbe9c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:53.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:53.375+0000 7fbeb2ffd640 1 --2- 172.21.15.3:0/4210525831 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbe9c0039b0 0x7fbe9c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:53.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:53.375+0000 7fbeb2ffd640 1 -- 172.21.15.3:0/4210525831 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe9c003520 con 0x7fbe9c004800 2024-05-09T06:36:53.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:53.376+0000 7fbeb2ffd640 1 --2- 172.21.15.3:0/4210525831 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbe9c004800 0x7fbe9c004c80 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fbea8009af0 tx=0x7fbe9c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f77125ea6d25ecd server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:53.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:53.376+0000 7fbeb27fc640 1 -- 172.21.15.3:0/4210525831 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbea8036070 con 0x7fbe9c004800 2024-05-09T06:36:53.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:53.376+0000 7fbeb27fc640 1 -- 172.21.15.3:0/4210525831 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbea8031400 con 0x7fbe9c004800 2024-05-09T06:36:53.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:53.376+0000 7fbeb27fc640 1 -- 172.21.15.3:0/4210525831 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbea8009690 con 0x7fbe9c004800 2024-05-09T06:36:53.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:53.377+0000 7fbeba142640 1 -- 172.21.15.3:0/4210525831 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbe9c004800 msgr2=0x7fbe9c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:53.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:53.377+0000 7fbeba142640 1 --2- 172.21.15.3:0/4210525831 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbe9c004800 0x7fbe9c004c80 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fbea8009af0 tx=0x7fbe9c005830 comp rx=0 tx=0).stop 2024-05-09T06:36:53.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:53.377+0000 7fbeb3fff640 1 -- 172.21.15.3:0/4210525831 reap_dead start 2024-05-09T06:36:53.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:53.377+0000 7fbeba142640 1 -- 172.21.15.3:0/4210525831 shutdown_connections 2024-05-09T06:36:53.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:53.377+0000 7fbeba142640 1 -- 172.21.15.3:0/4210525831 >> 172.21.15.3:0/4210525831 conn(0x7fbeb40b1c20 msgr2=0x7fbeb40b2030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:53.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:53.377+0000 7fbeba142640 1 -- 172.21.15.3:0/4210525831 shutdown_connections 2024-05-09T06:36:53.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:53.377+0000 7fbeba142640 1 -- 172.21.15.3:0/4210525831 wait complete. 2024-05-09T06:36:53.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:53.378+0000 7fbeba142640 1 Processor -- start 2024-05-09T06:36:53.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:53.378+0000 7fbeba142640 1 -- start start 2024-05-09T06:36:53.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:53.379+0000 7fbeba142640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbe9c004800 0x7fbeb4150390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:53.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:53.379+0000 7fbeba142640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbeb414aa40 0x7fbeb40722a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:53.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:53.379+0000 7fbeba142640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbeb415dc20 0x7fbeb40727e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:53.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:53.379+0000 7fbeba142640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fbeb40b38f0 con 0x7fbe9c004800 2024-05-09T06:36:53.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:53.379+0000 7fbeba142640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fbeb40b3610 con 0x7fbeb415dc20 2024-05-09T06:36:53.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:53.379+0000 7fbeba142640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fbeb40b3bd0 con 0x7fbeb414aa40 2024-05-09T06:36:53.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:53.379+0000 7fbeb37fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbe9c004800 0x7fbeb4150390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:53.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:53.379+0000 7fbeb37fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbe9c004800 0x7fbeb4150390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:56652/0 (socket says 172.21.15.3:56652) 2024-05-09T06:36:53.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:53.379+0000 7fbeb37fe640 1 -- 172.21.15.3:0/3933344967 learned_addr learned my addr 172.21.15.3:0/3933344967 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:36:53.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:53.379+0000 7fbeb2ffd640 1 --2- 172.21.15.3:0/3933344967 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbeb414aa40 0x7fbeb40722a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:53.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:53.380+0000 7fbeb3fff640 1 --2- 172.21.15.3:0/3933344967 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbeb415dc20 0x7fbeb40727e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:53.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:53.380+0000 7fbeb37fe640 1 -- 172.21.15.3:0/3933344967 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbeb414aa40 msgr2=0x7fbeb40722a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:53.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:53.380+0000 7fbeb37fe640 1 --2- 172.21.15.3:0/3933344967 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbeb414aa40 0x7fbeb40722a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:53.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:53.380+0000 7fbeb37fe640 1 -- 172.21.15.3:0/3933344967 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbeb415dc20 msgr2=0x7fbeb40727e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:53.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:53.380+0000 7fbeb37fe640 1 --2- 172.21.15.3:0/3933344967 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbeb415dc20 0x7fbeb40727e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:53.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:53.380+0000 7fbeb37fe640 1 -- 172.21.15.3:0/3933344967 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbeb415e100 con 0x7fbe9c004800 2024-05-09T06:36:53.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:53.380+0000 7fbeb2ffd640 1 --2- 172.21.15.3:0/3933344967 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbeb414aa40 0x7fbeb40722a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:36:53.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:53.380+0000 7fbeb3fff640 1 --2- 172.21.15.3:0/3933344967 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbeb415dc20 0x7fbeb40727e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:36:53.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:53.380+0000 7fbeb37fe640 1 --2- 172.21.15.3:0/3933344967 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbe9c004800 0x7fbeb4150390 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fbea4011960 tx=0x7fbea4011e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:53.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:53.380+0000 7fbe9bfff640 1 -- 172.21.15.3:0/3933344967 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbea400faf0 con 0x7fbe9c004800 2024-05-09T06:36:53.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:53.380+0000 7fbe9bfff640 1 -- 172.21.15.3:0/3933344967 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbea400fc90 con 0x7fbe9c004800 2024-05-09T06:36:53.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:53.380+0000 7fbeba142640 1 -- 172.21.15.3:0/3933344967 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbeb415e3e0 con 0x7fbe9c004800 2024-05-09T06:36:53.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:53.380+0000 7fbe9bfff640 1 -- 172.21.15.3:0/3933344967 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbea4014440 con 0x7fbe9c004800 2024-05-09T06:36:53.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:53.380+0000 7fbeba142640 1 -- 172.21.15.3:0/3933344967 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbeb415e9a0 con 0x7fbe9c004800 2024-05-09T06:36:53.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:53.381+0000 7fbeba142640 1 -- 172.21.15.3:0/3933344967 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbeb4069550 con 0x7fbe9c004800 2024-05-09T06:36:53.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:53.381+0000 7fbe9bfff640 1 -- 172.21.15.3:0/3933344967 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fbea4012040 con 0x7fbe9c004800 2024-05-09T06:36:53.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:53.382+0000 7fbe9bfff640 1 --2- 172.21.15.3:0/3933344967 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fbea006be30 0x7fbea006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:53.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:53.382+0000 7fbe9bfff640 1 -- 172.21.15.3:0/3933344967 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7fbea408b190 con 0x7fbe9c004800 2024-05-09T06:36:53.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:53.382+0000 7fbeb2ffd640 1 --2- 172.21.15.3:0/3933344967 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fbea006be30 0x7fbea006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:53.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:53.383+0000 7fbeb2ffd640 1 --2- 172.21.15.3:0/3933344967 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fbea006be30 0x7fbea006e2f0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fbea8033000 tx=0x7fbea8008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:53.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:53.386+0000 7fbe9bfff640 1 -- 172.21.15.3:0/3933344967 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fbea4019030 con 0x7fbe9c004800 2024-05-09T06:36:53.625 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:53.644+0000 7fbeba142640 1 -- 172.21.15.3:0/3933344967 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) -- 0x7fbeb414ae90 con 0x7fbe9c004800 2024-05-09T06:36:53.626 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:53.646+0000 7fbe9bfff640 1 -- 172.21.15.3:0/3933344967 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v6) ==== 90+0+0 (secure 0 0 0) 0x7fbea40600f0 con 0x7fbe9c004800 2024-05-09T06:36:53.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:53.648+0000 7fbeba142640 1 -- 172.21.15.3:0/3933344967 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fbea006be30 msgr2=0x7fbea006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:53.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:53.648+0000 7fbeba142640 1 --2- 172.21.15.3:0/3933344967 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fbea006be30 0x7fbea006e2f0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fbea8033000 tx=0x7fbea8008030 comp rx=0 tx=0).stop 2024-05-09T06:36:53.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:53.648+0000 7fbeba142640 1 -- 172.21.15.3:0/3933344967 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbe9c004800 msgr2=0x7fbeb4150390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:53.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:53.648+0000 7fbeba142640 1 --2- 172.21.15.3:0/3933344967 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbe9c004800 0x7fbeb4150390 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fbea4011960 tx=0x7fbea4011e20 comp rx=0 tx=0).stop 2024-05-09T06:36:53.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:53.648+0000 7fbeba142640 1 -- 172.21.15.3:0/3933344967 shutdown_connections 2024-05-09T06:36:53.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:53.648+0000 7fbeba142640 1 --2- 172.21.15.3:0/3933344967 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fbea006be30 0x7fbea006e2f0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:53.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:53.648+0000 7fbeba142640 1 --2- 172.21.15.3:0/3933344967 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbeb415dc20 0x7fbeb40727e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:53.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:53.648+0000 7fbeba142640 1 --2- 172.21.15.3:0/3933344967 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbeb414aa40 0x7fbeb40722a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:53.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:53.648+0000 7fbeba142640 1 --2- 172.21.15.3:0/3933344967 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbe9c004800 0x7fbeb4150390 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:53.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:53.648+0000 7fbeba142640 1 -- 172.21.15.3:0/3933344967 >> 172.21.15.3:0/3933344967 conn(0x7fbeb40b1c20 msgr2=0x7fbeb4158e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:53.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:53.648+0000 7fbeba142640 1 -- 172.21.15.3:0/3933344967 shutdown_connections 2024-05-09T06:36:53.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:53.648+0000 7fbeba142640 1 -- 172.21.15.3:0/3933344967 wait complete. 2024-05-09T06:36:53.640 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2024-05-09T06:36:53.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:53.765+0000 7f5c50bab640 1 Processor -- start 2024-05-09T06:36:53.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:53.765+0000 7f5c50bab640 1 -- start start 2024-05-09T06:36:53.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:53.766+0000 7f5c50bab640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f5c4c144a10 0x7f5c4c0cf8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:53.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:53.766+0000 7f5c50bab640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5c4c0b4130 0x7f5c4c0b45e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:53.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:53.766+0000 7f5c50bab640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5c4c071860 con 0x7f5c4c072900 2024-05-09T06:36:53.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:53.766+0000 7f5c50bab640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f5c4c072470 con 0x7f5c4c144a10 2024-05-09T06:36:53.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:53.766+0000 7f5c50bab640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f5c4c071b40 con 0x7f5c4c0b4130 2024-05-09T06:36:53.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:53.766+0000 7f5c4a59c640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f5c4c144a10 0x7f5c4c0cf8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:53.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:53.766+0000 7f5c43fff640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f5c4c072900 0x7f5c4c071080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:53362/0 (socket says 172.21.15.3:53362) 2024-05-09T06:36:53.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:53.766+0000 7f5c4a59c640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f5c4c144a10 0x7f5c4c0cf8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:56660/0 (socket says 172.21.15.3:56660) 2024-05-09T06:36:53.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:53.766+0000 7f5c43fff640 1 -- 172.21.15.3:0/864622380 learned_addr learned my addr 172.21.15.3:0/864622380 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-05-09T06:36:53.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:53.766+0000 7f5c4a59c640 1 -- 172.21.15.3:0/864622380 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5c4c0b4130 msgr2=0x7f5c4c0b45e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:53.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:53.766+0000 7f5c4ad9d640 1 --2- 172.21.15.3:0/864622380 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5c4c0b4130 0x7f5c4c0b45e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:53.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:53.766+0000 7f5c4a59c640 1 --2- 172.21.15.3:0/864622380 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5c4c0b4130 0x7f5c4c0b45e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:53.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:53.766+0000 7f5c49d9b640 1 -- 172.21.15.3:0/864622380 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3905621368 0 0) 0x7f5c4c071860 con 0x7f5c4c072900 2024-05-09T06:36:53.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:53.766+0000 7f5c4a59c640 1 -- 172.21.15.3:0/864622380 >> v1:172.21.15.3:6789/0 conn(0x7f5c4c072900 legacy=0x7f5c4c071080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:53.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:53.766+0000 7f5c4a59c640 1 -- 172.21.15.3:0/864622380 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c4c071e20 con 0x7f5c4c144a10 2024-05-09T06:36:53.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:53.767+0000 7f5c4a59c640 1 --2- 172.21.15.3:0/864622380 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5c4c144a10 0x7f5c4c0cf8a0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f5c3c0097a0 tx=0x7f5c3c02a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b13171118550a2c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:53.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:53.767+0000 7f5c49d9b640 1 -- 172.21.15.3:0/864622380 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c3c037070 con 0x7f5c4c144a10 2024-05-09T06:36:53.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:53.767+0000 7f5c49d9b640 1 -- 172.21.15.3:0/864622380 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c3c02ae50 con 0x7f5c4c144a10 2024-05-09T06:36:53.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:53.767+0000 7f5c49d9b640 1 -- 172.21.15.3:0/864622380 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c3c032660 con 0x7f5c4c144a10 2024-05-09T06:36:53.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:53.768+0000 7f5c50bab640 1 -- 172.21.15.3:0/864622380 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5c4c144a10 msgr2=0x7f5c4c0cf8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:53.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:53.768+0000 7f5c50bab640 1 --2- 172.21.15.3:0/864622380 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5c4c144a10 0x7f5c4c0cf8a0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f5c3c0097a0 tx=0x7f5c3c02a100 comp rx=0 tx=0).stop 2024-05-09T06:36:53.755 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:53.768+0000 7f5c50bab640 1 -- 172.21.15.3:0/864622380 shutdown_connections 2024-05-09T06:36:53.755 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:53.768+0000 7f5c50bab640 1 --2- 172.21.15.3:0/864622380 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5c4c0b4130 0x7f5c4c0b45e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:53.755 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:53.768+0000 7f5c50bab640 1 --2- 172.21.15.3:0/864622380 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5c4c144a10 0x7f5c4c0cf8a0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:53.755 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:53.768+0000 7f5c50bab640 1 -- 172.21.15.3:0/864622380 >> 172.21.15.3:0/864622380 conn(0x7f5c4c0b1c20 msgr2=0x7f5c4c0b2030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:53.755 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:53.768+0000 7f5c50bab640 1 -- 172.21.15.3:0/864622380 shutdown_connections 2024-05-09T06:36:53.755 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:53.768+0000 7f5c50bab640 1 -- 172.21.15.3:0/864622380 wait complete. 2024-05-09T06:36:53.755 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:53.769+0000 7f5c50bab640 1 Processor -- start 2024-05-09T06:36:53.755 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:53.769+0000 7f5c50bab640 1 -- start start 2024-05-09T06:36:53.755 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:53.770+0000 7f5c50bab640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5c4c072900 0x7f5c4c14dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:53.755 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:53.770+0000 7f5c50bab640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5c4c0b4130 0x7f5c4c14c690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:53.755 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:53.770+0000 7f5c50bab640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5c4c144a10 0x7f5c4c14e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:53.755 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:53.770+0000 7f5c50bab640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f5c4c15b750 con 0x7f5c4c0b4130 2024-05-09T06:36:53.755 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:53.770+0000 7f5c50bab640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f5c4c15b5d0 con 0x7f5c4c144a10 2024-05-09T06:36:53.755 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:53.770+0000 7f5c50bab640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f5c4c15b8d0 con 0x7f5c4c072900 2024-05-09T06:36:53.755 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:53.770+0000 7f5c43fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5c4c0b4130 0x7f5c4c14c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:53.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:53.770+0000 7f5c43fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5c4c0b4130 0x7f5c4c14c690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:56672/0 (socket says 172.21.15.3:56672) 2024-05-09T06:36:53.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:53.770+0000 7f5c43fff640 1 -- 172.21.15.3:0/561204335 learned_addr learned my addr 172.21.15.3:0/561204335 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:36:53.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:53.770+0000 7f5c4a59c640 1 --2- 172.21.15.3:0/561204335 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5c4c072900 0x7f5c4c14dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:53.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:53.770+0000 7f5c43fff640 1 -- 172.21.15.3:0/561204335 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5c4c072900 msgr2=0x7f5c4c14dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:53.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:53.770+0000 7f5c43fff640 1 --2- 172.21.15.3:0/561204335 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5c4c072900 0x7f5c4c14dfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:53.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:53.770+0000 7f5c43fff640 1 -- 172.21.15.3:0/561204335 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5c4c144a10 msgr2=0x7f5c4c14e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:53.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:53.770+0000 7f5c43fff640 1 --2- 172.21.15.3:0/561204335 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5c4c144a10 0x7f5c4c14e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:53.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:53.770+0000 7f5c43fff640 1 -- 172.21.15.3:0/561204335 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c4c14cf30 con 0x7f5c4c0b4130 2024-05-09T06:36:53.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:53.770+0000 7f5c4a59c640 1 --2- 172.21.15.3:0/561204335 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5c4c072900 0x7f5c4c14dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:36:53.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:53.771+0000 7f5c43fff640 1 --2- 172.21.15.3:0/561204335 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5c4c0b4130 0x7f5c4c14c690 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f5c3400c4b0 tx=0x7f5c3400c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:53.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:53.771+0000 7f5c42ffd640 1 -- 172.21.15.3:0/561204335 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c34018070 con 0x7f5c4c0b4130 2024-05-09T06:36:53.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:53.771+0000 7f5c42ffd640 1 -- 172.21.15.3:0/561204335 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c34007500 con 0x7f5c4c0b4130 2024-05-09T06:36:53.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:53.771+0000 7f5c50bab640 1 -- 172.21.15.3:0/561204335 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c4c14d210 con 0x7f5c4c0b4130 2024-05-09T06:36:53.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:53.771+0000 7f5c42ffd640 1 -- 172.21.15.3:0/561204335 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c34004a20 con 0x7f5c4c0b4130 2024-05-09T06:36:53.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:53.771+0000 7f5c50bab640 1 -- 172.21.15.3:0/561204335 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c4c14d7d0 con 0x7f5c4c0b4130 2024-05-09T06:36:53.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:53.772+0000 7f5c50bab640 1 -- 172.21.15.3:0/561204335 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c4c074a50 con 0x7f5c4c0b4130 2024-05-09T06:36:53.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:53.772+0000 7f5c42ffd640 1 -- 172.21.15.3:0/561204335 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5c3401b070 con 0x7f5c4c0b4130 2024-05-09T06:36:53.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:53.773+0000 7f5c42ffd640 1 --2- 172.21.15.3:0/561204335 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f5c2c06be30 0x7f5c2c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:53.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:53.773+0000 7f5c42ffd640 1 -- 172.21.15.3:0/561204335 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7f5c34089db0 con 0x7f5c4c0b4130 2024-05-09T06:36:53.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:53.773+0000 7f5c4a59c640 1 --2- 172.21.15.3:0/561204335 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f5c2c06be30 0x7f5c2c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:53.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:53.774+0000 7f5c4a59c640 1 --2- 172.21.15.3:0/561204335 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f5c2c06be30 0x7f5c2c06e2f0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f5c3c034000 tx=0x7f5c3c02e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:53.758 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:53.777+0000 7f5c42ffd640 1 -- 172.21.15.3:0/561204335 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f5c34058c00 con 0x7f5c4c0b4130 2024-05-09T06:36:54.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.029+0000 7f5c50bab640 1 -- 172.21.15.3:0/561204335 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) -- 0x7f5c4c15afc0 con 0x7f5c4c0b4130 2024-05-09T06:36:54.012 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.031+0000 7f5c42ffd640 1 -- 172.21.15.3:0/561204335 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v7) ==== 90+0+0 (secure 0 0 0) 0x7f5c34058500 con 0x7f5c4c0b4130 2024-05-09T06:36:54.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.033+0000 7f5c50bab640 1 -- 172.21.15.3:0/561204335 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f5c2c06be30 msgr2=0x7f5c2c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:54.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.033+0000 7f5c50bab640 1 --2- 172.21.15.3:0/561204335 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f5c2c06be30 0x7f5c2c06e2f0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f5c3c034000 tx=0x7f5c3c02e000 comp rx=0 tx=0).stop 2024-05-09T06:36:54.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.033+0000 7f5c50bab640 1 -- 172.21.15.3:0/561204335 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5c4c0b4130 msgr2=0x7f5c4c14c690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:54.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.033+0000 7f5c50bab640 1 --2- 172.21.15.3:0/561204335 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5c4c0b4130 0x7f5c4c14c690 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f5c3400c4b0 tx=0x7f5c3400c970 comp rx=0 tx=0).stop 2024-05-09T06:36:54.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.033+0000 7f5c50bab640 1 -- 172.21.15.3:0/561204335 shutdown_connections 2024-05-09T06:36:54.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.033+0000 7f5c50bab640 1 --2- 172.21.15.3:0/561204335 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f5c2c06be30 0x7f5c2c06e2f0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:54.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.033+0000 7f5c50bab640 1 --2- 172.21.15.3:0/561204335 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5c4c144a10 0x7f5c4c14e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:54.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.033+0000 7f5c50bab640 1 --2- 172.21.15.3:0/561204335 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5c4c0b4130 0x7f5c4c14c690 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:54.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.033+0000 7f5c50bab640 1 --2- 172.21.15.3:0/561204335 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5c4c072900 0x7f5c4c14dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:54.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.033+0000 7f5c50bab640 1 -- 172.21.15.3:0/561204335 >> 172.21.15.3:0/561204335 conn(0x7f5c4c0b1c20 msgr2=0x7f5c4c0b1460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:54.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.034+0000 7f5c50bab640 1 -- 172.21.15.3:0/561204335 shutdown_connections 2024-05-09T06:36:54.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.034+0000 7f5c50bab640 1 -- 172.21.15.3:0/561204335 wait complete. 2024-05-09T06:36:54.025 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2024-05-09T06:36:54.130 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.149+0000 7f1586d7e640 1 Processor -- start 2024-05-09T06:36:54.131 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.150+0000 7f1586d7e640 1 -- start start 2024-05-09T06:36:54.132 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.150+0000 7f1586d7e640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f1580144910 0x7f15800cf8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:54.132 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.150+0000 7f1586d7e640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f15800b4130 0x7f15800b4560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:54.132 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.150+0000 7f1586d7e640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1580144cf0 con 0x7f1580072970 2024-05-09T06:36:54.132 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.150+0000 7f1586d7e640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f15800724e0 con 0x7f15800b4130 2024-05-09T06:36:54.132 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.150+0000 7f1586d7e640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f15800b4aa0 con 0x7f1580144910 2024-05-09T06:36:54.132 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.151+0000 7f158531b640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f15800b4130 0x7f15800b4560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:54.132 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.151+0000 7f158531b640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f15800b4130 0x7f15800b4560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:56674/0 (socket says 172.21.15.3:56674) 2024-05-09T06:36:54.132 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.151+0000 7f158531b640 1 -- 172.21.15.3:0/2998883392 learned_addr learned my addr 172.21.15.3:0/2998883392 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:36:54.132 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.151+0000 7f1584b1a640 1 --2- 172.21.15.3:0/2998883392 >> v2:172.21.15.190:3300/0 conn(0x7f1580144910 0x7f15800cf8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:54.133 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.151+0000 7f158531b640 1 -- 172.21.15.3:0/2998883392 >> v2:172.21.15.190:3300/0 conn(0x7f1580144910 msgr2=0x7f15800cf8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:54.133 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.151+0000 7f158531b640 1 --2- 172.21.15.3:0/2998883392 >> v2:172.21.15.190:3300/0 conn(0x7f1580144910 0x7f15800cf8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:54.133 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.151+0000 7f158531b640 1 -- 172.21.15.3:0/2998883392 >> v1:172.21.15.190:6789/0 conn(0x7f1580072970 legacy=0x7f1580071080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:54.133 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.151+0000 7f158531b640 1 -- 172.21.15.3:0/2998883392 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15800718d0 con 0x7f15800b4130 2024-05-09T06:36:54.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.152+0000 7f158531b640 1 --2- 172.21.15.3:0/2998883392 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f15800b4130 0x7f15800b4560 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f15800b2860 tx=0x7f157802c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b15e0ff0677f155 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:54.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.152+0000 7f157f7fe640 1 -- 172.21.15.3:0/2998883392 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1578009600 con 0x7f15800b4130 2024-05-09T06:36:54.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.152+0000 7f157f7fe640 1 -- 172.21.15.3:0/2998883392 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f15780097a0 con 0x7f15800b4130 2024-05-09T06:36:54.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.152+0000 7f157f7fe640 1 -- 172.21.15.3:0/2998883392 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1578009b70 con 0x7f15800b4130 2024-05-09T06:36:54.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.152+0000 7f1586d7e640 1 -- 172.21.15.3:0/2998883392 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f15800b4130 msgr2=0x7f15800b4560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:54.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.152+0000 7f1586d7e640 1 --2- 172.21.15.3:0/2998883392 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f15800b4130 0x7f15800b4560 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f15800b2860 tx=0x7f157802c900 comp rx=0 tx=0).stop 2024-05-09T06:36:54.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.152+0000 7f1586d7e640 1 -- 172.21.15.3:0/2998883392 shutdown_connections 2024-05-09T06:36:54.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.152+0000 7f1586d7e640 1 --2- 172.21.15.3:0/2998883392 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f15800b4130 0x7f15800b4560 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:54.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.152+0000 7f1586d7e640 1 --2- 172.21.15.3:0/2998883392 >> v2:172.21.15.190:3300/0 conn(0x7f1580144910 0x7f15800cf8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:54.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.152+0000 7f1586d7e640 1 -- 172.21.15.3:0/2998883392 >> 172.21.15.3:0/2998883392 conn(0x7f15800b1c20 msgr2=0x7f15800b2030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:54.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.152+0000 7f1586d7e640 1 -- 172.21.15.3:0/2998883392 shutdown_connections 2024-05-09T06:36:54.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.152+0000 7f1586d7e640 1 -- 172.21.15.3:0/2998883392 wait complete. 2024-05-09T06:36:54.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.154+0000 7f1586d7e640 1 Processor -- start 2024-05-09T06:36:54.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.154+0000 7f1586d7e640 1 -- start start 2024-05-09T06:36:54.138 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.154+0000 7f1586d7e640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1580072970 0x7f158013dc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:54.138 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.154+0000 7f1586d7e640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f15800b4130 0x7f158015aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:54.138 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.154+0000 7f1586d7e640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1580144910 0x7f15800ad8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:54.138 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.154+0000 7f1586d7e640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f1580072170 con 0x7f15800b4130 2024-05-09T06:36:54.138 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.154+0000 7f1586d7e640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f1580071e90 con 0x7f1580072970 2024-05-09T06:36:54.138 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.154+0000 7f1586d7e640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f15800b3020 con 0x7f1580144910 2024-05-09T06:36:54.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.155+0000 7f157ffff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f15800b4130 0x7f158015aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:54.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.155+0000 7f157ffff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f15800b4130 0x7f158015aa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:56686/0 (socket says 172.21.15.3:56686) 2024-05-09T06:36:54.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.155+0000 7f157ffff640 1 -- 172.21.15.3:0/2895359605 learned_addr learned my addr 172.21.15.3:0/2895359605 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:36:54.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.155+0000 7f1584b1a640 1 --2- 172.21.15.3:0/2895359605 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1580072970 0x7f158013dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:54.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.155+0000 7f158531b640 1 --2- 172.21.15.3:0/2895359605 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1580144910 0x7f15800ad8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:54.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.155+0000 7f157ffff640 1 -- 172.21.15.3:0/2895359605 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1580144910 msgr2=0x7f15800ad8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:54.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.155+0000 7f157ffff640 1 --2- 172.21.15.3:0/2895359605 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1580144910 0x7f15800ad8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:54.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.155+0000 7f157ffff640 1 -- 172.21.15.3:0/2895359605 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1580072970 msgr2=0x7f158013dc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:54.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.155+0000 7f157ffff640 1 --2- 172.21.15.3:0/2895359605 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1580072970 0x7f158013dc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:54.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.155+0000 7f157ffff640 1 -- 172.21.15.3:0/2895359605 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f158013c7b0 con 0x7f15800b4130 2024-05-09T06:36:54.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.155+0000 7f157ffff640 1 --2- 172.21.15.3:0/2895359605 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f15800b4130 0x7f158015aa00 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f156c00c460 tx=0x7f156c00c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:54.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.155+0000 7f157d7fa640 1 -- 172.21.15.3:0/2895359605 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f156c016020 con 0x7f15800b4130 2024-05-09T06:36:54.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.155+0000 7f157d7fa640 1 -- 172.21.15.3:0/2895359605 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f156c012870 con 0x7f15800b4130 2024-05-09T06:36:54.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.155+0000 7f157d7fa640 1 -- 172.21.15.3:0/2895359605 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f156c012c60 con 0x7f15800b4130 2024-05-09T06:36:54.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.155+0000 7f1586d7e640 1 -- 172.21.15.3:0/2895359605 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f158015b880 con 0x7f15800b4130 2024-05-09T06:36:54.140 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.155+0000 7f1586d7e640 1 -- 172.21.15.3:0/2895359605 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f158015be40 con 0x7f15800b4130 2024-05-09T06:36:54.140 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.156+0000 7f1586d7e640 1 -- 172.21.15.3:0/2895359605 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f158006ef10 con 0x7f15800b4130 2024-05-09T06:36:54.140 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.157+0000 7f157d7fa640 1 -- 172.21.15.3:0/2895359605 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f156c019020 con 0x7f15800b4130 2024-05-09T06:36:54.140 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.157+0000 7f157d7fa640 1 --2- 172.21.15.3:0/2895359605 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f156406be30 0x7f156406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:54.140 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.157+0000 7f157d7fa640 1 -- 172.21.15.3:0/2895359605 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7f156c08aa70 con 0x7f15800b4130 2024-05-09T06:36:54.140 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.158+0000 7f1584b1a640 1 --2- 172.21.15.3:0/2895359605 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f156406be30 0x7f156406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:54.140 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.158+0000 7f1584b1a640 1 --2- 172.21.15.3:0/2895359605 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f156406be30 0x7f156406e2f0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f15740079a0 tx=0x7f1574008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:54.142 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.162+0000 7f157d7fa640 1 -- 172.21.15.3:0/2895359605 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f158006ef10 con 0x7f15800b4130 2024-05-09T06:36:54.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.421+0000 7f1586d7e640 1 -- 172.21.15.3:0/2895359605 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) -- 0x7f15800cd120 con 0x7f15800b4130 2024-05-09T06:36:54.404 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.423+0000 7f157d7fa640 1 -- 172.21.15.3:0/2895359605 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v8) ==== 90+0+0 (secure 0 0 0) 0x7f156c020020 con 0x7f15800b4130 2024-05-09T06:36:54.407 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.426+0000 7f1586d7e640 1 -- 172.21.15.3:0/2895359605 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f156406be30 msgr2=0x7f156406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:54.407 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.426+0000 7f1586d7e640 1 --2- 172.21.15.3:0/2895359605 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f156406be30 0x7f156406e2f0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f15740079a0 tx=0x7f1574008040 comp rx=0 tx=0).stop 2024-05-09T06:36:54.407 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.426+0000 7f1586d7e640 1 -- 172.21.15.3:0/2895359605 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f15800b4130 msgr2=0x7f158015aa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:54.407 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.426+0000 7f1586d7e640 1 --2- 172.21.15.3:0/2895359605 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f15800b4130 0x7f158015aa00 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f156c00c460 tx=0x7f156c00c920 comp rx=0 tx=0).stop 2024-05-09T06:36:54.407 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.426+0000 7f1586d7e640 1 -- 172.21.15.3:0/2895359605 shutdown_connections 2024-05-09T06:36:54.407 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.426+0000 7f1586d7e640 1 --2- 172.21.15.3:0/2895359605 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f156406be30 0x7f156406e2f0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:54.407 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.426+0000 7f1586d7e640 1 --2- 172.21.15.3:0/2895359605 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1580144910 0x7f15800ad8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:54.407 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.426+0000 7f1586d7e640 1 --2- 172.21.15.3:0/2895359605 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f15800b4130 0x7f158015aa00 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:54.407 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.426+0000 7f1586d7e640 1 --2- 172.21.15.3:0/2895359605 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1580072970 0x7f158013dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:54.407 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.426+0000 7f1586d7e640 1 -- 172.21.15.3:0/2895359605 >> 172.21.15.3:0/2895359605 conn(0x7f15800b1c20 msgr2=0x7f158006dc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:54.407 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.426+0000 7f1586d7e640 1 -- 172.21.15.3:0/2895359605 shutdown_connections 2024-05-09T06:36:54.408 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.426+0000 7f1586d7e640 1 -- 172.21.15.3:0/2895359605 wait complete. 2024-05-09T06:36:54.417 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2024-05-09T06:36:54.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.543+0000 7f865c8c1640 1 Processor -- start 2024-05-09T06:36:54.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.543+0000 7f865c8c1640 1 -- start start 2024-05-09T06:36:54.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.543+0000 7f865c8c1640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f865815e5c0 0x7f86580d2800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:54.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.543+0000 7f865c8c1640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f86580d4c40 0x7f86580d0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:54.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.543+0000 7f865c8c1640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f86580d1130 0x7f86580ced30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:54.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.543+0000 7f865c8c1640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f865815e9a0 con 0x7f86580d4c40 2024-05-09T06:36:54.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.543+0000 7f865c8c1640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f86580d31c0 con 0x7f86580d1130 2024-05-09T06:36:54.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.543+0000 7f865c8c1640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f86580d50a0 con 0x7f865815e5c0 2024-05-09T06:36:54.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.544+0000 7f864ffff640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f86580d4c40 0x7f86580d0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:54.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.544+0000 7f864ffff640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f86580d4c40 0x7f86580d0bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:56702/0 (socket says 172.21.15.3:56702) 2024-05-09T06:36:54.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.544+0000 7f864ffff640 1 -- 172.21.15.3:0/4130458671 learned_addr learned my addr 172.21.15.3:0/4130458671 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:36:54.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.544+0000 7f865659c640 1 --2- 172.21.15.3:0/4130458671 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f865815e5c0 0x7f86580d2800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:54.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.544+0000 7f8656d9d640 1 --2- 172.21.15.3:0/4130458671 >> v2:172.21.15.190:3300/0 conn(0x7f86580d1130 0x7f86580ced30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:54.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.544+0000 7f864ffff640 1 -- 172.21.15.3:0/4130458671 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f865815e5c0 msgr2=0x7f86580d2800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:54.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.544+0000 7f864ffff640 1 --2- 172.21.15.3:0/4130458671 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f865815e5c0 0x7f86580d2800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:54.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.544+0000 7f864ffff640 1 -- 172.21.15.3:0/4130458671 >> v2:172.21.15.190:3300/0 conn(0x7f86580d1130 msgr2=0x7f86580ced30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:54.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.544+0000 7f864ffff640 1 --2- 172.21.15.3:0/4130458671 >> v2:172.21.15.190:3300/0 conn(0x7f86580d1130 0x7f86580ced30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:54.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.544+0000 7f864ffff640 1 -- 172.21.15.3:0/4130458671 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86580d2ee0 con 0x7f86580d4c40 2024-05-09T06:36:54.530 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.545+0000 7f864ffff640 1 --2- 172.21.15.3:0/4130458671 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f86580d4c40 0x7f86580d0bf0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f86580cb250 tx=0x7f864002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84fc2783f91e4c1c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:54.530 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.545+0000 7f8655d9b640 1 -- 172.21.15.3:0/4130458671 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8640037070 con 0x7f86580d4c40 2024-05-09T06:36:54.531 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.545+0000 7f8655d9b640 1 -- 172.21.15.3:0/4130458671 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8640032440 con 0x7f86580d4c40 2024-05-09T06:36:54.531 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.545+0000 7f8655d9b640 1 -- 172.21.15.3:0/4130458671 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8640032810 con 0x7f86580d4c40 2024-05-09T06:36:54.531 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.545+0000 7f865c8c1640 1 -- 172.21.15.3:0/4130458671 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f86580d4c40 msgr2=0x7f86580d0bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:54.531 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.545+0000 7f865c8c1640 1 --2- 172.21.15.3:0/4130458671 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f86580d4c40 0x7f86580d0bf0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f86580cb250 tx=0x7f864002a1e0 comp rx=0 tx=0).stop 2024-05-09T06:36:54.531 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.545+0000 7f865c8c1640 1 -- 172.21.15.3:0/4130458671 shutdown_connections 2024-05-09T06:36:54.531 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.545+0000 7f865c8c1640 1 --2- 172.21.15.3:0/4130458671 >> v2:172.21.15.190:3300/0 conn(0x7f86580d1130 0x7f86580ced30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:54.531 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.545+0000 7f865c8c1640 1 --2- 172.21.15.3:0/4130458671 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f86580d4c40 0x7f86580d0bf0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:54.531 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.545+0000 7f865c8c1640 1 --2- 172.21.15.3:0/4130458671 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f865815e5c0 0x7f86580d2800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:54.531 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.545+0000 7f865c8c1640 1 -- 172.21.15.3:0/4130458671 >> 172.21.15.3:0/4130458671 conn(0x7f865809d500 msgr2=0x7f86580d4220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:54.531 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.545+0000 7f865c8c1640 1 -- 172.21.15.3:0/4130458671 shutdown_connections 2024-05-09T06:36:54.531 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.546+0000 7f865c8c1640 1 -- 172.21.15.3:0/4130458671 wait complete. 2024-05-09T06:36:54.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.547+0000 7f865c8c1640 1 Processor -- start 2024-05-09T06:36:54.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.547+0000 7f865c8c1640 1 -- start start 2024-05-09T06:36:54.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.548+0000 7f865c8c1640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f86580d1130 0x7f86580cdfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:54.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.548+0000 7f865c8c1640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f86580d4c40 0x7f86581f7a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:54.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.548+0000 7f865c8c1640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f865815e5c0 0x7f865815c080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:54.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.548+0000 7f865c8c1640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f86580b42c0 con 0x7f86580d1130 2024-05-09T06:36:54.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.548+0000 7f865c8c1640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f86580b3fe0 con 0x7f86580d4c40 2024-05-09T06:36:54.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.548+0000 7f865c8c1640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f86580cf270 con 0x7f865815e5c0 2024-05-09T06:36:54.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.548+0000 7f865659c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f86580d1130 0x7f86580cdfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:54.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.548+0000 7f865659c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f86580d1130 0x7f86580cdfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:56704/0 (socket says 172.21.15.3:56704) 2024-05-09T06:36:54.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.548+0000 7f865659c640 1 -- 172.21.15.3:0/4062614849 learned_addr learned my addr 172.21.15.3:0/4062614849 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:36:54.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.548+0000 7f864ffff640 1 --2- 172.21.15.3:0/4062614849 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f86580d4c40 0x7f86581f7a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:54.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.548+0000 7f8656d9d640 1 --2- 172.21.15.3:0/4062614849 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f865815e5c0 0x7f865815c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:54.533 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.548+0000 7f865659c640 1 -- 172.21.15.3:0/4062614849 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f865815e5c0 msgr2=0x7f865815c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:54.533 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.548+0000 7f865659c640 1 --2- 172.21.15.3:0/4062614849 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f865815e5c0 0x7f865815c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:54.533 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.548+0000 7f865659c640 1 -- 172.21.15.3:0/4062614849 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f86580d4c40 msgr2=0x7f86581f7a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:54.533 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.548+0000 7f865659c640 1 --2- 172.21.15.3:0/4062614849 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f86580d4c40 0x7f86581f7a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:54.533 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.548+0000 7f865659c640 1 -- 172.21.15.3:0/4062614849 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86581f8220 con 0x7f86580d1130 2024-05-09T06:36:54.533 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.548+0000 7f864ffff640 1 --2- 172.21.15.3:0/4062614849 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f86580d4c40 0x7f86581f7a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:36:54.533 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.548+0000 7f8656d9d640 1 --2- 172.21.15.3:0/4062614849 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f865815e5c0 0x7f865815c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:36:54.533 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.548+0000 7f865659c640 1 --2- 172.21.15.3:0/4062614849 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f86580d1130 0x7f86580cdfd0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f864800e870 tx=0x7f864800ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:54.533 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.548+0000 7f864effd640 1 -- 172.21.15.3:0/4062614849 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8648017070 con 0x7f86580d1130 2024-05-09T06:36:54.533 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.548+0000 7f864effd640 1 -- 172.21.15.3:0/4062614849 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f864800c940 con 0x7f86580d1130 2024-05-09T06:36:54.533 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.548+0000 7f865c8c1640 1 -- 172.21.15.3:0/4062614849 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86581f8500 con 0x7f86580d1130 2024-05-09T06:36:54.533 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.548+0000 7f864effd640 1 -- 172.21.15.3:0/4062614849 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f864800cd30 con 0x7f86580d1130 2024-05-09T06:36:54.533 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.548+0000 7f865c8c1640 1 -- 172.21.15.3:0/4062614849 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f865815c5c0 con 0x7f86580d1130 2024-05-09T06:36:54.533 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.549+0000 7f865c8c1640 1 -- 172.21.15.3:0/4062614849 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8658069550 con 0x7f86580d1130 2024-05-09T06:36:54.534 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.550+0000 7f864effd640 1 -- 172.21.15.3:0/4062614849 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f864801a070 con 0x7f86580d1130 2024-05-09T06:36:54.534 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.550+0000 7f864effd640 1 --2- 172.21.15.3:0/4062614849 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f863806be30 0x7f863806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:54.534 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.551+0000 7f864effd640 1 -- 172.21.15.3:0/4062614849 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7f864808ac20 con 0x7f86580d1130 2024-05-09T06:36:54.535 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.551+0000 7f864ffff640 1 --2- 172.21.15.3:0/4062614849 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f863806be30 0x7f863806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:54.535 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.551+0000 7f864ffff640 1 --2- 172.21.15.3:0/4062614849 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f863806be30 0x7f863806e2f0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f8640034000 tx=0x7f864002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:54.535 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.555+0000 7f864effd640 1 -- 172.21.15.3:0/4062614849 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f86480599b0 con 0x7f86580d1130 2024-05-09T06:36:54.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.806+0000 7f865c8c1640 1 -- 172.21.15.3:0/4062614849 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) -- 0x7f865808def0 con 0x7f86580d1130 2024-05-09T06:36:54.788 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.808+0000 7f864effd640 1 -- 172.21.15.3:0/4062614849 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v9) ==== 97+0+0 (secure 0 0 0) 0x7f86480592b0 con 0x7f86580d1130 2024-05-09T06:36:54.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.810+0000 7f865c8c1640 1 -- 172.21.15.3:0/4062614849 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f863806be30 msgr2=0x7f863806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:54.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.810+0000 7f865c8c1640 1 --2- 172.21.15.3:0/4062614849 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f863806be30 0x7f863806e2f0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f8640034000 tx=0x7f864002e000 comp rx=0 tx=0).stop 2024-05-09T06:36:54.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.810+0000 7f865c8c1640 1 -- 172.21.15.3:0/4062614849 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f86580d1130 msgr2=0x7f86580cdfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:54.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.810+0000 7f865c8c1640 1 --2- 172.21.15.3:0/4062614849 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f86580d1130 0x7f86580cdfd0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f864800e870 tx=0x7f864800ed30 comp rx=0 tx=0).stop 2024-05-09T06:36:54.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.810+0000 7f865c8c1640 1 -- 172.21.15.3:0/4062614849 shutdown_connections 2024-05-09T06:36:54.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.810+0000 7f865c8c1640 1 --2- 172.21.15.3:0/4062614849 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f863806be30 0x7f863806e2f0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:54.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.810+0000 7f865c8c1640 1 --2- 172.21.15.3:0/4062614849 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f865815e5c0 0x7f865815c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:54.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.810+0000 7f865c8c1640 1 --2- 172.21.15.3:0/4062614849 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f86580d4c40 0x7f86581f7a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:54.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.810+0000 7f865c8c1640 1 --2- 172.21.15.3:0/4062614849 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f86580d1130 0x7f86580cdfd0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:54.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.810+0000 7f865c8c1640 1 -- 172.21.15.3:0/4062614849 >> 172.21.15.3:0/4062614849 conn(0x7f865809d500 msgr2=0x7f86580b2540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:54.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.810+0000 7f865c8c1640 1 -- 172.21.15.3:0/4062614849 shutdown_connections 2024-05-09T06:36:54.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.810+0000 7f865c8c1640 1 -- 172.21.15.3:0/4062614849 wait complete. 2024-05-09T06:36:54.802 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2024-05-09T06:36:54.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.926+0000 7f330c948640 1 Processor -- start 2024-05-09T06:36:54.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.926+0000 7f330c948640 1 -- start start 2024-05-09T06:36:54.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.927+0000 7f330c948640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f33080d2800 0x7f33080d4c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:54.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.927+0000 7f330c948640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f330815e9a0 con 0x7f33080d1300 2024-05-09T06:36:54.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.927+0000 7f330c948640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f33080d2c50 con 0x7f330815e5c0 2024-05-09T06:36:54.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.927+0000 7f330c948640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f33080d31c0 con 0x7f33080d2800 2024-05-09T06:36:54.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.927+0000 7f3305d9b640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f33080d1300 0x7f33080b47e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:53366/0 (socket says 172.21.15.3:53366) 2024-05-09T06:36:54.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.928+0000 7f3305d9b640 1 -- 172.21.15.3:0/582080064 learned_addr learned my addr 172.21.15.3:0/582080064 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-05-09T06:36:54.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.928+0000 7f330559a640 1 -- 172.21.15.3:0/582080064 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3996342326 0 0) 0x7f330815e9a0 con 0x7f33080d1300 2024-05-09T06:36:54.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.928+0000 7f330559a640 1 -- 172.21.15.3:0/582080064 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f32f4003520 con 0x7f33080d1300 2024-05-09T06:36:54.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.928+0000 7f3306d9d640 1 --2- 172.21.15.3:0/582080064 >> v2:172.21.15.190:3300/0 conn(0x7f33080d2800 0x7f33080d4c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:54.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.928+0000 7f330559a640 1 -- 172.21.15.3:0/582080064 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2047733507 0 0) 0x7f32f4003520 con 0x7f33080d1300 2024-05-09T06:36:54.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.928+0000 7f330559a640 1 -- 172.21.15.3:0/582080064 >> v2:172.21.15.190:3300/0 conn(0x7f33080d2800 msgr2=0x7f33080d4c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:54.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.928+0000 7f330559a640 1 --2- 172.21.15.3:0/582080064 >> v2:172.21.15.190:3300/0 conn(0x7f33080d2800 0x7f33080d4c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:54.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.928+0000 7f330559a640 1 -- 172.21.15.3:0/582080064 >> v1:172.21.15.190:6789/0 conn(0x7f330815e5c0 legacy=0x7f33080d0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:54.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.928+0000 7f330559a640 1 -- 172.21.15.3:0/582080064 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33080d2ee0 con 0x7f33080d1300 2024-05-09T06:36:54.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.928+0000 7f330559a640 1 -- 172.21.15.3:0/582080064 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 563312031 0 0) 0x7f32f0001670 con 0x7f33080d1300 2024-05-09T06:36:54.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.928+0000 7f330559a640 1 -- 172.21.15.3:0/582080064 >> v1:172.21.15.3:6789/0 conn(0x7f33080d1300 legacy=0x7f33080b47e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:54.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.928+0000 7f330559a640 1 --2- 172.21.15.3:0/582080064 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f32f40039b0 0x7f32f4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:54.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.928+0000 7f330559a640 1 --2- 172.21.15.3:0/582080064 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f32f4004800 0x7f32f4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:54.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.928+0000 7f330559a640 1 --2- 172.21.15.3:0/582080064 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f32f40051c0 0x7f32f400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:54.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.928+0000 7f330559a640 1 -- 172.21.15.3:0/582080064 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f33080d31c0 con 0x7f32f4004800 2024-05-09T06:36:54.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.928+0000 7f330559a640 1 -- 172.21.15.3:0/582080064 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f32f400c060 con 0x7f32f40051c0 2024-05-09T06:36:54.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.928+0000 7f330559a640 1 -- 172.21.15.3:0/582080064 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f32f400c260 con 0x7f32f40039b0 2024-05-09T06:36:54.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.928+0000 7f3305d9b640 1 --2- 172.21.15.3:0/582080064 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f32f4004800 0x7f32f4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:54.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.928+0000 7f330659c640 1 --2- 172.21.15.3:0/582080064 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f32f40039b0 0x7f32f4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:54.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.928+0000 7f3306d9d640 1 --2- 172.21.15.3:0/582080064 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f32f40051c0 0x7f32f400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:54.913 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.929+0000 7f3305d9b640 1 -- 172.21.15.3:0/582080064 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f32f40039b0 msgr2=0x7f32f4003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:54.913 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.929+0000 7f3305d9b640 1 --2- 172.21.15.3:0/582080064 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f32f40039b0 0x7f32f4003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:54.913 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.929+0000 7f3305d9b640 1 -- 172.21.15.3:0/582080064 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f32f40051c0 msgr2=0x7f32f400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:54.913 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.929+0000 7f3305d9b640 1 --2- 172.21.15.3:0/582080064 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f32f40051c0 0x7f32f400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:54.913 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.929+0000 7f3305d9b640 1 -- 172.21.15.3:0/582080064 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32f4003520 con 0x7f32f4004800 2024-05-09T06:36:54.913 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.929+0000 7f3306d9d640 1 -- 172.21.15.3:0/582080064 reap_dead start 2024-05-09T06:36:54.913 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.929+0000 7f3305d9b640 1 --2- 172.21.15.3:0/582080064 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f32f4004800 0x7f32f4004c80 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f32f000c240 tx=0x7f32f4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f98274086119b607 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:54.913 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.929+0000 7f330559a640 1 -- 172.21.15.3:0/582080064 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32f0007d10 con 0x7f32f4004800 2024-05-09T06:36:54.913 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.929+0000 7f330559a640 1 -- 172.21.15.3:0/582080064 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32f0005ad0 con 0x7f32f4004800 2024-05-09T06:36:54.913 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.929+0000 7f330559a640 1 -- 172.21.15.3:0/582080064 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32f0007070 con 0x7f32f4004800 2024-05-09T06:36:54.913 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.930+0000 7f330c948640 1 -- 172.21.15.3:0/582080064 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f32f4004800 msgr2=0x7f32f4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:54.914 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.930+0000 7f330c948640 1 --2- 172.21.15.3:0/582080064 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f32f4004800 0x7f32f4004c80 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f32f000c240 tx=0x7f32f4005830 comp rx=0 tx=0).stop 2024-05-09T06:36:54.914 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.930+0000 7f330c948640 1 -- 172.21.15.3:0/582080064 shutdown_connections 2024-05-09T06:36:54.914 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.930+0000 7f330c948640 1 --2- 172.21.15.3:0/582080064 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f32f4004800 0x7f32f4004c80 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:54.914 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.930+0000 7f330c948640 1 -- 172.21.15.3:0/582080064 >> 172.21.15.3:0/582080064 conn(0x7f330809d500 msgr2=0x7f33080d4220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:54.914 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.930+0000 7f330c948640 1 -- 172.21.15.3:0/582080064 shutdown_connections 2024-05-09T06:36:54.914 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.930+0000 7f330c948640 1 -- 172.21.15.3:0/582080064 wait complete. 2024-05-09T06:36:54.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.931+0000 7f330c948640 1 Processor -- start 2024-05-09T06:36:54.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.931+0000 7f330c948640 1 -- start start 2024-05-09T06:36:54.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.932+0000 7f330c948640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f32f4004800 0x7f33081536b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:54.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.932+0000 7f330c948640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f33080a17a0 0x7f33081f4230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:54.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.932+0000 7f330c948640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f33080a1c20 0x7f33081f4770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:54.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.932+0000 7f330c948640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f33080b42c0 con 0x7f32f4004800 2024-05-09T06:36:54.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.932+0000 7f330c948640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f33080b3fe0 con 0x7f33080a1c20 2024-05-09T06:36:54.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.932+0000 7f330c948640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f33080d4c40 con 0x7f33080a17a0 2024-05-09T06:36:54.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.932+0000 7f330659c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f32f4004800 0x7f33081536b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:54.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.932+0000 7f330659c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f32f4004800 0x7f33081536b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:56720/0 (socket says 172.21.15.3:56720) 2024-05-09T06:36:54.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.932+0000 7f330659c640 1 -- 172.21.15.3:0/1077129832 learned_addr learned my addr 172.21.15.3:0/1077129832 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:36:54.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.932+0000 7f3306d9d640 1 --2- 172.21.15.3:0/1077129832 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f33080a1c20 0x7f33081f4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:54.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.933+0000 7f3305d9b640 1 --2- 172.21.15.3:0/1077129832 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f33080a17a0 0x7f33081f4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:54.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.933+0000 7f330659c640 1 -- 172.21.15.3:0/1077129832 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f33080a17a0 msgr2=0x7f33081f4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:54.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.933+0000 7f330659c640 1 --2- 172.21.15.3:0/1077129832 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f33080a17a0 0x7f33081f4230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:54.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.933+0000 7f330659c640 1 -- 172.21.15.3:0/1077129832 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f33080a1c20 msgr2=0x7f33081f4770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:54.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.933+0000 7f330659c640 1 --2- 172.21.15.3:0/1077129832 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f33080a1c20 0x7f33081f4770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:54.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.933+0000 7f330659c640 1 -- 172.21.15.3:0/1077129832 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f330809f930 con 0x7f32f4004800 2024-05-09T06:36:54.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.933+0000 7f3306d9d640 1 --2- 172.21.15.3:0/1077129832 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f33080a1c20 0x7f33081f4770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:36:54.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.933+0000 7f330659c640 1 --2- 172.21.15.3:0/1077129832 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f32f4004800 0x7f33081536b0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f330000fb90 tx=0x7f330000d520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:54.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.933+0000 7f32feffd640 1 -- 172.21.15.3:0/1077129832 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f330000fe90 con 0x7f32f4004800 2024-05-09T06:36:54.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.933+0000 7f32feffd640 1 -- 172.21.15.3:0/1077129832 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3300018070 con 0x7f32f4004800 2024-05-09T06:36:54.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.933+0000 7f330c948640 1 -- 172.21.15.3:0/1077129832 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f330809fc10 con 0x7f32f4004800 2024-05-09T06:36:54.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.933+0000 7f32feffd640 1 -- 172.21.15.3:0/1077129832 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f330000ddf0 con 0x7f32f4004800 2024-05-09T06:36:54.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.933+0000 7f330c948640 1 -- 172.21.15.3:0/1077129832 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33080a01d0 con 0x7f32f4004800 2024-05-09T06:36:54.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.934+0000 7f330c948640 1 -- 172.21.15.3:0/1077129832 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3308069550 con 0x7f32f4004800 2024-05-09T06:36:54.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.935+0000 7f32feffd640 1 -- 172.21.15.3:0/1077129832 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3300003970 con 0x7f32f4004800 2024-05-09T06:36:54.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.935+0000 7f32feffd640 1 --2- 172.21.15.3:0/1077129832 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f32e806c000 0x7f32e806e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:54.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.935+0000 7f32feffd640 1 -- 172.21.15.3:0/1077129832 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7f330008abc0 con 0x7f32f4004800 2024-05-09T06:36:54.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.935+0000 7f3305d9b640 1 --2- 172.21.15.3:0/1077129832 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f32e806c000 0x7f32e806e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:54.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.936+0000 7f3305d9b640 1 --2- 172.21.15.3:0/1077129832 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f32e806c000 0x7f32e806e4c0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f32f0032000 tx=0x7f32f0034040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:54.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:54.939+0000 7f32feffd640 1 -- 172.21.15.3:0/1077129832 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f3300059950 con 0x7f32f4004800 2024-05-09T06:36:55.174 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.193+0000 7f330c948640 1 -- 172.21.15.3:0/1077129832 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) -- 0x7f33080d10e0 con 0x7f32f4004800 2024-05-09T06:36:55.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.196+0000 7f32feffd640 1 -- 172.21.15.3:0/1077129832 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v10) ==== 90+0+0 (secure 0 0 0) 0x7f3300059250 con 0x7f32f4004800 2024-05-09T06:36:55.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.198+0000 7f330c948640 1 -- 172.21.15.3:0/1077129832 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f32e806c000 msgr2=0x7f32e806e4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:55.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.198+0000 7f330c948640 1 --2- 172.21.15.3:0/1077129832 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f32e806c000 0x7f32e806e4c0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f32f0032000 tx=0x7f32f0034040 comp rx=0 tx=0).stop 2024-05-09T06:36:55.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.198+0000 7f330c948640 1 -- 172.21.15.3:0/1077129832 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f32f4004800 msgr2=0x7f33081536b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:55.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.198+0000 7f330c948640 1 --2- 172.21.15.3:0/1077129832 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f32f4004800 0x7f33081536b0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f330000fb90 tx=0x7f330000d520 comp rx=0 tx=0).stop 2024-05-09T06:36:55.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.198+0000 7f330c948640 1 -- 172.21.15.3:0/1077129832 shutdown_connections 2024-05-09T06:36:55.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.198+0000 7f330c948640 1 --2- 172.21.15.3:0/1077129832 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f32e806c000 0x7f32e806e4c0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:55.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.198+0000 7f330c948640 1 --2- 172.21.15.3:0/1077129832 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f33080a1c20 0x7f33081f4770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:55.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.198+0000 7f330c948640 1 --2- 172.21.15.3:0/1077129832 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f33080a17a0 0x7f33081f4230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:55.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.198+0000 7f330c948640 1 --2- 172.21.15.3:0/1077129832 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f32f4004800 0x7f33081536b0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:55.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.198+0000 7f330c948640 1 -- 172.21.15.3:0/1077129832 >> 172.21.15.3:0/1077129832 conn(0x7f330809d500 msgr2=0x7f33080d0980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:55.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.198+0000 7f330c948640 1 -- 172.21.15.3:0/1077129832 shutdown_connections 2024-05-09T06:36:55.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.198+0000 7f330c948640 1 -- 172.21.15.3:0/1077129832 wait complete. 2024-05-09T06:36:55.190 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T06:36:55.295 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.314+0000 7f8a59e5d640 1 Processor -- start 2024-05-09T06:36:55.295 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.314+0000 7f8a59e5d640 1 -- start start 2024-05-09T06:36:55.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.315+0000 7f8a59e5d640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8a5406d8f0 0x7f8a5406bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:55.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.315+0000 7f8a59e5d640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8a5406cab0 con 0x7f8a5415e100 2024-05-09T06:36:55.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.315+0000 7f8a59e5d640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8a540b4120 con 0x7f8a540ae9c0 2024-05-09T06:36:55.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.315+0000 7f8a59e5d640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f8a540b3ed0 con 0x7f8a5406d8f0 2024-05-09T06:36:55.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.315+0000 7f8a537fe640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f8a5415e100 0x7f8a540ae2b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:53376/0 (socket says 172.21.15.3:53376) 2024-05-09T06:36:55.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.315+0000 7f8a537fe640 1 -- 172.21.15.3:0/1658354058 learned_addr learned my addr 172.21.15.3:0/1658354058 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-05-09T06:36:55.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.315+0000 7f8a527fc640 1 -- 172.21.15.3:0/1658354058 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4051482572 0 0) 0x7f8a5406cab0 con 0x7f8a5415e100 2024-05-09T06:36:55.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.315+0000 7f8a52ffd640 1 --2- 172.21.15.3:0/1658354058 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8a5406d8f0 0x7f8a5406bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:55.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.315+0000 7f8a527fc640 1 -- 172.21.15.3:0/1658354058 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8a3c003520 con 0x7f8a5415e100 2024-05-09T06:36:55.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.315+0000 7f8a527fc640 1 -- 172.21.15.3:0/1658354058 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 970004858 0 0) 0x7f8a3c003520 con 0x7f8a5415e100 2024-05-09T06:36:55.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.315+0000 7f8a527fc640 1 -- 172.21.15.3:0/1658354058 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8a5406d8f0 msgr2=0x7f8a5406bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:55.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.315+0000 7f8a527fc640 1 --2- 172.21.15.3:0/1658354058 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8a5406d8f0 0x7f8a5406bcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:55.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.315+0000 7f8a527fc640 1 -- 172.21.15.3:0/1658354058 >> v1:172.21.15.190:6789/0 conn(0x7f8a540ae9c0 legacy=0x7f8a5406c230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:55.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.316+0000 7f8a527fc640 1 -- 172.21.15.3:0/1658354058 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a540b4400 con 0x7f8a5415e100 2024-05-09T06:36:55.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.316+0000 7f8a527fc640 1 -- 172.21.15.3:0/1658354058 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 563312031 0 0) 0x7f8a44001670 con 0x7f8a5415e100 2024-05-09T06:36:55.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.316+0000 7f8a527fc640 1 -- 172.21.15.3:0/1658354058 >> v1:172.21.15.3:6789/0 conn(0x7f8a5415e100 legacy=0x7f8a540ae2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:55.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.316+0000 7f8a527fc640 1 --2- 172.21.15.3:0/1658354058 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8a3c0039e0 0x7f8a3c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:55.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.316+0000 7f8a527fc640 1 --2- 172.21.15.3:0/1658354058 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8a5406d8f0 0x7f8a3c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:55.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.316+0000 7f8a527fc640 1 --2- 172.21.15.3:0/1658354058 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8a3c004d40 0x7f8a3c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:55.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.316+0000 7f8a527fc640 1 -- 172.21.15.3:0/1658354058 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f8a540b3ed0 con 0x7f8a3c0039e0 2024-05-09T06:36:55.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.316+0000 7f8a527fc640 1 -- 172.21.15.3:0/1658354058 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f8a3c00ba70 con 0x7f8a3c004d40 2024-05-09T06:36:55.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.316+0000 7f8a527fc640 1 -- 172.21.15.3:0/1658354058 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f8a3c00bc70 con 0x7f8a5406d8f0 2024-05-09T06:36:55.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.316+0000 7f8a537fe640 1 --2- 172.21.15.3:0/1658354058 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8a3c0039e0 0x7f8a3c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:55.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.316+0000 7f8a52ffd640 1 --2- 172.21.15.3:0/1658354058 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8a5406d8f0 0x7f8a3c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:55.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.316+0000 7f8a53fff640 1 --2- 172.21.15.3:0/1658354058 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8a3c004d40 0x7f8a3c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:55.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.316+0000 7f8a537fe640 1 -- 172.21.15.3:0/1658354058 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8a5406d8f0 msgr2=0x7f8a3c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:55.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.316+0000 7f8a537fe640 1 --2- 172.21.15.3:0/1658354058 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8a5406d8f0 0x7f8a3c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:55.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.316+0000 7f8a537fe640 1 -- 172.21.15.3:0/1658354058 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8a3c004d40 msgr2=0x7f8a3c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:55.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.316+0000 7f8a537fe640 1 --2- 172.21.15.3:0/1658354058 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8a3c004d40 0x7f8a3c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:55.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.316+0000 7f8a537fe640 1 -- 172.21.15.3:0/1658354058 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a3c003520 con 0x7f8a3c0039e0 2024-05-09T06:36:55.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.317+0000 7f8a537fe640 1 --2- 172.21.15.3:0/1658354058 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8a3c0039e0 0x7f8a3c003de0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f8a4400c240 tx=0x7f8a3c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=850c3e8cb98e4e21 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:55.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.317+0000 7f8a527fc640 1 -- 172.21.15.3:0/1658354058 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a44007d10 con 0x7f8a3c0039e0 2024-05-09T06:36:55.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.317+0000 7f8a527fc640 1 -- 172.21.15.3:0/1658354058 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a44005ad0 con 0x7f8a3c0039e0 2024-05-09T06:36:55.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.317+0000 7f8a527fc640 1 -- 172.21.15.3:0/1658354058 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a44007070 con 0x7f8a3c0039e0 2024-05-09T06:36:55.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.317+0000 7f8a59e5d640 1 -- 172.21.15.3:0/1658354058 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8a3c0039e0 msgr2=0x7f8a3c003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:55.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.318+0000 7f8a59e5d640 1 --2- 172.21.15.3:0/1658354058 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8a3c0039e0 0x7f8a3c003de0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f8a4400c240 tx=0x7f8a3c005830 comp rx=0 tx=0).stop 2024-05-09T06:36:55.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.318+0000 7f8a53fff640 1 -- 172.21.15.3:0/1658354058 reap_dead start 2024-05-09T06:36:55.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.318+0000 7f8a59e5d640 1 -- 172.21.15.3:0/1658354058 shutdown_connections 2024-05-09T06:36:55.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.318+0000 7f8a59e5d640 1 -- 172.21.15.3:0/1658354058 >> 172.21.15.3:0/1658354058 conn(0x7f8a540d05d0 msgr2=0x7f8a540d09e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:55.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.318+0000 7f8a59e5d640 1 -- 172.21.15.3:0/1658354058 shutdown_connections 2024-05-09T06:36:55.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.318+0000 7f8a59e5d640 1 -- 172.21.15.3:0/1658354058 wait complete. 2024-05-09T06:36:55.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.319+0000 7f8a59e5d640 1 Processor -- start 2024-05-09T06:36:55.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.319+0000 7f8a59e5d640 1 -- start start 2024-05-09T06:36:55.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.320+0000 7f8a59e5d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8a3c0039e0 0x7f8a540cedc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:55.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.320+0000 7f8a59e5d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8a5415f090 0x7f8a541f5770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:55.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.320+0000 7f8a59e5d640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8a5415f510 0x7f8a541f5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:55.304 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.320+0000 7f8a59e5d640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f8a5406bcf0 con 0x7f8a5415f090 2024-05-09T06:36:55.304 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.320+0000 7f8a59e5d640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f8a540b4770 con 0x7f8a3c0039e0 2024-05-09T06:36:55.304 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.320+0000 7f8a59e5d640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f8a5406be70 con 0x7f8a5415f510 2024-05-09T06:36:55.304 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.320+0000 7f8a52ffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8a5415f090 0x7f8a541f5770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:55.304 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.320+0000 7f8a52ffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8a5415f090 0x7f8a541f5770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:56736/0 (socket says 172.21.15.3:56736) 2024-05-09T06:36:55.304 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.320+0000 7f8a52ffd640 1 -- 172.21.15.3:0/1615933343 learned_addr learned my addr 172.21.15.3:0/1615933343 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:36:55.304 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.320+0000 7f8a537fe640 1 --2- 172.21.15.3:0/1615933343 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8a3c0039e0 0x7f8a540cedc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:55.304 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.320+0000 7f8a53fff640 1 --2- 172.21.15.3:0/1615933343 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8a5415f510 0x7f8a541f5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:55.304 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.320+0000 7f8a52ffd640 1 -- 172.21.15.3:0/1615933343 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8a5415f510 msgr2=0x7f8a541f5cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:55.304 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.320+0000 7f8a52ffd640 1 --2- 172.21.15.3:0/1615933343 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8a5415f510 0x7f8a541f5cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:55.304 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.320+0000 7f8a52ffd640 1 -- 172.21.15.3:0/1615933343 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8a3c0039e0 msgr2=0x7f8a540cedc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:55.304 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.320+0000 7f8a52ffd640 1 --2- 172.21.15.3:0/1615933343 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8a3c0039e0 0x7f8a540cedc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:55.304 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.320+0000 7f8a52ffd640 1 -- 172.21.15.3:0/1615933343 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a540cdd20 con 0x7f8a5415f090 2024-05-09T06:36:55.304 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.320+0000 7f8a537fe640 1 --2- 172.21.15.3:0/1615933343 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8a3c0039e0 0x7f8a540cedc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:36:55.305 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.320+0000 7f8a52ffd640 1 --2- 172.21.15.3:0/1615933343 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8a5415f090 0x7f8a541f5770 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f8a48011940 tx=0x7f8a48011e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:55.305 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.320+0000 7f8a3bfff640 1 -- 172.21.15.3:0/1615933343 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a4800fa90 con 0x7f8a5415f090 2024-05-09T06:36:55.305 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.320+0000 7f8a3bfff640 1 -- 172.21.15.3:0/1615933343 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a4800fc30 con 0x7f8a5415f090 2024-05-09T06:36:55.305 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.320+0000 7f8a3bfff640 1 -- 172.21.15.3:0/1615933343 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a48014440 con 0x7f8a5415f090 2024-05-09T06:36:55.305 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.320+0000 7f8a59e5d640 1 -- 172.21.15.3:0/1615933343 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a540ce000 con 0x7f8a5415f090 2024-05-09T06:36:55.305 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.320+0000 7f8a59e5d640 1 -- 172.21.15.3:0/1615933343 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a540ce5c0 con 0x7f8a5415f090 2024-05-09T06:36:55.305 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.321+0000 7f8a59e5d640 1 -- 172.21.15.3:0/1615933343 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a5404f660 con 0x7f8a5415f090 2024-05-09T06:36:55.305 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.322+0000 7f8a3bfff640 1 -- 172.21.15.3:0/1615933343 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8a48012040 con 0x7f8a5415f090 2024-05-09T06:36:55.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.323+0000 7f8a3bfff640 1 --2- 172.21.15.3:0/1615933343 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f8a4006be60 0x7f8a4006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:55.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.323+0000 7f8a3bfff640 1 -- 172.21.15.3:0/1615933343 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7f8a4808b020 con 0x7f8a5415f090 2024-05-09T06:36:55.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.323+0000 7f8a537fe640 1 --2- 172.21.15.3:0/1615933343 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f8a4006be60 0x7f8a4006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:55.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.323+0000 7f8a537fe640 1 --2- 172.21.15.3:0/1615933343 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f8a4006be60 0x7f8a4006e320 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f8a44033000 tx=0x7f8a44009030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:55.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.327+0000 7f8a3bfff640 1 -- 172.21.15.3:0/1615933343 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f8a48019030 con 0x7f8a5415f090 2024-05-09T06:36:55.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.512+0000 7f8a59e5d640 1 -- 172.21.15.3:0/1615933343 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8a540b3cb0 con 0x7f8a5415f090 2024-05-09T06:36:55.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.512+0000 7f8a3bfff640 1 -- 172.21.15.3:0/1615933343 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) ==== 74+0+12807 (secure 0 0 0) 0x7f8a48059d20 con 0x7f8a5415f090 2024-05-09T06:36:55.493 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-09T06:36:55.493 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":29,"fsid":"16ed737d-2665-431a-8ac6-ef242be3975a","created":"2024-05-09T06:35:42.541368+0000","modified":"2024-05-09T06:36:52.481812+0000","last_up_change":"2024-05-09T06:35:53.778274+0000","last_in_change":"2024-05-09T06:35:44.780586+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T06:35:54.819999+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fae3a8db-4826-4f3c-b33a-12c09bb5df6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":1897849827},{"type":"v1","addr":"172.21.15.3:6817","nonce":1897849827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":1897849827},{"type":"v1","addr":"172.21.15.3:6819","nonce":1897849827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":1897849827},{"type":"v1","addr":"172.21.15.3:6823","nonce":1897849827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":1897849827},{"type":"v1","addr":"172.21.15.3:6821","nonce":1897849827}]},"public_addr":"172.21.15.3:6817/1897849827","cluster_addr":"172.21.15.3:6819/1897849827","heartbeat_back_addr":"172.21.15.3:6823/1897849827","heartbeat_front_addr":"172.21.15.3:6821/1897849827","state":["exists","up"]},{"osd":1,"uuid":"646e882d-4ce4-4720-8c84-f71df563a35e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":926805498},{"type":"v1","addr":"172.21.15.3:6801","nonce":926805498}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":926805498},{"type":"v1","addr":"172.21.15.3:6803","nonce":926805498}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":926805498},{"type":"v1","addr":"172.21.15.3:6807","nonce":926805498}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":926805498},{"type":"v1","addr":"172.21.15.3:6805","nonce":926805498}]},"public_addr":"172.21.15.3:6801/926805498","cluster_addr":"172.21.15.3:6803/926805498","heartbeat_back_addr":"172.21.15.3:6807/926805498","heartbeat_front_addr":"172.21.15.3:6805/926805498","state":["exists","up"]},{"osd":2,"uuid":"1abbadb5-a570-4359-9643-acff3ede35a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":801295451},{"type":"v1","addr":"172.21.15.3:6809","nonce":801295451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":801295451},{"type":"v1","addr":"172.21.15.3:6811","nonce":801295451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":801295451},{"type":"v1","addr":"172.21.15.3:6815","nonce":801295451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":801295451},{"type":"v1","addr":"172.21.15.3:6813","nonce":801295451}]},"public_addr":"172.21.15.3:6809/801295451","cluster_addr":"172.21.15.3:6811/801295451","heartbeat_back_addr":"172.21.15.3:6815/801295451","heartbeat_front_addr":"172.21.15.3:6813/801295451","state":["exists","up"]},{"osd":3,"uuid":"62ac4dc1-2371-4dd2-acd3-e2dbb11aedf5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":1903389857},{"type":"v1","addr":"172.21.15.3:6825","nonce":1903389857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":1903389857},{"type":"v1","addr":"172.21.15.3:6827","nonce":1903389857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":1903389857},{"type":"v1","addr":"172.21.15.3:6831","nonce":1903389857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":1903389857},{"type":"v1","addr":"172.21.15.3:6829","nonce":1903389857}]},"public_addr":"172.21.15.3:6825/1903389857","cluster_addr":"172.21.15.3:6827/1903389857","heartbeat_back_addr":"172.21.15.3:6831/1903389857","heartbeat_front_addr":"172.21.15.3:6829/1903389857","state":["exists","up"]},{"osd":4,"uuid":"ae808398-8326-4d7c-9319-a2a0eafffb59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":2143624576},{"type":"v1","addr":"172.21.15.190:6801","nonce":2143624576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":2143624576},{"type":"v1","addr":"172.21.15.190:6803","nonce":2143624576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":2143624576},{"type":"v1","addr":"172.21.15.190:6807","nonce":2143624576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":2143624576},{"type":"v1","addr":"172.21.15.190:6805","nonce":2143624576}]},"public_addr":"172.21.15.190:6801/2143624576","cluster_addr":"172.21.15.190:6803/2143624576","heartbeat_back_addr":"172.21.15.190:6807/2143624576","heartbeat_front_addr":"172.21.15.190:6805/2143624576","state":["exists","up"]},{"osd":5,"uuid":"918adf50-c440-48ac-9a8d-4837562d7275","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":1766923126},{"type":"v1","addr":"172.21.15.190:6814","nonce":1766923126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":1766923126},{"type":"v1","addr":"172.21.15.190:6818","nonce":1766923126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":1766923126},{"type":"v1","addr":"172.21.15.190:6823","nonce":1766923126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":1766923126},{"type":"v1","addr":"172.21.15.190:6821","nonce":1766923126}]},"public_addr":"172.21.15.190:6814/1766923126","cluster_addr":"172.21.15.190:6818/1766923126","heartbeat_back_addr":"172.21.15.190:6823/1766923126","heartbeat_front_addr":"172.21.15.190:6821/1766923126","state":["exists","up"]},{"osd":6,"uuid":"ac2882bc-2a7c-463c-8f6c-76ff49828adc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":342414248},{"type":"v1","addr":"172.21.15.190:6825","nonce":342414248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":342414248},{"type":"v1","addr":"172.21.15.190:6827","nonce":342414248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":342414248},{"type":"v1","addr":"172.21.15.190:6831","nonce":342414248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":342414248},{"type":"v1","addr":"172.21.15.190:6829","nonce":342414248}]},"public_addr":"172.21.15.190:6825/342414248","cluster_addr":"172.21.15.190:6827/342414248","heartbeat_back_addr":"172.21.15.190:6831/342414248","heartbeat_front_addr":"172.21.15.190:6829/342414248","state":["exists","up"]},{"osd":7,"uuid":"dd4e767a-4730-4c70-8bd5-3f088140a5a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":537849891},{"type":"v1","addr":"172.21.15.190:6809","nonce":537849891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":537849891},{"type":"v1","addr":"172.21.15.190:6811","nonce":537849891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6817","nonce":537849891},{"type":"v1","addr":"172.21.15.190:6819","nonce":537849891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6813","nonce":537849891},{"type":"v1","addr":"172.21.15.190:6815","nonce":537849891}]},"public_addr":"172.21.15.190:6809/537849891","cluster_addr":"172.21.15.190:6811/537849891","heartbeat_back_addr":"172.21.15.190:6819/537849891","heartbeat_front_addr":"172.21.15.190:6815/537849891","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.889784+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.974002+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.973549+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.955396+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.928471+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.902418+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.958696+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.926474+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T06:36:55.495 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.514+0000 7f8a59e5d640 1 -- 172.21.15.3:0/1615933343 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f8a4006be60 msgr2=0x7f8a4006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:55.495 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.514+0000 7f8a59e5d640 1 --2- 172.21.15.3:0/1615933343 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f8a4006be60 0x7f8a4006e320 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f8a44033000 tx=0x7f8a44009030 comp rx=0 tx=0).stop 2024-05-09T06:36:55.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.514+0000 7f8a59e5d640 1 -- 172.21.15.3:0/1615933343 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8a5415f090 msgr2=0x7f8a541f5770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:55.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.514+0000 7f8a59e5d640 1 --2- 172.21.15.3:0/1615933343 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8a5415f090 0x7f8a541f5770 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f8a48011940 tx=0x7f8a48011e00 comp rx=0 tx=0).stop 2024-05-09T06:36:55.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.515+0000 7f8a59e5d640 1 -- 172.21.15.3:0/1615933343 shutdown_connections 2024-05-09T06:36:55.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.515+0000 7f8a59e5d640 1 --2- 172.21.15.3:0/1615933343 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f8a4006be60 0x7f8a4006e320 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:55.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.515+0000 7f8a59e5d640 1 --2- 172.21.15.3:0/1615933343 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8a5415f510 0x7f8a541f5cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:55.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.515+0000 7f8a59e5d640 1 --2- 172.21.15.3:0/1615933343 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8a5415f090 0x7f8a541f5770 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:55.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.515+0000 7f8a59e5d640 1 --2- 172.21.15.3:0/1615933343 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8a3c0039e0 0x7f8a540cedc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:55.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.515+0000 7f8a59e5d640 1 -- 172.21.15.3:0/1615933343 >> 172.21.15.3:0/1615933343 conn(0x7f8a540d05d0 msgr2=0x7f8a5406e410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:55.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.515+0000 7f8a59e5d640 1 -- 172.21.15.3:0/1615933343 shutdown_connections 2024-05-09T06:36:55.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.515+0000 7f8a59e5d640 1 -- 172.21.15.3:0/1615933343 wait complete. 2024-05-09T06:36:55.507 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-09T06:35:54.819999+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-09T06:36:55.507 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-09T06:36:55.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.633+0000 7f76dafc8640 1 Processor -- start 2024-05-09T06:36:55.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.633+0000 7f76dafc8640 1 -- start start 2024-05-09T06:36:55.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.634+0000 7f76dafc8640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f76d40d1300 0x7f76d40d2800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:55.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.634+0000 7f76dafc8640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f76d40d4c40 0x7f76d40b47e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:55.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.634+0000 7f76dafc8640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f76d415e9a0 con 0x7f76d415e5c0 2024-05-09T06:36:55.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.634+0000 7f76dafc8640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f76d40d50d0 con 0x7f76d40d1300 2024-05-09T06:36:55.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.634+0000 7f76dafc8640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f76d40d31c0 con 0x7f76d40d4c40 2024-05-09T06:36:55.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.635+0000 7f76d9565640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f76d40d4c40 0x7f76d40b47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:55.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.635+0000 7f76d3fff640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f76d40d1300 0x7f76d40d2800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:55.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.635+0000 7f76d8d64640 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7f76d415e5c0 0x7f76d40d0bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.3:35996/0 (socket says 172.21.15.3:35996) 2024-05-09T06:36:55.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.635+0000 7f76d8d64640 1 -- 172.21.15.3:0/2549816727 learned_addr learned my addr 172.21.15.3:0/2549816727 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-05-09T06:36:55.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.635+0000 7f76d9565640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f76d40d4c40 0x7f76d40b47e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.3:60976/0 (socket says 172.21.15.3:60976) 2024-05-09T06:36:55.617 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.635+0000 7f76d3fff640 1 -- 172.21.15.3:0/2549816727 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f76d40d4c40 msgr2=0x7f76d40b47e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:55.617 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.635+0000 7f76d3fff640 1 --2- 172.21.15.3:0/2549816727 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f76d40d4c40 0x7f76d40b47e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:55.617 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.635+0000 7f76d3fff640 1 -- 172.21.15.3:0/2549816727 >> v1:172.21.15.190:6789/0 conn(0x7f76d415e5c0 legacy=0x7f76d40d0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:55.617 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.635+0000 7f76d3fff640 1 -- 172.21.15.3:0/2549816727 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76d40d2ee0 con 0x7f76d40d1300 2024-05-09T06:36:55.617 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.635+0000 7f76d9565640 1 --2- 172.21.15.3:0/2549816727 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f76d40d4c40 0x7f76d40b47e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T06:36:55.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.636+0000 7f76d3fff640 1 --2- 172.21.15.3:0/2549816727 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f76d40d1300 0x7f76d40d2800 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f76c80097a0 tx=0x7f76c802a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7fe3098ca585b52f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:55.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.636+0000 7f76d37fe640 1 -- 172.21.15.3:0/2549816727 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76c8037070 con 0x7f76d40d1300 2024-05-09T06:36:55.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.636+0000 7f76d37fe640 1 -- 172.21.15.3:0/2549816727 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f76c8032440 con 0x7f76d40d1300 2024-05-09T06:36:55.619 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.636+0000 7f76d37fe640 1 -- 172.21.15.3:0/2549816727 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76c8032810 con 0x7f76d40d1300 2024-05-09T06:36:55.619 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.637+0000 7f76dafc8640 1 -- 172.21.15.3:0/2549816727 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f76d40d1300 msgr2=0x7f76d40d2800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:55.619 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.637+0000 7f76dafc8640 1 --2- 172.21.15.3:0/2549816727 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f76d40d1300 0x7f76d40d2800 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f76c80097a0 tx=0x7f76c802a180 comp rx=0 tx=0).stop 2024-05-09T06:36:55.619 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.637+0000 7f76dafc8640 1 -- 172.21.15.3:0/2549816727 shutdown_connections 2024-05-09T06:36:55.619 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.637+0000 7f76dafc8640 1 --2- 172.21.15.3:0/2549816727 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f76d40d4c40 0x7f76d40b47e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:55.619 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.637+0000 7f76dafc8640 1 --2- 172.21.15.3:0/2549816727 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f76d40d1300 0x7f76d40d2800 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:55.619 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.637+0000 7f76dafc8640 1 -- 172.21.15.3:0/2549816727 >> 172.21.15.3:0/2549816727 conn(0x7f76d409d500 msgr2=0x7f76d40d4220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:55.619 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.637+0000 7f76dafc8640 1 -- 172.21.15.3:0/2549816727 shutdown_connections 2024-05-09T06:36:55.619 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.637+0000 7f76dafc8640 1 -- 172.21.15.3:0/2549816727 wait complete. 2024-05-09T06:36:55.619 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.638+0000 7f76dafc8640 1 Processor -- start 2024-05-09T06:36:55.619 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.638+0000 7f76dafc8640 1 -- start start 2024-05-09T06:36:55.621 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.639+0000 7f76dafc8640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f76d40d1300 0x7f76d40b6730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:55.621 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.639+0000 7f76dafc8640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f76d40d4c40 0x7f76d40b83a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:55.621 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.639+0000 7f76dafc8640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f76d415e5c0 0x7f76d40b88e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:55.621 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.639+0000 7f76dafc8640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f76d40b42c0 con 0x7f76d40d1300 2024-05-09T06:36:55.621 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.639+0000 7f76dafc8640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f76d40b3fe0 con 0x7f76d40d4c40 2024-05-09T06:36:55.621 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.639+0000 7f76dafc8640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f76d40b4d20 con 0x7f76d415e5c0 2024-05-09T06:36:55.621 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.639+0000 7f76d8d64640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f76d40d1300 0x7f76d40b6730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:55.621 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.639+0000 7f76d8d64640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f76d40d1300 0x7f76d40b6730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:56742/0 (socket says 172.21.15.3:56742) 2024-05-09T06:36:55.622 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.639+0000 7f76d8d64640 1 -- 172.21.15.3:0/20375643 learned_addr learned my addr 172.21.15.3:0/20375643 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:36:55.622 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.639+0000 7f76d9565640 1 --2- 172.21.15.3:0/20375643 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f76d415e5c0 0x7f76d40b88e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:55.622 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.639+0000 7f76d3fff640 1 --2- 172.21.15.3:0/20375643 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f76d40d4c40 0x7f76d40b83a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:55.622 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.639+0000 7f76d8d64640 1 -- 172.21.15.3:0/20375643 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f76d415e5c0 msgr2=0x7f76d40b88e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:55.622 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.639+0000 7f76d8d64640 1 --2- 172.21.15.3:0/20375643 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f76d415e5c0 0x7f76d40b88e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:55.622 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.639+0000 7f76d8d64640 1 -- 172.21.15.3:0/20375643 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f76d40d4c40 msgr2=0x7f76d40b83a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:55.622 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.639+0000 7f76d8d64640 1 --2- 172.21.15.3:0/20375643 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f76d40d4c40 0x7f76d40b83a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:55.622 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.639+0000 7f76d8d64640 1 -- 172.21.15.3:0/20375643 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76d415ae00 con 0x7f76d40d1300 2024-05-09T06:36:55.622 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.639+0000 7f76d8d64640 1 --2- 172.21.15.3:0/20375643 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f76d40d1300 0x7f76d40b6730 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f76cc00ca70 tx=0x7f76cc00cf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:55.622 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.640+0000 7f76d17fa640 1 -- 172.21.15.3:0/20375643 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76cc017070 con 0x7f76d40d1300 2024-05-09T06:36:55.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.640+0000 7f76d17fa640 1 -- 172.21.15.3:0/20375643 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f76cc00abd0 con 0x7f76d40d1300 2024-05-09T06:36:55.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.640+0000 7f76d17fa640 1 -- 172.21.15.3:0/20375643 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76cc012400 con 0x7f76d40d1300 2024-05-09T06:36:55.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.640+0000 7f76dafc8640 1 -- 172.21.15.3:0/20375643 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76d41609d0 con 0x7f76d40d1300 2024-05-09T06:36:55.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.640+0000 7f76dafc8640 1 -- 172.21.15.3:0/20375643 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76d4161270 con 0x7f76d40d1300 2024-05-09T06:36:55.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.641+0000 7f76dafc8640 1 -- 172.21.15.3:0/20375643 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76d4069550 con 0x7f76d40d1300 2024-05-09T06:36:55.624 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.641+0000 7f76d17fa640 1 -- 172.21.15.3:0/20375643 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f76cc01a070 con 0x7f76d40d1300 2024-05-09T06:36:55.624 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.642+0000 7f76d17fa640 1 --2- 172.21.15.3:0/20375643 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f76c006be30 0x7f76c006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:55.624 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.642+0000 7f76d17fa640 1 -- 172.21.15.3:0/20375643 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7f76cc08a780 con 0x7f76d40d1300 2024-05-09T06:36:55.624 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.642+0000 7f76d3fff640 1 --2- 172.21.15.3:0/20375643 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f76c006be30 0x7f76c006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:55.624 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.642+0000 7f76d3fff640 1 --2- 172.21.15.3:0/20375643 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f76c006be30 0x7f76c006e2f0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f76c8034000 tx=0x7f76c802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:55.626 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.646+0000 7f76d17fa640 1 -- 172.21.15.3:0/20375643 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f76cc059510 con 0x7f76d40d1300 2024-05-09T06:36:55.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.827+0000 7f76dafc8640 1 -- 172.21.15.3:0/20375643 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f76d40b9c40 con 0x7f76d40d1300 2024-05-09T06:36:55.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.828+0000 7f76d17fa640 1 -- 172.21.15.3:0/20375643 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v29) ==== 93+0+10 (secure 0 0 0) 0x7f76cc058e10 con 0x7f76d40d1300 2024-05-09T06:36:55.809 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 1 2024-05-09T06:36:55.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.830+0000 7f76dafc8640 1 -- 172.21.15.3:0/20375643 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f76c006be30 msgr2=0x7f76c006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:55.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.830+0000 7f76dafc8640 1 --2- 172.21.15.3:0/20375643 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f76c006be30 0x7f76c006e2f0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f76c8034000 tx=0x7f76c802e000 comp rx=0 tx=0).stop 2024-05-09T06:36:55.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.830+0000 7f76dafc8640 1 -- 172.21.15.3:0/20375643 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f76d40d1300 msgr2=0x7f76d40b6730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:55.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.830+0000 7f76dafc8640 1 --2- 172.21.15.3:0/20375643 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f76d40d1300 0x7f76d40b6730 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f76cc00ca70 tx=0x7f76cc00cf30 comp rx=0 tx=0).stop 2024-05-09T06:36:55.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.830+0000 7f76dafc8640 1 -- 172.21.15.3:0/20375643 shutdown_connections 2024-05-09T06:36:55.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.830+0000 7f76dafc8640 1 --2- 172.21.15.3:0/20375643 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f76c006be30 0x7f76c006e2f0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:55.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.830+0000 7f76dafc8640 1 --2- 172.21.15.3:0/20375643 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f76d415e5c0 0x7f76d40b88e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:55.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.830+0000 7f76dafc8640 1 --2- 172.21.15.3:0/20375643 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f76d40d4c40 0x7f76d40b83a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:55.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.830+0000 7f76dafc8640 1 --2- 172.21.15.3:0/20375643 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f76d40d1300 0x7f76d40b6730 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:55.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.830+0000 7f76dafc8640 1 -- 172.21.15.3:0/20375643 >> 172.21.15.3:0/20375643 conn(0x7f76d409d500 msgr2=0x7f76d40b1000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:55.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.830+0000 7f76dafc8640 1 -- 172.21.15.3:0/20375643 shutdown_connections 2024-05-09T06:36:55.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.830+0000 7f76dafc8640 1 -- 172.21.15.3:0/20375643 wait complete. 2024-05-09T06:36:55.822 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-05-09T06:36:55.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.945+0000 7f0d35316640 1 Processor -- start 2024-05-09T06:36:55.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.945+0000 7f0d35316640 1 -- start start 2024-05-09T06:36:55.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.946+0000 7f0d35316640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f0d3006d8f0 0x7f0d3006bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:55.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.946+0000 7f0d35316640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f0d300ae9c0 0x7f0d3006c230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:55.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.946+0000 7f0d35316640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0d3006cab0 con 0x7f0d3015e100 2024-05-09T06:36:55.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.946+0000 7f0d35316640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f0d300b3ed0 con 0x7f0d3006d8f0 2024-05-09T06:36:55.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.946+0000 7f0d35316640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f0d300b4770 con 0x7f0d300ae9c0 2024-05-09T06:36:55.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.946+0000 7f0d2effd640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f0d3015e100 0x7f0d300ae2b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:53384/0 (socket says 172.21.15.3:53384) 2024-05-09T06:36:55.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.946+0000 7f0d2effd640 1 -- 172.21.15.3:0/223612853 learned_addr learned my addr 172.21.15.3:0/223612853 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-05-09T06:36:55.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.946+0000 7f0d27fff640 1 --2- 172.21.15.3:0/223612853 >> v2:172.21.15.3:3300/0 conn(0x7f0d3006d8f0 0x7f0d3006bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:55.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.947+0000 7f0d2f7fe640 1 --2- 172.21.15.3:0/223612853 >> v2:172.21.15.190:3300/0 conn(0x7f0d300ae9c0 0x7f0d3006c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:55.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.947+0000 7f0d27fff640 1 -- 172.21.15.3:0/223612853 >> v2:172.21.15.190:3300/0 conn(0x7f0d300ae9c0 msgr2=0x7f0d3006c230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:55.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.947+0000 7f0d27fff640 1 --2- 172.21.15.3:0/223612853 >> v2:172.21.15.190:3300/0 conn(0x7f0d300ae9c0 0x7f0d3006c230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:55.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.947+0000 7f0d27fff640 1 -- 172.21.15.3:0/223612853 >> v1:172.21.15.3:6789/0 conn(0x7f0d3015e100 legacy=0x7f0d300ae2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:55.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.947+0000 7f0d27fff640 1 -- 172.21.15.3:0/223612853 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d300b4120 con 0x7f0d3006d8f0 2024-05-09T06:36:55.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.947+0000 7f0d2e7fc640 1 -- 172.21.15.3:0/223612853 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 201505597 0 0) 0x7f0d3006cab0 con 0x7f0d3015e100 2024-05-09T06:36:55.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.947+0000 7f0d27fff640 1 --2- 172.21.15.3:0/223612853 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0d3006d8f0 0x7f0d3006bcf0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f0d300ccad0 tx=0x7f0d1802a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e22808cddc3bd82 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:55.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.948+0000 7f0d2e7fc640 1 -- 172.21.15.3:0/223612853 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d18037070 con 0x7f0d3006d8f0 2024-05-09T06:36:55.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.948+0000 7f0d2e7fc640 1 -- 172.21.15.3:0/223612853 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d18032440 con 0x7f0d3006d8f0 2024-05-09T06:36:55.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.948+0000 7f0d2e7fc640 1 -- 172.21.15.3:0/223612853 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d1802ae60 con 0x7f0d3006d8f0 2024-05-09T06:36:55.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.948+0000 7f0d35316640 1 -- 172.21.15.3:0/223612853 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0d3006d8f0 msgr2=0x7f0d3006bcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:55.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.948+0000 7f0d35316640 1 --2- 172.21.15.3:0/223612853 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0d3006d8f0 0x7f0d3006bcf0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f0d300ccad0 tx=0x7f0d1802a110 comp rx=0 tx=0).stop 2024-05-09T06:36:55.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.948+0000 7f0d35316640 1 -- 172.21.15.3:0/223612853 shutdown_connections 2024-05-09T06:36:55.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.948+0000 7f0d35316640 1 --2- 172.21.15.3:0/223612853 >> v2:172.21.15.190:3300/0 conn(0x7f0d300ae9c0 0x7f0d3006c230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:55.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.948+0000 7f0d35316640 1 --2- 172.21.15.3:0/223612853 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0d3006d8f0 0x7f0d3006bcf0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:55.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.948+0000 7f0d35316640 1 -- 172.21.15.3:0/223612853 >> 172.21.15.3:0/223612853 conn(0x7f0d300d05d0 msgr2=0x7f0d300d09e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:55.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.948+0000 7f0d35316640 1 -- 172.21.15.3:0/223612853 shutdown_connections 2024-05-09T06:36:55.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.948+0000 7f0d35316640 1 -- 172.21.15.3:0/223612853 wait complete. 2024-05-09T06:36:55.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.949+0000 7f0d35316640 1 Processor -- start 2024-05-09T06:36:55.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.950+0000 7f0d35316640 1 -- start start 2024-05-09T06:36:55.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.950+0000 7f0d35316640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0d3006d8f0 0x7f0d3006b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:55.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.950+0000 7f0d35316640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0d300ae9c0 0x7f0d300ce730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:55.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.950+0000 7f0d35316640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0d3015e100 0x7f0d300cec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:55.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.950+0000 7f0d35316640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f0d3015e4e0 con 0x7f0d3006d8f0 2024-05-09T06:36:55.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.950+0000 7f0d35316640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f0d3006c770 con 0x7f0d300ae9c0 2024-05-09T06:36:55.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.950+0000 7f0d35316640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f0d3015e660 con 0x7f0d3015e100 2024-05-09T06:36:55.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.950+0000 7f0d2effd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0d3006d8f0 0x7f0d3006b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:55.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.950+0000 7f0d2effd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0d3006d8f0 0x7f0d3006b4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:56752/0 (socket says 172.21.15.3:56752) 2024-05-09T06:36:55.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.950+0000 7f0d2effd640 1 -- 172.21.15.3:0/2801142614 learned_addr learned my addr 172.21.15.3:0/2801142614 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:36:55.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.951+0000 7f0d27fff640 1 --2- 172.21.15.3:0/2801142614 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0d300ae9c0 0x7f0d300ce730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:55.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.951+0000 7f0d2f7fe640 1 --2- 172.21.15.3:0/2801142614 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0d3015e100 0x7f0d300cec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:55.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.951+0000 7f0d2effd640 1 -- 172.21.15.3:0/2801142614 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0d3015e100 msgr2=0x7f0d300cec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:55.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.951+0000 7f0d2effd640 1 --2- 172.21.15.3:0/2801142614 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0d3015e100 0x7f0d300cec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:55.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.951+0000 7f0d2effd640 1 -- 172.21.15.3:0/2801142614 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0d300ae9c0 msgr2=0x7f0d300ce730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:55.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.951+0000 7f0d2effd640 1 --2- 172.21.15.3:0/2801142614 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0d300ae9c0 0x7f0d300ce730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:55.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.951+0000 7f0d2effd640 1 -- 172.21.15.3:0/2801142614 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d300cce30 con 0x7f0d3006d8f0 2024-05-09T06:36:55.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.951+0000 7f0d27fff640 1 --2- 172.21.15.3:0/2801142614 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0d300ae9c0 0x7f0d300ce730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:36:55.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.951+0000 7f0d2f7fe640 1 --2- 172.21.15.3:0/2801142614 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0d3015e100 0x7f0d300cec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:36:55.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.951+0000 7f0d2effd640 1 --2- 172.21.15.3:0/2801142614 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0d3006d8f0 0x7f0d3006b4c0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f0d2000c430 tx=0x7f0d2000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:55.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.951+0000 7f0d277fe640 1 -- 172.21.15.3:0/2801142614 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d200074e0 con 0x7f0d3006d8f0 2024-05-09T06:36:55.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.951+0000 7f0d277fe640 1 -- 172.21.15.3:0/2801142614 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d200047e0 con 0x7f0d3006d8f0 2024-05-09T06:36:55.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.951+0000 7f0d277fe640 1 -- 172.21.15.3:0/2801142614 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d20004bd0 con 0x7f0d3006d8f0 2024-05-09T06:36:55.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.951+0000 7f0d35316640 1 -- 172.21.15.3:0/2801142614 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d300cf310 con 0x7f0d3006d8f0 2024-05-09T06:36:55.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.951+0000 7f0d35316640 1 -- 172.21.15.3:0/2801142614 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d30153f30 con 0x7f0d3006d8f0 2024-05-09T06:36:55.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.952+0000 7f0d35316640 1 -- 172.21.15.3:0/2801142614 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d3004f660 con 0x7f0d3006d8f0 2024-05-09T06:36:55.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.952+0000 7f0d277fe640 1 -- 172.21.15.3:0/2801142614 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0d20011070 con 0x7f0d3006d8f0 2024-05-09T06:36:55.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.953+0000 7f0d277fe640 1 --2- 172.21.15.3:0/2801142614 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f0d1006be30 0x7f0d1006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:55.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.953+0000 7f0d277fe640 1 -- 172.21.15.3:0/2801142614 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7f0d2008a780 con 0x7f0d3006d8f0 2024-05-09T06:36:55.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.953+0000 7f0d27fff640 1 --2- 172.21.15.3:0/2801142614 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f0d1006be30 0x7f0d1006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:55.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.954+0000 7f0d27fff640 1 --2- 172.21.15.3:0/2801142614 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f0d1006be30 0x7f0d1006e2f0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f0d18034000 tx=0x7f0d1802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:55.938 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:55.957+0000 7f0d277fe640 1 -- 172.21.15.3:0/2801142614 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f0d20059480 con 0x7f0d3006d8f0 2024-05-09T06:36:56.121 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:56.141+0000 7f0d35316640 1 -- 172.21.15.3:0/2801142614 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f0d300b3750 con 0x7f0d3006d8f0 2024-05-09T06:36:56.122 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:56.141+0000 7f0d277fe640 1 -- 172.21.15.3:0/2801142614 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v29) ==== 84+0+65 (secure 0 0 0) 0x7f0d20058d80 con 0x7f0d3006d8f0 2024-05-09T06:36:56.122 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-09T06:36:56.122 INFO:teuthology.orchestra.run.smithi003.stdout:[ 2024-05-09T06:36:56.122 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-09T06:36:56.122 INFO:teuthology.orchestra.run.smithi003.stdout: "poolnum": 1, 2024-05-09T06:36:56.122 INFO:teuthology.orchestra.run.smithi003.stdout: "poolname": ".mgr" 2024-05-09T06:36:56.122 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-09T06:36:56.122 INFO:teuthology.orchestra.run.smithi003.stdout:] 2024-05-09T06:36:56.124 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:56.143+0000 7f0d35316640 1 -- 172.21.15.3:0/2801142614 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f0d1006be30 msgr2=0x7f0d1006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:56.124 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:56.143+0000 7f0d35316640 1 --2- 172.21.15.3:0/2801142614 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f0d1006be30 0x7f0d1006e2f0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f0d18034000 tx=0x7f0d1802e000 comp rx=0 tx=0).stop 2024-05-09T06:36:56.124 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:56.143+0000 7f0d35316640 1 -- 172.21.15.3:0/2801142614 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0d3006d8f0 msgr2=0x7f0d3006b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:56.124 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:56.143+0000 7f0d35316640 1 --2- 172.21.15.3:0/2801142614 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0d3006d8f0 0x7f0d3006b4c0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f0d2000c430 tx=0x7f0d2000c8f0 comp rx=0 tx=0).stop 2024-05-09T06:36:56.124 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:56.143+0000 7f0d35316640 1 -- 172.21.15.3:0/2801142614 shutdown_connections 2024-05-09T06:36:56.124 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:56.143+0000 7f0d35316640 1 --2- 172.21.15.3:0/2801142614 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f0d1006be30 0x7f0d1006e2f0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:56.124 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:56.143+0000 7f0d35316640 1 --2- 172.21.15.3:0/2801142614 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0d3015e100 0x7f0d300cec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:56.124 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:56.143+0000 7f0d35316640 1 --2- 172.21.15.3:0/2801142614 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0d300ae9c0 0x7f0d300ce730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:56.124 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:56.143+0000 7f0d35316640 1 --2- 172.21.15.3:0/2801142614 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0d3006d8f0 0x7f0d3006b4c0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:56.124 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:56.143+0000 7f0d35316640 1 -- 172.21.15.3:0/2801142614 >> 172.21.15.3:0/2801142614 conn(0x7f0d300d05d0 msgr2=0x7f0d300adfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:56.125 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:56.143+0000 7f0d35316640 1 -- 172.21.15.3:0/2801142614 shutdown_connections 2024-05-09T06:36:56.125 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:56.143+0000 7f0d35316640 1 -- 172.21.15.3:0/2801142614 wait complete. 2024-05-09T06:36:56.134 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-05-09T06:36:56.135 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-05-09T06:36:56.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:56.262+0000 7fc1a94f5640 1 Processor -- start 2024-05-09T06:36:56.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:56.262+0000 7fc1a94f5640 1 -- start start 2024-05-09T06:36:56.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:56.263+0000 7fc1a94f5640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fc1a40cb330 0x7fc1a406cde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:56.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:56.263+0000 7fc1a94f5640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc1a406d320 0x7fc1a40d0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:56.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:56.263+0000 7fc1a94f5640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc1a40cb7c0 con 0x7fc1a4144ac0 2024-05-09T06:36:56.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:56.263+0000 7fc1a94f5640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7fc1a40cbaa0 con 0x7fc1a40cb330 2024-05-09T06:36:56.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:56.263+0000 7fc1a94f5640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fc1a40cbd80 con 0x7fc1a406d320 2024-05-09T06:36:56.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:56.263+0000 7fc1a2ffd640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fc1a4144ac0 0x7fc1a4071550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:53388/0 (socket says 172.21.15.3:53388) 2024-05-09T06:36:56.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:56.263+0000 7fc1a2ffd640 1 -- 172.21.15.3:0/2375313932 learned_addr learned my addr 172.21.15.3:0/2375313932 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-05-09T06:36:56.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:56.263+0000 7fc1a37fe640 1 --2- 172.21.15.3:0/2375313932 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc1a406d320 0x7fc1a40d0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:56.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:56.263+0000 7fc1a1ffb640 1 -- 172.21.15.3:0/2375313932 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2552497694 0 0) 0x7fc1a40cb7c0 con 0x7fc1a4144ac0 2024-05-09T06:36:56.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:56.263+0000 7fc1a1ffb640 1 -- 172.21.15.3:0/2375313932 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc190003520 con 0x7fc1a4144ac0 2024-05-09T06:36:56.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:56.263+0000 7fc1a27fc640 1 --2- 172.21.15.3:0/2375313932 >> v2:172.21.15.190:3300/0 conn(0x7fc1a40cb330 0x7fc1a406cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:56.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:56.264+0000 7fc1a1ffb640 1 -- 172.21.15.3:0/2375313932 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2177303621 0 0) 0x7fc190003520 con 0x7fc1a4144ac0 2024-05-09T06:36:56.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:56.264+0000 7fc1a1ffb640 1 -- 172.21.15.3:0/2375313932 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc1a406d320 msgr2=0x7fc1a40d0aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:56.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:56.264+0000 7fc1a1ffb640 1 --2- 172.21.15.3:0/2375313932 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc1a406d320 0x7fc1a40d0aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:56.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:56.264+0000 7fc1a1ffb640 1 -- 172.21.15.3:0/2375313932 >> v2:172.21.15.190:3300/0 conn(0x7fc1a40cb330 msgr2=0x7fc1a406cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:56.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:56.264+0000 7fc1a1ffb640 1 --2- 172.21.15.3:0/2375313932 >> v2:172.21.15.190:3300/0 conn(0x7fc1a40cb330 0x7fc1a406cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:56.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:56.264+0000 7fc1a1ffb640 1 -- 172.21.15.3:0/2375313932 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1a40cc060 con 0x7fc1a4144ac0 2024-05-09T06:36:56.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:56.264+0000 7fc1a1ffb640 1 -- 172.21.15.3:0/2375313932 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 563312031 0 0) 0x7fc19c001670 con 0x7fc1a4144ac0 2024-05-09T06:36:56.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:56.264+0000 7fc1a1ffb640 1 -- 172.21.15.3:0/2375313932 >> v1:172.21.15.3:6789/0 conn(0x7fc1a4144ac0 legacy=0x7fc1a4071550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:56.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:56.264+0000 7fc1a1ffb640 1 --2- 172.21.15.3:0/2375313932 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc1a406d320 0x7fc1900039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:56.247 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:56.264+0000 7fc1a1ffb640 1 --2- 172.21.15.3:0/2375313932 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc190003f80 0x7fc190004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:56.247 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:56.264+0000 7fc1a1ffb640 1 --2- 172.21.15.3:0/2375313932 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc190004d40 0x7fc1900051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:56.247 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:56.264+0000 7fc1a1ffb640 1 -- 172.21.15.3:0/2375313932 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fc1a40cbaa0 con 0x7fc190004d40 2024-05-09T06:36:56.247 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:56.264+0000 7fc1a1ffb640 1 -- 172.21.15.3:0/2375313932 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fc1a40cbd80 con 0x7fc190003f80 2024-05-09T06:36:56.247 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:56.264+0000 7fc1a1ffb640 1 -- 172.21.15.3:0/2375313932 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fc19000b9c0 con 0x7fc1a406d320 2024-05-09T06:36:56.247 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:56.264+0000 7fc1a37fe640 1 --2- 172.21.15.3:0/2375313932 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc190004d40 0x7fc1900051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:56.247 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:56.264+0000 7fc1a2ffd640 1 --2- 172.21.15.3:0/2375313932 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc1a406d320 0x7fc1900039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:56.247 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:56.264+0000 7fc1a27fc640 1 --2- 172.21.15.3:0/2375313932 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc190003f80 0x7fc190004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:56.247 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:56.264+0000 7fc1a37fe640 1 -- 172.21.15.3:0/2375313932 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc1a406d320 msgr2=0x7fc1900039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:56.247 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:56.264+0000 7fc1a37fe640 1 --2- 172.21.15.3:0/2375313932 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc1a406d320 0x7fc1900039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:56.247 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:56.264+0000 7fc1a37fe640 1 -- 172.21.15.3:0/2375313932 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc190003f80 msgr2=0x7fc190004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:56.247 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:56.264+0000 7fc1a37fe640 1 --2- 172.21.15.3:0/2375313932 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc190003f80 0x7fc190004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:56.247 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:56.264+0000 7fc1a37fe640 1 -- 172.21.15.3:0/2375313932 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc190003520 con 0x7fc190004d40 2024-05-09T06:36:56.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:56.265+0000 7fc1a37fe640 1 --2- 172.21.15.3:0/2375313932 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc190004d40 0x7fc1900051c0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fc19400e170 tx=0x7fc190005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=796cff7a603f4db5 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:56.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:56.265+0000 7fc1a1ffb640 1 -- 172.21.15.3:0/2375313932 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc194034020 con 0x7fc190004d40 2024-05-09T06:36:56.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:56.265+0000 7fc1a1ffb640 1 -- 172.21.15.3:0/2375313932 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc1940305f0 con 0x7fc190004d40 2024-05-09T06:36:56.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:56.265+0000 7fc1a1ffb640 1 -- 172.21.15.3:0/2375313932 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc194009c10 con 0x7fc190004d40 2024-05-09T06:36:56.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:56.266+0000 7fc1a94f5640 1 -- 172.21.15.3:0/2375313932 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc190004d40 msgr2=0x7fc1900051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:56.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:56.266+0000 7fc1a94f5640 1 --2- 172.21.15.3:0/2375313932 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc190004d40 0x7fc1900051c0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fc19400e170 tx=0x7fc190005830 comp rx=0 tx=0).stop 2024-05-09T06:36:56.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:56.266+0000 7fc1a37fe640 1 -- 172.21.15.3:0/2375313932 reap_dead start 2024-05-09T06:36:56.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:56.266+0000 7fc1a94f5640 1 -- 172.21.15.3:0/2375313932 shutdown_connections 2024-05-09T06:36:56.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:56.266+0000 7fc1a94f5640 1 -- 172.21.15.3:0/2375313932 >> 172.21.15.3:0/2375313932 conn(0x7fc1a40d2000 msgr2=0x7fc1a40d23e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:56.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:56.266+0000 7fc1a94f5640 1 -- 172.21.15.3:0/2375313932 shutdown_connections 2024-05-09T06:36:56.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:56.266+0000 7fc1a94f5640 1 -- 172.21.15.3:0/2375313932 wait complete. 2024-05-09T06:36:56.252 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:56.267+0000 7fc1a94f5640 1 Processor -- start 2024-05-09T06:36:56.252 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:56.267+0000 7fc1a94f5640 1 -- start start 2024-05-09T06:36:56.252 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:56.268+0000 7fc1a94f5640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc190004d40 0x7fc1a40b2da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:56.252 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:56.268+0000 7fc1a94f5640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc1a40b6750 0x7fc1a40b6bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:56.252 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:56.268+0000 7fc1a94f5640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc1a40a96c0 0x7fc1a40a9b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:56.252 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:56.268+0000 7fc1a94f5640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fc1a4144ea0 con 0x7fc1a40b6750 2024-05-09T06:36:56.252 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:56.268+0000 7fc1a94f5640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fc1a40cc650 con 0x7fc190004d40 2024-05-09T06:36:56.252 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:56.268+0000 7fc1a94f5640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fc1a40d0aa0 con 0x7fc1a40a96c0 2024-05-09T06:36:56.252 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:56.268+0000 7fc1a27fc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc1a40b6750 0x7fc1a40b6bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:56.252 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:56.268+0000 7fc1a27fc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc1a40b6750 0x7fc1a40b6bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:56770/0 (socket says 172.21.15.3:56770) 2024-05-09T06:36:56.252 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:56.268+0000 7fc1a27fc640 1 -- 172.21.15.3:0/3417833411 learned_addr learned my addr 172.21.15.3:0/3417833411 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:36:56.252 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:56.268+0000 7fc1a2ffd640 1 --2- 172.21.15.3:0/3417833411 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc190004d40 0x7fc1a40b2da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:56.252 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:56.268+0000 7fc1a37fe640 1 --2- 172.21.15.3:0/3417833411 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc1a40a96c0 0x7fc1a40a9b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:56.252 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:56.268+0000 7fc1a27fc640 1 -- 172.21.15.3:0/3417833411 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc1a40a96c0 msgr2=0x7fc1a40a9b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:56.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:56.268+0000 7fc1a27fc640 1 --2- 172.21.15.3:0/3417833411 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc1a40a96c0 0x7fc1a40a9b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:56.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:56.268+0000 7fc1a27fc640 1 -- 172.21.15.3:0/3417833411 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc190004d40 msgr2=0x7fc1a40b2da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:56.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:56.268+0000 7fc1a27fc640 1 --2- 172.21.15.3:0/3417833411 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc190004d40 0x7fc1a40b2da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:56.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:56.268+0000 7fc1a27fc640 1 -- 172.21.15.3:0/3417833411 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1a41368c0 con 0x7fc1a40b6750 2024-05-09T06:36:56.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:56.268+0000 7fc1a2ffd640 1 --2- 172.21.15.3:0/3417833411 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc190004d40 0x7fc1a40b2da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:36:56.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:56.268+0000 7fc1a37fe640 1 --2- 172.21.15.3:0/3417833411 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc1a40a96c0 0x7fc1a40a9b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:36:56.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:56.268+0000 7fc1a27fc640 1 --2- 172.21.15.3:0/3417833411 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc1a40b6750 0x7fc1a40b6bb0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fc18c011930 tx=0x7fc18c011df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:56.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:56.269+0000 7fc19b7fe640 1 -- 172.21.15.3:0/3417833411 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc18c00faf0 con 0x7fc1a40b6750 2024-05-09T06:36:56.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:56.269+0000 7fc19b7fe640 1 -- 172.21.15.3:0/3417833411 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc18c00fc90 con 0x7fc1a40b6750 2024-05-09T06:36:56.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:56.269+0000 7fc19b7fe640 1 -- 172.21.15.3:0/3417833411 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc18c004030 con 0x7fc1a40b6750 2024-05-09T06:36:56.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:56.269+0000 7fc1a94f5640 1 -- 172.21.15.3:0/3417833411 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1a4136ba0 con 0x7fc1a40b6750 2024-05-09T06:36:56.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:56.269+0000 7fc1a94f5640 1 -- 172.21.15.3:0/3417833411 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1a40b32e0 con 0x7fc1a40b6750 2024-05-09T06:36:56.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:56.270+0000 7fc1a94f5640 1 -- 172.21.15.3:0/3417833411 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1a4069550 con 0x7fc1a40b6750 2024-05-09T06:36:56.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:56.270+0000 7fc19b7fe640 1 -- 172.21.15.3:0/3417833411 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc18c012040 con 0x7fc1a40b6750 2024-05-09T06:36:56.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:56.271+0000 7fc19b7fe640 1 --2- 172.21.15.3:0/3417833411 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fc18406be30 0x7fc18406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:56.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:56.271+0000 7fc19b7fe640 1 -- 172.21.15.3:0/3417833411 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7fc18c08af50 con 0x7fc1a40b6750 2024-05-09T06:36:56.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:56.271+0000 7fc1a2ffd640 1 --2- 172.21.15.3:0/3417833411 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fc18406be30 0x7fc18406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:56.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:56.271+0000 7fc1a2ffd640 1 --2- 172.21.15.3:0/3417833411 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fc18406be30 0x7fc18406e2f0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fc19c000c00 tx=0x7fc19c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:56.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:56.275+0000 7fc19b7fe640 1 -- 172.21.15.3:0/3417833411 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fc18c019070 con 0x7fc1a40b6750 2024-05-09T06:36:56.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:56.462+0000 7fc1a94f5640 1 -- 172.21.15.3:0/3417833411 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fc1a4145080 con 0x7fc1a40b6750 2024-05-09T06:36:56.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:56.545+0000 7fc19b7fe640 1 -- 172.21.15.3:0/3417833411 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v30) ==== 138+0+0 (secure 0 0 0) 0x7fc18c059c50 con 0x7fc1a40b6750 2024-05-09T06:36:56.526 INFO:teuthology.orchestra.run.smithi003.stderr:pool 'cephfs_metadata' created 2024-05-09T06:36:56.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:56.547+0000 7fc1a94f5640 1 -- 172.21.15.3:0/3417833411 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fc18406be30 msgr2=0x7fc18406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:56.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:56.547+0000 7fc1a94f5640 1 --2- 172.21.15.3:0/3417833411 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fc18406be30 0x7fc18406e2f0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fc19c000c00 tx=0x7fc19c009040 comp rx=0 tx=0).stop 2024-05-09T06:36:56.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:56.547+0000 7fc1a94f5640 1 -- 172.21.15.3:0/3417833411 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc1a40b6750 msgr2=0x7fc1a40b6bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:56.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:56.547+0000 7fc1a94f5640 1 --2- 172.21.15.3:0/3417833411 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc1a40b6750 0x7fc1a40b6bb0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fc18c011930 tx=0x7fc18c011df0 comp rx=0 tx=0).stop 2024-05-09T06:36:56.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:56.547+0000 7fc1a94f5640 1 -- 172.21.15.3:0/3417833411 shutdown_connections 2024-05-09T06:36:56.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:56.547+0000 7fc1a94f5640 1 --2- 172.21.15.3:0/3417833411 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fc18406be30 0x7fc18406e2f0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:56.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:56.547+0000 7fc1a94f5640 1 --2- 172.21.15.3:0/3417833411 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc1a40a96c0 0x7fc1a40a9b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:56.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:56.547+0000 7fc1a94f5640 1 --2- 172.21.15.3:0/3417833411 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc1a40b6750 0x7fc1a40b6bb0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:56.529 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:56.547+0000 7fc1a94f5640 1 --2- 172.21.15.3:0/3417833411 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc190004d40 0x7fc1a40b2da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:56.529 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:56.547+0000 7fc1a94f5640 1 -- 172.21.15.3:0/3417833411 >> 172.21.15.3:0/3417833411 conn(0x7fc1a40d2000 msgr2=0x7fc1a41478d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:56.529 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:56.547+0000 7fc1a94f5640 1 -- 172.21.15.3:0/3417833411 shutdown_connections 2024-05-09T06:36:56.529 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:56.547+0000 7fc1a94f5640 1 -- 172.21.15.3:0/3417833411 wait complete. 2024-05-09T06:36:56.541 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-05-09T06:36:56.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:56.673+0000 7f927ab42640 1 Processor -- start 2024-05-09T06:36:56.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:56.673+0000 7f927ab42640 1 -- start start 2024-05-09T06:36:56.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:56.674+0000 7f927ab42640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f927415e470 0x7f92740d2550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:56.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:56.674+0000 7f927ab42640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f92740cf660 0x7f92740b9c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:56.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:56.674+0000 7f927ab42640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f927415e850 con 0x7f92740d3c60 2024-05-09T06:36:56.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:56.674+0000 7f927ab42640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f92740ba1a0 con 0x7f927415e470 2024-05-09T06:36:56.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:56.674+0000 7f927ab42640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f92740b82c0 con 0x7f92740cf660 2024-05-09T06:36:56.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:56.674+0000 7f92788de640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f927415e470 0x7f92740d2550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:56.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:56.674+0000 7f92788de640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f927415e470 0x7f92740d2550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:56784/0 (socket says 172.21.15.3:56784) 2024-05-09T06:36:56.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:56.674+0000 7f92788de640 1 -- 172.21.15.3:0/3638583036 learned_addr learned my addr 172.21.15.3:0/3638583036 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:36:56.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:56.674+0000 7f92790df640 1 --2- 172.21.15.3:0/3638583036 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f92740cf660 0x7f92740b9c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:56.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:56.675+0000 7f92788de640 1 -- 172.21.15.3:0/3638583036 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f92740cf660 msgr2=0x7f92740b9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:56.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:56.675+0000 7f92788de640 1 --2- 172.21.15.3:0/3638583036 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f92740cf660 0x7f92740b9c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:56.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:56.675+0000 7f92788de640 1 -- 172.21.15.3:0/3638583036 >> v1:172.21.15.190:6789/0 conn(0x7f92740d3c60 legacy=0x7f92740cef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:56.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:56.675+0000 7f92788de640 1 -- 172.21.15.3:0/3638583036 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92740d2bc0 con 0x7f927415e470 2024-05-09T06:36:56.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:56.675+0000 7f92790df640 1 --2- 172.21.15.3:0/3638583036 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f92740cf660 0x7f92740b9c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:36:56.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:56.675+0000 7f92788de640 1 --2- 172.21.15.3:0/3638583036 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f927415e470 0x7f92740d2550 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f9268009870 tx=0x7f926802a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cbd3396201852bb server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:56.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:56.675+0000 7f92737fe640 1 -- 172.21.15.3:0/3638583036 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9268037070 con 0x7f927415e470 2024-05-09T06:36:56.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:56.675+0000 7f92737fe640 1 -- 172.21.15.3:0/3638583036 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9268032440 con 0x7f927415e470 2024-05-09T06:36:56.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:56.675+0000 7f92737fe640 1 -- 172.21.15.3:0/3638583036 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9268032810 con 0x7f927415e470 2024-05-09T06:36:56.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:56.676+0000 7f927ab42640 1 -- 172.21.15.3:0/3638583036 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f927415e470 msgr2=0x7f92740d2550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:56.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:56.676+0000 7f927ab42640 1 --2- 172.21.15.3:0/3638583036 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f927415e470 0x7f92740d2550 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f9268009870 tx=0x7f926802a1c0 comp rx=0 tx=0).stop 2024-05-09T06:36:56.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:56.676+0000 7f927ab42640 1 -- 172.21.15.3:0/3638583036 shutdown_connections 2024-05-09T06:36:56.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:56.676+0000 7f927ab42640 1 --2- 172.21.15.3:0/3638583036 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f92740cf660 0x7f92740b9c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:56.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:56.676+0000 7f927ab42640 1 --2- 172.21.15.3:0/3638583036 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f927415e470 0x7f92740d2550 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:56.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:56.676+0000 7f927ab42640 1 -- 172.21.15.3:0/3638583036 >> 172.21.15.3:0/3638583036 conn(0x7f92740675c0 msgr2=0x7f927406c2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:56.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:56.676+0000 7f927ab42640 1 -- 172.21.15.3:0/3638583036 shutdown_connections 2024-05-09T06:36:56.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:56.676+0000 7f927ab42640 1 -- 172.21.15.3:0/3638583036 wait complete. 2024-05-09T06:36:56.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:56.677+0000 7f927ab42640 1 Processor -- start 2024-05-09T06:36:56.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:56.677+0000 7f927ab42640 1 -- start start 2024-05-09T06:36:56.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:56.678+0000 7f927ab42640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f92740cf660 0x7f92740c9ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:56.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:56.678+0000 7f927ab42640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f92740d3c60 0x7f92741475c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:56.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:56.678+0000 7f927ab42640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f927415e470 0x7f92740cbc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:56.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:56.678+0000 7f927ab42640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f92740d3460 con 0x7f927415e470 2024-05-09T06:36:56.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:56.678+0000 7f927ab42640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f92740d3180 con 0x7f92740d3c60 2024-05-09T06:36:56.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:56.678+0000 7f927ab42640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f92740d3740 con 0x7f92740cf660 2024-05-09T06:36:56.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:56.679+0000 7f92790df640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f927415e470 0x7f92740cbc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:56.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:56.679+0000 7f92790df640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f927415e470 0x7f92740cbc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:56794/0 (socket says 172.21.15.3:56794) 2024-05-09T06:36:56.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:56.679+0000 7f92790df640 1 -- 172.21.15.3:0/3589921948 learned_addr learned my addr 172.21.15.3:0/3589921948 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:36:56.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:56.679+0000 7f92788de640 1 --2- 172.21.15.3:0/3589921948 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f92740cf660 0x7f92740c9ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:56.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:56.679+0000 7f9273fff640 1 --2- 172.21.15.3:0/3589921948 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f92740d3c60 0x7f92741475c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:56.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:56.679+0000 7f92790df640 1 -- 172.21.15.3:0/3589921948 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f92740cf660 msgr2=0x7f92740c9ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:56.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:56.679+0000 7f92790df640 1 --2- 172.21.15.3:0/3589921948 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f92740cf660 0x7f92740c9ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:56.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:56.679+0000 7f92790df640 1 -- 172.21.15.3:0/3589921948 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f92740d3c60 msgr2=0x7f92741475c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:56.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:56.679+0000 7f92790df640 1 --2- 172.21.15.3:0/3589921948 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f92740d3c60 0x7f92741475c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:56.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:56.679+0000 7f92790df640 1 -- 172.21.15.3:0/3589921948 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92741f3ca0 con 0x7f927415e470 2024-05-09T06:36:56.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:56.679+0000 7f92790df640 1 --2- 172.21.15.3:0/3589921948 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f927415e470 0x7f92740cbc10 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f926c007f20 tx=0x7f926c00d560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:56.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:56.679+0000 7f92717fa640 1 -- 172.21.15.3:0/3589921948 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f926c01a070 con 0x7f927415e470 2024-05-09T06:36:56.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:56.679+0000 7f92717fa640 1 -- 172.21.15.3:0/3589921948 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f926c003b90 con 0x7f927415e470 2024-05-09T06:36:56.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:56.679+0000 7f927ab42640 1 -- 172.21.15.3:0/3589921948 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92740cc150 con 0x7f927415e470 2024-05-09T06:36:56.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:56.679+0000 7f92717fa640 1 -- 172.21.15.3:0/3589921948 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f926c004030 con 0x7f927415e470 2024-05-09T06:36:56.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:56.679+0000 7f927ab42640 1 -- 172.21.15.3:0/3589921948 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92740cc9e0 con 0x7f927415e470 2024-05-09T06:36:56.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:56.680+0000 7f927ab42640 1 -- 172.21.15.3:0/3589921948 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9274069550 con 0x7f927415e470 2024-05-09T06:36:56.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:56.681+0000 7f92717fa640 1 -- 172.21.15.3:0/3589921948 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f926c01d070 con 0x7f927415e470 2024-05-09T06:36:56.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:56.681+0000 7f92717fa640 1 --2- 172.21.15.3:0/3589921948 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f925806be30 0x7f925806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:56.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:56.681+0000 7f92717fa640 1 -- 172.21.15.3:0/3589921948 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 5227+0+0 (secure 0 0 0) 0x7f926c08afa0 con 0x7f927415e470 2024-05-09T06:36:56.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:56.682+0000 7f92788de640 1 --2- 172.21.15.3:0/3589921948 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f925806be30 0x7f925806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:56.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:56.682+0000 7f92788de640 1 --2- 172.21.15.3:0/3589921948 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f925806be30 0x7f925806e2f0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f9268034000 tx=0x7f926802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:56.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:56.686+0000 7f92717fa640 1 -- 172.21.15.3:0/3589921948 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f926c010070 con 0x7f927415e470 2024-05-09T06:36:56.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:56.868+0000 7f927ab42640 1 -- 172.21.15.3:0/3589921948 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f927406b250 con 0x7f927415e470 2024-05-09T06:36:57.533 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:57.552+0000 7f92717fa640 1 -- 172.21.15.3:0/3589921948 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v31) ==== 170+0+0 (secure 0 0 0) 0x7f926c059440 con 0x7f927415e470 2024-05-09T06:36:57.533 INFO:teuthology.orchestra.run.smithi003.stderr:pool 'cephfs_data' created 2024-05-09T06:36:57.535 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:57.555+0000 7f927ab42640 1 -- 172.21.15.3:0/3589921948 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f925806be30 msgr2=0x7f925806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:57.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:57.555+0000 7f927ab42640 1 --2- 172.21.15.3:0/3589921948 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f925806be30 0x7f925806e2f0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f9268034000 tx=0x7f926802e000 comp rx=0 tx=0).stop 2024-05-09T06:36:57.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:57.555+0000 7f927ab42640 1 -- 172.21.15.3:0/3589921948 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f927415e470 msgr2=0x7f92740cbc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:57.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:57.555+0000 7f927ab42640 1 --2- 172.21.15.3:0/3589921948 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f927415e470 0x7f92740cbc10 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f926c007f20 tx=0x7f926c00d560 comp rx=0 tx=0).stop 2024-05-09T06:36:57.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:57.555+0000 7f927ab42640 1 -- 172.21.15.3:0/3589921948 shutdown_connections 2024-05-09T06:36:57.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:57.555+0000 7f927ab42640 1 --2- 172.21.15.3:0/3589921948 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f925806be30 0x7f925806e2f0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:57.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:57.555+0000 7f927ab42640 1 --2- 172.21.15.3:0/3589921948 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f927415e470 0x7f92740cbc10 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:57.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:57.555+0000 7f927ab42640 1 --2- 172.21.15.3:0/3589921948 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f92740d3c60 0x7f92741475c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:57.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:57.555+0000 7f927ab42640 1 --2- 172.21.15.3:0/3589921948 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f92740cf660 0x7f92740c9ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:57.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:57.555+0000 7f927ab42640 1 -- 172.21.15.3:0/3589921948 >> 172.21.15.3:0/3589921948 conn(0x7f92740675c0 msgr2=0x7f927415b1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:57.537 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:57.555+0000 7f927ab42640 1 -- 172.21.15.3:0/3589921948 shutdown_connections 2024-05-09T06:36:57.537 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:57.556+0000 7f927ab42640 1 -- 172.21.15.3:0/3589921948 wait complete. 2024-05-09T06:36:57.561 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-05-09T06:36:57.673 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:57.693+0000 7f33ccd29640 1 Processor -- start 2024-05-09T06:36:57.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:57.693+0000 7f33ccd29640 1 -- start start 2024-05-09T06:36:57.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:57.694+0000 7f33ccd29640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f33c815e0b0 0x7f33c8161e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:57.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:57.694+0000 7f33ccd29640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f33c806b250 0x7f33c806b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:57.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:57.694+0000 7f33ccd29640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f33c806cad0 con 0x7f33c815e490 2024-05-09T06:36:57.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:57.694+0000 7f33ccd29640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f33c80b3eb0 con 0x7f33c806b250 2024-05-09T06:36:57.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:57.694+0000 7f33ccd29640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f33c80b4190 con 0x7f33c815e0b0 2024-05-09T06:36:57.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:57.694+0000 7f33c6d9d640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f33c806b250 0x7f33c806b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:57.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:57.694+0000 7f33c6d9d640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f33c806b250 0x7f33c806b6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:56804/0 (socket says 172.21.15.3:56804) 2024-05-09T06:36:57.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:57.694+0000 7f33c6d9d640 1 -- 172.21.15.3:0/3925942702 learned_addr learned my addr 172.21.15.3:0/3925942702 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:36:57.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:57.694+0000 7f33c659c640 1 --2- 172.21.15.3:0/3925942702 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f33c815e0b0 0x7f33c8161e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:57.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:57.694+0000 7f33c6d9d640 1 -- 172.21.15.3:0/3925942702 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f33c815e0b0 msgr2=0x7f33c8161e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:57.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:57.694+0000 7f33c6d9d640 1 --2- 172.21.15.3:0/3925942702 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f33c815e0b0 0x7f33c8161e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:57.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:57.694+0000 7f33c6d9d640 1 -- 172.21.15.3:0/3925942702 >> v1:172.21.15.190:6789/0 conn(0x7f33c815e490 legacy=0x7f33c81623c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:57.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:57.694+0000 7f33c6d9d640 1 -- 172.21.15.3:0/3925942702 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33c80b4470 con 0x7f33c806b250 2024-05-09T06:36:57.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:57.694+0000 7f33c659c640 1 --2- 172.21.15.3:0/3925942702 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f33c815e0b0 0x7f33c8161e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:36:57.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:57.695+0000 7f33c6d9d640 1 --2- 172.21.15.3:0/3925942702 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f33c806b250 0x7f33c806b6c0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f33c0009e50 tx=0x7f33c002a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74d9f3c94ec254da server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:57.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:57.695+0000 7f33c559a640 1 -- 172.21.15.3:0/3925942702 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33c0036070 con 0x7f33c806b250 2024-05-09T06:36:57.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:57.695+0000 7f33c559a640 1 -- 172.21.15.3:0/3925942702 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f33c0031600 con 0x7f33c806b250 2024-05-09T06:36:57.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:57.695+0000 7f33c559a640 1 -- 172.21.15.3:0/3925942702 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33c00319d0 con 0x7f33c806b250 2024-05-09T06:36:57.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:57.696+0000 7f33ccd29640 1 -- 172.21.15.3:0/3925942702 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f33c806b250 msgr2=0x7f33c806b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:57.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:57.696+0000 7f33ccd29640 1 --2- 172.21.15.3:0/3925942702 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f33c806b250 0x7f33c806b6c0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f33c0009e50 tx=0x7f33c002a6c0 comp rx=0 tx=0).stop 2024-05-09T06:36:57.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:57.696+0000 7f33ccd29640 1 -- 172.21.15.3:0/3925942702 shutdown_connections 2024-05-09T06:36:57.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:57.696+0000 7f33ccd29640 1 --2- 172.21.15.3:0/3925942702 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f33c806b250 0x7f33c806b6c0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:57.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:57.696+0000 7f33ccd29640 1 --2- 172.21.15.3:0/3925942702 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f33c815e0b0 0x7f33c8161e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:57.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:57.696+0000 7f33ccd29640 1 -- 172.21.15.3:0/3925942702 >> 172.21.15.3:0/3925942702 conn(0x7f33c80d07b0 msgr2=0x7f33c80d0bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:57.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:57.696+0000 7f33ccd29640 1 -- 172.21.15.3:0/3925942702 shutdown_connections 2024-05-09T06:36:57.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:57.696+0000 7f33ccd29640 1 -- 172.21.15.3:0/3925942702 wait complete. 2024-05-09T06:36:57.680 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:57.697+0000 7f33ccd29640 1 Processor -- start 2024-05-09T06:36:57.680 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:57.697+0000 7f33ccd29640 1 -- start start 2024-05-09T06:36:57.680 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:57.698+0000 7f33ccd29640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f33c806b250 0x7f33c815fee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:57.680 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:57.698+0000 7f33ccd29640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f33c815e0b0 0x7f33c815a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:57.680 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:57.698+0000 7f33ccd29640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f33c815e490 0x7f33c815ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:57.680 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:57.698+0000 7f33ccd29640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f33c806da70 con 0x7f33c806b250 2024-05-09T06:36:57.680 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:57.698+0000 7f33ccd29640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f33c806d8f0 con 0x7f33c815e490 2024-05-09T06:36:57.680 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:57.698+0000 7f33ccd29640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f33c8162ad0 con 0x7f33c815e0b0 2024-05-09T06:36:57.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:57.698+0000 7f33c659c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f33c806b250 0x7f33c815fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:57.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:57.698+0000 7f33c659c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f33c806b250 0x7f33c815fee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:56812/0 (socket says 172.21.15.3:56812) 2024-05-09T06:36:57.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:57.698+0000 7f33c659c640 1 -- 172.21.15.3:0/4288811230 learned_addr learned my addr 172.21.15.3:0/4288811230 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:36:57.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:57.698+0000 7f33c6d9d640 1 --2- 172.21.15.3:0/4288811230 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f33c815e490 0x7f33c815ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:57.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:57.698+0000 7f33c5d9b640 1 --2- 172.21.15.3:0/4288811230 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f33c815e0b0 0x7f33c815a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:57.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:57.698+0000 7f33c659c640 1 -- 172.21.15.3:0/4288811230 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f33c815e0b0 msgr2=0x7f33c815a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:57.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:57.698+0000 7f33c659c640 1 --2- 172.21.15.3:0/4288811230 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f33c815e0b0 0x7f33c815a950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:57.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:57.698+0000 7f33c659c640 1 -- 172.21.15.3:0/4288811230 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f33c815e490 msgr2=0x7f33c815ae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:57.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:57.698+0000 7f33c659c640 1 --2- 172.21.15.3:0/4288811230 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f33c815e490 0x7f33c815ae90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:57.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:57.698+0000 7f33c659c640 1 -- 172.21.15.3:0/4288811230 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33c815d390 con 0x7f33c806b250 2024-05-09T06:36:57.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:57.698+0000 7f33c5d9b640 1 --2- 172.21.15.3:0/4288811230 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f33c815e0b0 0x7f33c815a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:36:57.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:57.698+0000 7f33c6d9d640 1 --2- 172.21.15.3:0/4288811230 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f33c815e490 0x7f33c815ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:36:57.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:57.699+0000 7f33c659c640 1 --2- 172.21.15.3:0/4288811230 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f33c806b250 0x7f33c815fee0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f33b800c7f0 tx=0x7f33b800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:57.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:57.699+0000 7f33b6ffd640 1 -- 172.21.15.3:0/4288811230 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33b8015070 con 0x7f33c806b250 2024-05-09T06:36:57.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:57.699+0000 7f33b6ffd640 1 -- 172.21.15.3:0/4288811230 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f33b80079d0 con 0x7f33c806b250 2024-05-09T06:36:57.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:57.699+0000 7f33ccd29640 1 -- 172.21.15.3:0/4288811230 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33c813ef90 con 0x7f33c806b250 2024-05-09T06:36:57.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:57.699+0000 7f33b6ffd640 1 -- 172.21.15.3:0/4288811230 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33b8007da0 con 0x7f33c806b250 2024-05-09T06:36:57.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:57.699+0000 7f33ccd29640 1 -- 172.21.15.3:0/4288811230 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33c8160420 con 0x7f33c806b250 2024-05-09T06:36:57.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:57.700+0000 7f33ccd29640 1 -- 172.21.15.3:0/4288811230 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f33c80adde0 con 0x7f33c806b250 2024-05-09T06:36:57.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:57.701+0000 7f33b6ffd640 1 -- 172.21.15.3:0/4288811230 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f33b8018070 con 0x7f33c806b250 2024-05-09T06:36:57.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:57.701+0000 7f33b6ffd640 1 --2- 172.21.15.3:0/4288811230 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f33a806be30 0x7f33a806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:57.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:57.701+0000 7f33b6ffd640 1 -- 172.21.15.3:0/4288811230 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 5609+0+0 (secure 0 0 0) 0x7f33b808ac90 con 0x7f33c806b250 2024-05-09T06:36:57.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:57.701+0000 7f33c5d9b640 1 --2- 172.21.15.3:0/4288811230 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f33a806be30 0x7f33a806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:57.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:57.702+0000 7f33c5d9b640 1 --2- 172.21.15.3:0/4288811230 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f33a806be30 0x7f33a806e2f0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f33c81579a0 tx=0x7f33bc009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:57.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:57.706+0000 7f33b6ffd640 1 -- 172.21.15.3:0/4288811230 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f33c80adde0 con 0x7f33c806b250 2024-05-09T06:36:57.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:57.927+0000 7f33ccd29640 1 -- 172.21.15.3:0/4288811230 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f33c80b3a50 con 0x7f33c806b250 2024-05-09T06:36:58.534 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:58.554+0000 7f33b6ffd640 1 -- 172.21.15.3:0/4288811230 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 4 and data pool 5 v13) ==== 172+0+0 (secure 0 0 0) 0x7f33b8059730 con 0x7f33c806b250 2024-05-09T06:36:58.535 INFO:teuthology.orchestra.run.smithi003.stderr:new fs with metadata pool 4 and data pool 5 2024-05-09T06:36:58.540 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:58.559+0000 7f33ccd29640 1 -- 172.21.15.3:0/4288811230 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f33a806be30 msgr2=0x7f33a806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:58.540 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:58.559+0000 7f33ccd29640 1 --2- 172.21.15.3:0/4288811230 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f33a806be30 0x7f33a806e2f0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f33c81579a0 tx=0x7f33bc009040 comp rx=0 tx=0).stop 2024-05-09T06:36:58.540 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:58.559+0000 7f33ccd29640 1 -- 172.21.15.3:0/4288811230 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f33c806b250 msgr2=0x7f33c815fee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:58.540 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:58.559+0000 7f33ccd29640 1 --2- 172.21.15.3:0/4288811230 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f33c806b250 0x7f33c815fee0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f33b800c7f0 tx=0x7f33b800cbf0 comp rx=0 tx=0).stop 2024-05-09T06:36:58.541 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:58.560+0000 7f33ccd29640 1 -- 172.21.15.3:0/4288811230 shutdown_connections 2024-05-09T06:36:58.541 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:58.560+0000 7f33ccd29640 1 --2- 172.21.15.3:0/4288811230 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f33a806be30 0x7f33a806e2f0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:58.541 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:58.560+0000 7f33ccd29640 1 --2- 172.21.15.3:0/4288811230 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f33c815e490 0x7f33c815ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:58.541 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:58.560+0000 7f33ccd29640 1 --2- 172.21.15.3:0/4288811230 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f33c815e0b0 0x7f33c815a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:58.542 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:58.560+0000 7f33ccd29640 1 --2- 172.21.15.3:0/4288811230 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f33c806b250 0x7f33c815fee0 secure :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f33b800c7f0 tx=0x7f33b800cbf0 comp rx=0 tx=0).stop 2024-05-09T06:36:58.542 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:58.560+0000 7f33ccd29640 1 -- 172.21.15.3:0/4288811230 >> 172.21.15.3:0/4288811230 conn(0x7f33c80d07b0 msgr2=0x7f33c806e510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:58.544 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:58.564+0000 7f33ccd29640 1 -- 172.21.15.3:0/4288811230 shutdown_connections 2024-05-09T06:36:58.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:58.565+0000 7f33ccd29640 1 -- 172.21.15.3:0/4288811230 wait complete. 2024-05-09T06:36:58.577 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T06:36:58.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:58.704+0000 7f166f116640 1 Processor -- start 2024-05-09T06:36:58.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:58.704+0000 7f166f116640 1 -- start start 2024-05-09T06:36:58.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:58.705+0000 7f166f116640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f16680cc6f0 0x7f16680b4250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:58.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:58.705+0000 7f166f116640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f16680b48e0 con 0x7f16680ccad0 2024-05-09T06:36:58.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:58.705+0000 7f166f116640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f16680b4bc0 con 0x7f16680cde50 2024-05-09T06:36:58.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:58.705+0000 7f166f116640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f16680b4ea0 con 0x7f16680cc6f0 2024-05-09T06:36:58.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:58.705+0000 7f166ceb2640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f16680cc6f0 0x7f16680b4250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:58.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:58.705+0000 7f166ceb2640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f16680cc6f0 0x7f16680b4250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:56820/0 (socket says 172.21.15.3:56820) 2024-05-09T06:36:58.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:58.705+0000 7f165ffff640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f16680ccad0 0x7f16680cd740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:53402/0 (socket says 172.21.15.3:53402) 2024-05-09T06:36:58.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:58.705+0000 7f166ceb2640 1 -- 172.21.15.3:0/398741671 learned_addr learned my addr 172.21.15.3:0/398741671 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:36:58.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:58.705+0000 7f166ceb2640 1 -- 172.21.15.3:0/398741671 >> v1:172.21.15.190:6789/0 conn(0x7f16680cde50 legacy=0x7f166806da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:58.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:58.705+0000 7f166ceb2640 1 -- 172.21.15.3:0/398741671 >> v1:172.21.15.3:6789/0 conn(0x7f16680ccad0 legacy=0x7f16680cd740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:58.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:58.705+0000 7f166ceb2640 1 -- 172.21.15.3:0/398741671 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16680b5180 con 0x7f16680cc6f0 2024-05-09T06:36:58.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:58.705+0000 7f165f7fe640 1 -- 172.21.15.3:0/398741671 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2509250266 0 0) 0x7f16680b48e0 con 0x7f16680ccad0 2024-05-09T06:36:58.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:58.706+0000 7f166ceb2640 1 --2- 172.21.15.3:0/398741671 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f16680cc6f0 0x7f16680b4250 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f1664009870 tx=0x7f166402a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d8fcfe962f85f09 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:58.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:58.706+0000 7f165f7fe640 1 -- 172.21.15.3:0/398741671 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1664037070 con 0x7f16680cc6f0 2024-05-09T06:36:58.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:58.706+0000 7f165f7fe640 1 -- 172.21.15.3:0/398741671 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f166402ae50 con 0x7f16680cc6f0 2024-05-09T06:36:58.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:58.706+0000 7f165f7fe640 1 -- 172.21.15.3:0/398741671 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1664032860 con 0x7f16680cc6f0 2024-05-09T06:36:58.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:58.707+0000 7f166f116640 1 -- 172.21.15.3:0/398741671 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f16680cc6f0 msgr2=0x7f16680b4250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:58.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:58.707+0000 7f166f116640 1 --2- 172.21.15.3:0/398741671 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f16680cc6f0 0x7f16680b4250 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f1664009870 tx=0x7f166402a300 comp rx=0 tx=0).stop 2024-05-09T06:36:58.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:58.707+0000 7f166f116640 1 -- 172.21.15.3:0/398741671 shutdown_connections 2024-05-09T06:36:58.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:58.707+0000 7f166f116640 1 --2- 172.21.15.3:0/398741671 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f16680cc6f0 0x7f16680b4250 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:58.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:58.707+0000 7f166f116640 1 -- 172.21.15.3:0/398741671 >> 172.21.15.3:0/398741671 conn(0x7f166806b430 msgr2=0x7f166806b810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:58.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:58.707+0000 7f166f116640 1 -- 172.21.15.3:0/398741671 shutdown_connections 2024-05-09T06:36:58.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:58.707+0000 7f166f116640 1 -- 172.21.15.3:0/398741671 wait complete. 2024-05-09T06:36:58.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:58.708+0000 7f166f116640 1 Processor -- start 2024-05-09T06:36:58.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:58.708+0000 7f166f116640 1 -- start start 2024-05-09T06:36:58.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:58.709+0000 7f166f116640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f16680cc6f0 0x7f1668148880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:58.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:58.709+0000 7f166f116640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f16680ccad0 0x7f166813e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:58.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:58.709+0000 7f166f116640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f16680cde50 0x7f166814be30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:58.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:58.709+0000 7f166f116640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f166815d2e0 con 0x7f16680cc6f0 2024-05-09T06:36:58.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:58.709+0000 7f166f116640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f166815d160 con 0x7f16680ccad0 2024-05-09T06:36:58.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:58.709+0000 7f166f116640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f166815d460 con 0x7f16680cde50 2024-05-09T06:36:58.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:58.709+0000 7f166ceb2640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f16680cc6f0 0x7f1668148880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:58.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:58.709+0000 7f166ceb2640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f16680cc6f0 0x7f1668148880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:56828/0 (socket says 172.21.15.3:56828) 2024-05-09T06:36:58.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:58.709+0000 7f166ceb2640 1 -- 172.21.15.3:0/3661195338 learned_addr learned my addr 172.21.15.3:0/3661195338 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:36:58.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:58.709+0000 7f165ffff640 1 --2- 172.21.15.3:0/3661195338 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f16680ccad0 0x7f166813e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:58.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:58.709+0000 7f166d6b3640 1 --2- 172.21.15.3:0/3661195338 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f16680cde50 0x7f166814be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:58.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:58.709+0000 7f166ceb2640 1 -- 172.21.15.3:0/3661195338 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f16680cde50 msgr2=0x7f166814be30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:58.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:58.709+0000 7f166ceb2640 1 --2- 172.21.15.3:0/3661195338 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f16680cde50 0x7f166814be30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:58.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:58.709+0000 7f166ceb2640 1 -- 172.21.15.3:0/3661195338 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f16680ccad0 msgr2=0x7f166813e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:58.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:58.709+0000 7f166ceb2640 1 --2- 172.21.15.3:0/3661195338 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f16680ccad0 0x7f166813e840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:58.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:58.709+0000 7f166ceb2640 1 -- 172.21.15.3:0/3661195338 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f166814c370 con 0x7f16680cc6f0 2024-05-09T06:36:58.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:58.709+0000 7f165ffff640 1 --2- 172.21.15.3:0/3661195338 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f16680ccad0 0x7f166813e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:36:58.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:58.709+0000 7f166d6b3640 1 --2- 172.21.15.3:0/3661195338 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f16680cde50 0x7f166814be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:36:58.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:58.710+0000 7f166ceb2640 1 --2- 172.21.15.3:0/3661195338 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f16680cc6f0 0x7f1668148880 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f1664009870 tx=0x7f16640046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:58.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:58.710+0000 7f165d7fa640 1 -- 172.21.15.3:0/3661195338 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1664042070 con 0x7f16680cc6f0 2024-05-09T06:36:58.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:58.710+0000 7f165d7fa640 1 -- 172.21.15.3:0/3661195338 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f166403e870 con 0x7f16680cc6f0 2024-05-09T06:36:58.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:58.710+0000 7f165d7fa640 1 -- 172.21.15.3:0/3661195338 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1664037040 con 0x7f16680cc6f0 2024-05-09T06:36:58.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:58.710+0000 7f166f116640 1 -- 172.21.15.3:0/3661195338 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16681f57f0 con 0x7f16680cc6f0 2024-05-09T06:36:58.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:58.710+0000 7f166f116640 1 -- 172.21.15.3:0/3661195338 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16681f5db0 con 0x7f16680cc6f0 2024-05-09T06:36:58.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:58.711+0000 7f166f116640 1 -- 172.21.15.3:0/3661195338 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1668156a60 con 0x7f16680cc6f0 2024-05-09T06:36:58.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:58.711+0000 7f165d7fa640 1 -- 172.21.15.3:0/3661195338 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1664045070 con 0x7f16680cc6f0 2024-05-09T06:36:58.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:58.712+0000 7f165d7fa640 1 --2- 172.21.15.3:0/3661195338 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f165406be30 0x7f165406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:58.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:58.712+0000 7f165d7fa640 1 -- 172.21.15.3:0/3661195338 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f16640abb60 con 0x7f16680cc6f0 2024-05-09T06:36:58.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:58.712+0000 7f165ffff640 1 --2- 172.21.15.3:0/3661195338 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f165406be30 0x7f165406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:58.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:58.713+0000 7f165ffff640 1 --2- 172.21.15.3:0/3661195338 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f165406be30 0x7f165406e2f0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f166806b880 tx=0x7f1658007c50 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:58.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:58.717+0000 7f165d7fa640 1 -- 172.21.15.3:0/3661195338 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f1668156a60 con 0x7f16680cc6f0 2024-05-09T06:36:58.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:58.900+0000 7f166f116640 1 -- 172.21.15.3:0/3661195338 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f166804f660 con 0x7f16680cc6f0 2024-05-09T06:36:58.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:58.902+0000 7f165d7fa640 1 -- 172.21.15.3:0/3661195338 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v32) ==== 74+0+16598 (secure 0 0 0) 0x7f166407a590 con 0x7f16680cc6f0 2024-05-09T06:36:58.883 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-09T06:36:58.883 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":32,"fsid":"16ed737d-2665-431a-8ac6-ef242be3975a","created":"2024-05-09T06:35:42.541368+0000","modified":"2024-05-09T06:36:57.929110+0000","last_up_change":"2024-05-09T06:35:53.778274+0000","last_in_change":"2024-05-09T06:35:44.780586+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T06:35:54.819999+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-05-09T06:36:56.494700+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-05-09T06:36:56.900006+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fae3a8db-4826-4f3c-b33a-12c09bb5df6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":1897849827},{"type":"v1","addr":"172.21.15.3:6817","nonce":1897849827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":1897849827},{"type":"v1","addr":"172.21.15.3:6819","nonce":1897849827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":1897849827},{"type":"v1","addr":"172.21.15.3:6823","nonce":1897849827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":1897849827},{"type":"v1","addr":"172.21.15.3:6821","nonce":1897849827}]},"public_addr":"172.21.15.3:6817/1897849827","cluster_addr":"172.21.15.3:6819/1897849827","heartbeat_back_addr":"172.21.15.3:6823/1897849827","heartbeat_front_addr":"172.21.15.3:6821/1897849827","state":["exists","up"]},{"osd":1,"uuid":"646e882d-4ce4-4720-8c84-f71df563a35e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":926805498},{"type":"v1","addr":"172.21.15.3:6801","nonce":926805498}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":926805498},{"type":"v1","addr":"172.21.15.3:6803","nonce":926805498}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":926805498},{"type":"v1","addr":"172.21.15.3:6807","nonce":926805498}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":926805498},{"type":"v1","addr":"172.21.15.3:6805","nonce":926805498}]},"public_addr":"172.21.15.3:6801/926805498","cluster_addr":"172.21.15.3:6803/926805498","heartbeat_back_addr":"172.21.15.3:6807/926805498","heartbeat_front_addr":"172.21.15.3:6805/926805498","state":["exists","up"]},{"osd":2,"uuid":"1abbadb5-a570-4359-9643-acff3ede35a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":801295451},{"type":"v1","addr":"172.21.15.3:6809","nonce":801295451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":801295451},{"type":"v1","addr":"172.21.15.3:6811","nonce":801295451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":801295451},{"type":"v1","addr":"172.21.15.3:6815","nonce":801295451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":801295451},{"type":"v1","addr":"172.21.15.3:6813","nonce":801295451}]},"public_addr":"172.21.15.3:6809/801295451","cluster_addr":"172.21.15.3:6811/801295451","heartbeat_back_addr":"172.21.15.3:6815/801295451","heartbeat_front_addr":"172.21.15.3:6813/801295451","state":["exists","up"]},{"osd":3,"uuid":"62ac4dc1-2371-4dd2-acd3-e2dbb11aedf5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":1903389857},{"type":"v1","addr":"172.21.15.3:6825","nonce":1903389857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":1903389857},{"type":"v1","addr":"172.21.15.3:6827","nonce":1903389857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":1903389857},{"type":"v1","addr":"172.21.15.3:6831","nonce":1903389857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":1903389857},{"type":"v1","addr":"172.21.15.3:6829","nonce":1903389857}]},"public_addr":"172.21.15.3:6825/1903389857","cluster_addr":"172.21.15.3:6827/1903389857","heartbeat_back_addr":"172.21.15.3:6831/1903389857","heartbeat_front_addr":"172.21.15.3:6829/1903389857","state":["exists","up"]},{"osd":4,"uuid":"ae808398-8326-4d7c-9319-a2a0eafffb59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":2143624576},{"type":"v1","addr":"172.21.15.190:6801","nonce":2143624576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":2143624576},{"type":"v1","addr":"172.21.15.190:6803","nonce":2143624576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":2143624576},{"type":"v1","addr":"172.21.15.190:6807","nonce":2143624576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":2143624576},{"type":"v1","addr":"172.21.15.190:6805","nonce":2143624576}]},"public_addr":"172.21.15.190:6801/2143624576","cluster_addr":"172.21.15.190:6803/2143624576","heartbeat_back_addr":"172.21.15.190:6807/2143624576","heartbeat_front_addr":"172.21.15.190:6805/2143624576","state":["exists","up"]},{"osd":5,"uuid":"918adf50-c440-48ac-9a8d-4837562d7275","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":1766923126},{"type":"v1","addr":"172.21.15.190:6814","nonce":1766923126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":1766923126},{"type":"v1","addr":"172.21.15.190:6818","nonce":1766923126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":1766923126},{"type":"v1","addr":"172.21.15.190:6823","nonce":1766923126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":1766923126},{"type":"v1","addr":"172.21.15.190:6821","nonce":1766923126}]},"public_addr":"172.21.15.190:6814/1766923126","cluster_addr":"172.21.15.190:6818/1766923126","heartbeat_back_addr":"172.21.15.190:6823/1766923126","heartbeat_front_addr":"172.21.15.190:6821/1766923126","state":["exists","up"]},{"osd":6,"uuid":"ac2882bc-2a7c-463c-8f6c-76ff49828adc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":342414248},{"type":"v1","addr":"172.21.15.190:6825","nonce":342414248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":342414248},{"type":"v1","addr":"172.21.15.190:6827","nonce":342414248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":342414248},{"type":"v1","addr":"172.21.15.190:6831","nonce":342414248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":342414248},{"type":"v1","addr":"172.21.15.190:6829","nonce":342414248}]},"public_addr":"172.21.15.190:6825/342414248","cluster_addr":"172.21.15.190:6827/342414248","heartbeat_back_addr":"172.21.15.190:6831/342414248","heartbeat_front_addr":"172.21.15.190:6829/342414248","state":["exists","up"]},{"osd":7,"uuid":"dd4e767a-4730-4c70-8bd5-3f088140a5a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":537849891},{"type":"v1","addr":"172.21.15.190:6809","nonce":537849891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":537849891},{"type":"v1","addr":"172.21.15.190:6811","nonce":537849891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6817","nonce":537849891},{"type":"v1","addr":"172.21.15.190:6819","nonce":537849891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6813","nonce":537849891},{"type":"v1","addr":"172.21.15.190:6815","nonce":537849891}]},"public_addr":"172.21.15.190:6809/537849891","cluster_addr":"172.21.15.190:6811/537849891","heartbeat_back_addr":"172.21.15.190:6819/537849891","heartbeat_front_addr":"172.21.15.190:6815/537849891","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.889784+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.974002+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.973549+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.955396+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.928471+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.902418+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.958696+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.926474+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T06:36:58.885 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:58.904+0000 7f166f116640 1 -- 172.21.15.3:0/3661195338 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f165406be30 msgr2=0x7f165406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:58.885 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:58.904+0000 7f166f116640 1 --2- 172.21.15.3:0/3661195338 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f165406be30 0x7f165406e2f0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f166806b880 tx=0x7f1658007c50 comp rx=0 tx=0).stop 2024-05-09T06:36:58.885 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:58.904+0000 7f166f116640 1 -- 172.21.15.3:0/3661195338 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f16680cc6f0 msgr2=0x7f1668148880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:58.885 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:58.904+0000 7f166f116640 1 --2- 172.21.15.3:0/3661195338 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f16680cc6f0 0x7f1668148880 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f1664009870 tx=0x7f16640046c0 comp rx=0 tx=0).stop 2024-05-09T06:36:58.885 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:58.904+0000 7f166f116640 1 -- 172.21.15.3:0/3661195338 shutdown_connections 2024-05-09T06:36:58.885 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:58.904+0000 7f166f116640 1 --2- 172.21.15.3:0/3661195338 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f165406be30 0x7f165406e2f0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:58.885 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:58.904+0000 7f166f116640 1 --2- 172.21.15.3:0/3661195338 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f16680cde50 0x7f166814be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:58.885 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:58.904+0000 7f166f116640 1 --2- 172.21.15.3:0/3661195338 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f16680ccad0 0x7f166813e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:58.885 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:58.904+0000 7f166f116640 1 --2- 172.21.15.3:0/3661195338 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f16680cc6f0 0x7f1668148880 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:58.886 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:58.904+0000 7f166f116640 1 -- 172.21.15.3:0/3661195338 >> 172.21.15.3:0/3661195338 conn(0x7f166806b430 msgr2=0x7f16680cd740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:58.886 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:58.904+0000 7f166f116640 1 -- 172.21.15.3:0/3661195338 shutdown_connections 2024-05-09T06:36:58.886 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:58.904+0000 7f166f116640 1 -- 172.21.15.3:0/3661195338 wait complete. 2024-05-09T06:36:58.896 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T06:36:59.000 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:59.019+0000 7fcecb612640 1 Processor -- start 2024-05-09T06:36:59.000 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:59.020+0000 7fcecb612640 1 -- start start 2024-05-09T06:36:59.002 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:59.020+0000 7fcecb612640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fcec415e480 0x7fcec40d1dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:59.002 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:59.020+0000 7fcecb612640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcec40d2310 0x7fcec40d27c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:59.002 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:59.020+0000 7fcecb612640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcec40d4e50 con 0x7fcec40d2d00 2024-05-09T06:36:59.002 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:59.020+0000 7fcecb612640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fcec40d0270 con 0x7fcec415e480 2024-05-09T06:36:59.002 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:59.020+0000 7fcecb612640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fcec40cf1d0 con 0x7fcec40d2310 2024-05-09T06:36:59.002 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:59.021+0000 7fcec93ae640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fcec415e480 0x7fcec40d1dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:59.002 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:59.021+0000 7fcec93ae640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fcec415e480 0x7fcec40d1dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:56836/0 (socket says 172.21.15.3:56836) 2024-05-09T06:36:59.002 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:59.021+0000 7fcec93ae640 1 -- 172.21.15.3:0/1288545535 learned_addr learned my addr 172.21.15.3:0/1288545535 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:36:59.002 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:59.021+0000 7fcec8bad640 1 --2- 172.21.15.3:0/1288545535 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcec40d2310 0x7fcec40d27c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:59.002 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:59.021+0000 7fcec93ae640 1 -- 172.21.15.3:0/1288545535 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcec40d2310 msgr2=0x7fcec40d27c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:59.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:59.021+0000 7fcec93ae640 1 --2- 172.21.15.3:0/1288545535 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcec40d2310 0x7fcec40d27c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:59.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:59.021+0000 7fcec93ae640 1 -- 172.21.15.3:0/1288545535 >> v1:172.21.15.3:6789/0 conn(0x7fcec40d2d00 legacy=0x7fcec4162070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:59.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:59.021+0000 7fcec93ae640 1 -- 172.21.15.3:0/1288545535 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcec40cf4b0 con 0x7fcec415e480 2024-05-09T06:36:59.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:59.021+0000 7fcebbfff640 1 -- 172.21.15.3:0/1288545535 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 254443324 0 0) 0x7fcebc001490 con 0x7fcec40d2d00 2024-05-09T06:36:59.004 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:59.022+0000 7fcec93ae640 1 --2- 172.21.15.3:0/1288545535 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcec415e480 0x7fcec40d1dd0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fcec0009870 tx=0x7fcec002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=937533be995dd35d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:59.004 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:59.022+0000 7fcebbfff640 1 -- 172.21.15.3:0/1288545535 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcec0037070 con 0x7fcec415e480 2024-05-09T06:36:59.004 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:59.022+0000 7fcebbfff640 1 -- 172.21.15.3:0/1288545535 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcec0032440 con 0x7fcec415e480 2024-05-09T06:36:59.004 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:59.022+0000 7fcebbfff640 1 -- 172.21.15.3:0/1288545535 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcec0032810 con 0x7fcec415e480 2024-05-09T06:36:59.004 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:59.022+0000 7fcecb612640 1 -- 172.21.15.3:0/1288545535 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcec415e480 msgr2=0x7fcec40d1dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:59.004 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:59.022+0000 7fcecb612640 1 --2- 172.21.15.3:0/1288545535 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcec415e480 0x7fcec40d1dd0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fcec0009870 tx=0x7fcec002a270 comp rx=0 tx=0).stop 2024-05-09T06:36:59.005 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:59.022+0000 7fcecb612640 1 -- 172.21.15.3:0/1288545535 shutdown_connections 2024-05-09T06:36:59.005 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:59.022+0000 7fcecb612640 1 --2- 172.21.15.3:0/1288545535 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcec40d2310 0x7fcec40d27c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:59.005 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:59.022+0000 7fcecb612640 1 --2- 172.21.15.3:0/1288545535 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcec415e480 0x7fcec40d1dd0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:59.005 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:59.022+0000 7fcecb612640 1 -- 172.21.15.3:0/1288545535 >> 172.21.15.3:0/1288545535 conn(0x7fcec409d4d0 msgr2=0x7fcec40d4200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:59.005 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:59.023+0000 7fcecb612640 1 -- 172.21.15.3:0/1288545535 shutdown_connections 2024-05-09T06:36:59.005 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:59.023+0000 7fcecb612640 1 -- 172.21.15.3:0/1288545535 wait complete. 2024-05-09T06:36:59.005 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:59.024+0000 7fcecb612640 1 Processor -- start 2024-05-09T06:36:59.005 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:59.024+0000 7fcecb612640 1 -- start start 2024-05-09T06:36:59.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:59.024+0000 7fcecb612640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcec40d2310 0x7fcec415cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:59.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:59.024+0000 7fcecb612640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcec40d2d00 0x7fcec409f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:59.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:59.024+0000 7fcecb612640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcec415e480 0x7fcec40b2930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:59.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:59.024+0000 7fcecb612640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fcec40cfd50 con 0x7fcec40d2d00 2024-05-09T06:36:59.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:59.024+0000 7fcecb612640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fcec40cfa70 con 0x7fcec40d2310 2024-05-09T06:36:59.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:59.024+0000 7fcecb612640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fcec40b5ab0 con 0x7fcec415e480 2024-05-09T06:36:59.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:59.025+0000 7fcec8bad640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcec40d2d00 0x7fcec409f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:59.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:59.025+0000 7fcec8bad640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcec40d2d00 0x7fcec409f5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:56848/0 (socket says 172.21.15.3:56848) 2024-05-09T06:36:59.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:59.025+0000 7fcec8bad640 1 -- 172.21.15.3:0/1246814500 learned_addr learned my addr 172.21.15.3:0/1246814500 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:36:59.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:59.025+0000 7fcec9baf640 1 --2- 172.21.15.3:0/1246814500 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcec415e480 0x7fcec40b2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:59.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:59.025+0000 7fcec93ae640 1 --2- 172.21.15.3:0/1246814500 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcec40d2310 0x7fcec415cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:59.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:59.025+0000 7fcec8bad640 1 -- 172.21.15.3:0/1246814500 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcec415e480 msgr2=0x7fcec40b2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:59.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:59.025+0000 7fcec8bad640 1 --2- 172.21.15.3:0/1246814500 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcec415e480 0x7fcec40b2930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:59.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:59.025+0000 7fcec8bad640 1 -- 172.21.15.3:0/1246814500 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcec40d2310 msgr2=0x7fcec415cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:59.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:59.025+0000 7fcec8bad640 1 --2- 172.21.15.3:0/1246814500 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcec40d2310 0x7fcec415cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:59.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:59.025+0000 7fcec8bad640 1 -- 172.21.15.3:0/1246814500 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcec40a1050 con 0x7fcec40d2d00 2024-05-09T06:36:59.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:59.025+0000 7fcec8bad640 1 --2- 172.21.15.3:0/1246814500 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcec40d2d00 0x7fcec409f5d0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fceb400d600 tx=0x7fceb400dac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:59.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:59.025+0000 7fceb9ffb640 1 -- 172.21.15.3:0/1246814500 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fceb4004240 con 0x7fcec40d2d00 2024-05-09T06:36:59.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:59.025+0000 7fceb9ffb640 1 -- 172.21.15.3:0/1246814500 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fceb40043e0 con 0x7fcec40d2d00 2024-05-09T06:36:59.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:59.025+0000 7fcecb612640 1 -- 172.21.15.3:0/1246814500 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcec409ffb0 con 0x7fcec40d2d00 2024-05-09T06:36:59.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:59.025+0000 7fceb9ffb640 1 -- 172.21.15.3:0/1246814500 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fceb4003830 con 0x7fcec40d2d00 2024-05-09T06:36:59.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:59.025+0000 7fcecb612640 1 -- 172.21.15.3:0/1246814500 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcec40a0570 con 0x7fcec40d2d00 2024-05-09T06:36:59.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:59.027+0000 7fcecb612640 1 -- 172.21.15.3:0/1246814500 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcec404f660 con 0x7fcec40d2d00 2024-05-09T06:36:59.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:59.027+0000 7fceb9ffb640 1 -- 172.21.15.3:0/1246814500 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fceb40039d0 con 0x7fcec40d2d00 2024-05-09T06:36:59.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:59.027+0000 7fceb9ffb640 1 --2- 172.21.15.3:0/1246814500 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fcea806be30 0x7fcea806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:59.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:59.027+0000 7fceb9ffb640 1 -- 172.21.15.3:0/1246814500 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7fceb408b1d0 con 0x7fcec40d2d00 2024-05-09T06:36:59.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:59.028+0000 7fcec93ae640 1 --2- 172.21.15.3:0/1246814500 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fcea806be30 0x7fcea806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:59.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:59.028+0000 7fcec93ae640 1 --2- 172.21.15.3:0/1246814500 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fcea806be30 0x7fcea806e2f0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fcec0034000 tx=0x7fcec002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:59.012 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:59.032+0000 7fceb9ffb640 1 -- 172.21.15.3:0/1246814500 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fceb401c360 con 0x7fcec40d2d00 2024-05-09T06:36:59.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:59.216+0000 7fcecb612640 1 -- 172.21.15.3:0/1246814500 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fcec40d1500 con 0x7fcec40d2d00 2024-05-09T06:36:59.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:59.217+0000 7fceb9ffb640 1 -- 172.21.15.3:0/1246814500 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v32) ==== 74+0+16598 (secure 0 0 0) 0x7fceb4059ae0 con 0x7fcec40d2d00 2024-05-09T06:36:59.198 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-09T06:36:59.198 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":32,"fsid":"16ed737d-2665-431a-8ac6-ef242be3975a","created":"2024-05-09T06:35:42.541368+0000","modified":"2024-05-09T06:36:57.929110+0000","last_up_change":"2024-05-09T06:35:53.778274+0000","last_in_change":"2024-05-09T06:35:44.780586+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T06:35:54.819999+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-05-09T06:36:56.494700+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-05-09T06:36:56.900006+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fae3a8db-4826-4f3c-b33a-12c09bb5df6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":1897849827},{"type":"v1","addr":"172.21.15.3:6817","nonce":1897849827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":1897849827},{"type":"v1","addr":"172.21.15.3:6819","nonce":1897849827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":1897849827},{"type":"v1","addr":"172.21.15.3:6823","nonce":1897849827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":1897849827},{"type":"v1","addr":"172.21.15.3:6821","nonce":1897849827}]},"public_addr":"172.21.15.3:6817/1897849827","cluster_addr":"172.21.15.3:6819/1897849827","heartbeat_back_addr":"172.21.15.3:6823/1897849827","heartbeat_front_addr":"172.21.15.3:6821/1897849827","state":["exists","up"]},{"osd":1,"uuid":"646e882d-4ce4-4720-8c84-f71df563a35e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":926805498},{"type":"v1","addr":"172.21.15.3:6801","nonce":926805498}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":926805498},{"type":"v1","addr":"172.21.15.3:6803","nonce":926805498}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":926805498},{"type":"v1","addr":"172.21.15.3:6807","nonce":926805498}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":926805498},{"type":"v1","addr":"172.21.15.3:6805","nonce":926805498}]},"public_addr":"172.21.15.3:6801/926805498","cluster_addr":"172.21.15.3:6803/926805498","heartbeat_back_addr":"172.21.15.3:6807/926805498","heartbeat_front_addr":"172.21.15.3:6805/926805498","state":["exists","up"]},{"osd":2,"uuid":"1abbadb5-a570-4359-9643-acff3ede35a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":801295451},{"type":"v1","addr":"172.21.15.3:6809","nonce":801295451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":801295451},{"type":"v1","addr":"172.21.15.3:6811","nonce":801295451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":801295451},{"type":"v1","addr":"172.21.15.3:6815","nonce":801295451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":801295451},{"type":"v1","addr":"172.21.15.3:6813","nonce":801295451}]},"public_addr":"172.21.15.3:6809/801295451","cluster_addr":"172.21.15.3:6811/801295451","heartbeat_back_addr":"172.21.15.3:6815/801295451","heartbeat_front_addr":"172.21.15.3:6813/801295451","state":["exists","up"]},{"osd":3,"uuid":"62ac4dc1-2371-4dd2-acd3-e2dbb11aedf5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":1903389857},{"type":"v1","addr":"172.21.15.3:6825","nonce":1903389857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":1903389857},{"type":"v1","addr":"172.21.15.3:6827","nonce":1903389857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":1903389857},{"type":"v1","addr":"172.21.15.3:6831","nonce":1903389857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":1903389857},{"type":"v1","addr":"172.21.15.3:6829","nonce":1903389857}]},"public_addr":"172.21.15.3:6825/1903389857","cluster_addr":"172.21.15.3:6827/1903389857","heartbeat_back_addr":"172.21.15.3:6831/1903389857","heartbeat_front_addr":"172.21.15.3:6829/1903389857","state":["exists","up"]},{"osd":4,"uuid":"ae808398-8326-4d7c-9319-a2a0eafffb59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":2143624576},{"type":"v1","addr":"172.21.15.190:6801","nonce":2143624576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":2143624576},{"type":"v1","addr":"172.21.15.190:6803","nonce":2143624576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":2143624576},{"type":"v1","addr":"172.21.15.190:6807","nonce":2143624576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":2143624576},{"type":"v1","addr":"172.21.15.190:6805","nonce":2143624576}]},"public_addr":"172.21.15.190:6801/2143624576","cluster_addr":"172.21.15.190:6803/2143624576","heartbeat_back_addr":"172.21.15.190:6807/2143624576","heartbeat_front_addr":"172.21.15.190:6805/2143624576","state":["exists","up"]},{"osd":5,"uuid":"918adf50-c440-48ac-9a8d-4837562d7275","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":1766923126},{"type":"v1","addr":"172.21.15.190:6814","nonce":1766923126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":1766923126},{"type":"v1","addr":"172.21.15.190:6818","nonce":1766923126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":1766923126},{"type":"v1","addr":"172.21.15.190:6823","nonce":1766923126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":1766923126},{"type":"v1","addr":"172.21.15.190:6821","nonce":1766923126}]},"public_addr":"172.21.15.190:6814/1766923126","cluster_addr":"172.21.15.190:6818/1766923126","heartbeat_back_addr":"172.21.15.190:6823/1766923126","heartbeat_front_addr":"172.21.15.190:6821/1766923126","state":["exists","up"]},{"osd":6,"uuid":"ac2882bc-2a7c-463c-8f6c-76ff49828adc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":342414248},{"type":"v1","addr":"172.21.15.190:6825","nonce":342414248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":342414248},{"type":"v1","addr":"172.21.15.190:6827","nonce":342414248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":342414248},{"type":"v1","addr":"172.21.15.190:6831","nonce":342414248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":342414248},{"type":"v1","addr":"172.21.15.190:6829","nonce":342414248}]},"public_addr":"172.21.15.190:6825/342414248","cluster_addr":"172.21.15.190:6827/342414248","heartbeat_back_addr":"172.21.15.190:6831/342414248","heartbeat_front_addr":"172.21.15.190:6829/342414248","state":["exists","up"]},{"osd":7,"uuid":"dd4e767a-4730-4c70-8bd5-3f088140a5a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":537849891},{"type":"v1","addr":"172.21.15.190:6809","nonce":537849891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":537849891},{"type":"v1","addr":"172.21.15.190:6811","nonce":537849891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6817","nonce":537849891},{"type":"v1","addr":"172.21.15.190:6819","nonce":537849891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6813","nonce":537849891},{"type":"v1","addr":"172.21.15.190:6815","nonce":537849891}]},"public_addr":"172.21.15.190:6809/537849891","cluster_addr":"172.21.15.190:6811/537849891","heartbeat_back_addr":"172.21.15.190:6819/537849891","heartbeat_front_addr":"172.21.15.190:6815/537849891","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.889784+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.974002+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.973549+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.955396+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.928471+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.902418+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.958696+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.926474+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T06:36:59.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:59.219+0000 7fcecb612640 1 -- 172.21.15.3:0/1246814500 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fcea806be30 msgr2=0x7fcea806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:59.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:59.219+0000 7fcecb612640 1 --2- 172.21.15.3:0/1246814500 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fcea806be30 0x7fcea806e2f0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fcec0034000 tx=0x7fcec002e000 comp rx=0 tx=0).stop 2024-05-09T06:36:59.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:59.219+0000 7fcecb612640 1 -- 172.21.15.3:0/1246814500 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcec40d2d00 msgr2=0x7fcec409f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:59.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:59.219+0000 7fcecb612640 1 --2- 172.21.15.3:0/1246814500 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcec40d2d00 0x7fcec409f5d0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fceb400d600 tx=0x7fceb400dac0 comp rx=0 tx=0).stop 2024-05-09T06:36:59.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:59.219+0000 7fcecb612640 1 -- 172.21.15.3:0/1246814500 shutdown_connections 2024-05-09T06:36:59.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:59.219+0000 7fcecb612640 1 --2- 172.21.15.3:0/1246814500 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fcea806be30 0x7fcea806e2f0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:59.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:59.219+0000 7fcecb612640 1 --2- 172.21.15.3:0/1246814500 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcec415e480 0x7fcec40b2930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:59.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:59.219+0000 7fcecb612640 1 --2- 172.21.15.3:0/1246814500 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcec40d2d00 0x7fcec409f5d0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:59.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:59.219+0000 7fcecb612640 1 --2- 172.21.15.3:0/1246814500 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcec40d2310 0x7fcec415cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:59.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:59.219+0000 7fcecb612640 1 -- 172.21.15.3:0/1246814500 >> 172.21.15.3:0/1246814500 conn(0x7fcec409d4d0 msgr2=0x7fcec40b5270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:59.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:59.219+0000 7fcecb612640 1 -- 172.21.15.3:0/1246814500 shutdown_connections 2024-05-09T06:36:59.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:59.219+0000 7fcecb612640 1 -- 172.21.15.3:0/1246814500 wait complete. 2024-05-09T06:36:59.211 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-05-09T06:36:59.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:59.337+0000 7f2f2ba3f640 1 Processor -- start 2024-05-09T06:36:59.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:59.337+0000 7f2f2ba3f640 1 -- start start 2024-05-09T06:36:59.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:59.338+0000 7f2f2ba3f640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2f2415e5c0 0x7f2f240d2800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:59.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:59.338+0000 7f2f2ba3f640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f2f240d4c40 0x7f2f240d0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:59.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:59.338+0000 7f2f2ba3f640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2f2415e9a0 con 0x7f2f240d1130 2024-05-09T06:36:59.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:59.338+0000 7f2f2ba3f640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f2f240d31c0 con 0x7f2f240d4c40 2024-05-09T06:36:59.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:59.338+0000 7f2f2ba3f640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f2f240d50a0 con 0x7f2f2415e5c0 2024-05-09T06:36:59.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:59.339+0000 7f2f297db640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2f2415e5c0 0x7f2f240d2800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:59.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:59.339+0000 7f2f28fda640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f2f240d4c40 0x7f2f240d0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:59.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:59.339+0000 7f2f29fdc640 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7f2f240d1130 0x7f2f240ced30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.3:36022/0 (socket says 172.21.15.3:36022) 2024-05-09T06:36:59.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:59.339+0000 7f2f29fdc640 1 -- 172.21.15.3:0/394882364 learned_addr learned my addr 172.21.15.3:0/394882364 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-05-09T06:36:59.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:59.339+0000 7f2f1bfff640 1 -- 172.21.15.3:0/394882364 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3705990964 0 0) 0x7f2f2415e9a0 con 0x7f2f240d1130 2024-05-09T06:36:59.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:59.339+0000 7f2f1bfff640 1 -- 172.21.15.3:0/394882364 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2f10003520 con 0x7f2f240d1130 2024-05-09T06:36:59.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:59.339+0000 7f2f28fda640 1 -- 172.21.15.3:0/394882364 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2f2415e5c0 msgr2=0x7f2f240d2800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:36:59.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:59.339+0000 7f2f28fda640 1 --2- 172.21.15.3:0/394882364 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2f2415e5c0 0x7f2f240d2800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:59.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:59.339+0000 7f2f28fda640 1 -- 172.21.15.3:0/394882364 >> v1:172.21.15.190:6789/0 conn(0x7f2f240d1130 legacy=0x7f2f240ced30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:59.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:59.339+0000 7f2f28fda640 1 -- 172.21.15.3:0/394882364 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f240d2ee0 con 0x7f2f240d4c40 2024-05-09T06:36:59.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:59.339+0000 7f2f297db640 1 --2- 172.21.15.3:0/394882364 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2f2415e5c0 0x7f2f240d2800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T06:36:59.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:59.340+0000 7f2f28fda640 1 --2- 172.21.15.3:0/394882364 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2f240d4c40 0x7f2f240d0bf0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f2f140097a0 tx=0x7f2f1402a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7bef338c3a87937c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:59.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:59.340+0000 7f2f1bfff640 1 -- 172.21.15.3:0/394882364 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f14037070 con 0x7f2f240d4c40 2024-05-09T06:36:59.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:59.340+0000 7f2f1bfff640 1 -- 172.21.15.3:0/394882364 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f14032440 con 0x7f2f240d4c40 2024-05-09T06:36:59.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:59.340+0000 7f2f1bfff640 1 -- 172.21.15.3:0/394882364 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f140327f0 con 0x7f2f240d4c40 2024-05-09T06:36:59.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:59.341+0000 7f2f2ba3f640 1 -- 172.21.15.3:0/394882364 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2f240d4c40 msgr2=0x7f2f240d0bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:59.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:59.341+0000 7f2f2ba3f640 1 --2- 172.21.15.3:0/394882364 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2f240d4c40 0x7f2f240d0bf0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f2f140097a0 tx=0x7f2f1402a180 comp rx=0 tx=0).stop 2024-05-09T06:36:59.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:59.341+0000 7f2f2ba3f640 1 -- 172.21.15.3:0/394882364 shutdown_connections 2024-05-09T06:36:59.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:59.341+0000 7f2f2ba3f640 1 --2- 172.21.15.3:0/394882364 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2f240d4c40 0x7f2f240d0bf0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:59.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:59.341+0000 7f2f2ba3f640 1 --2- 172.21.15.3:0/394882364 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2f2415e5c0 0x7f2f240d2800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:59.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:59.341+0000 7f2f2ba3f640 1 -- 172.21.15.3:0/394882364 >> 172.21.15.3:0/394882364 conn(0x7f2f2409d500 msgr2=0x7f2f240d4220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:36:59.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:59.341+0000 7f2f2ba3f640 1 -- 172.21.15.3:0/394882364 shutdown_connections 2024-05-09T06:36:59.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:59.341+0000 7f2f2ba3f640 1 -- 172.21.15.3:0/394882364 wait complete. 2024-05-09T06:36:59.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:59.342+0000 7f2f2ba3f640 1 Processor -- start 2024-05-09T06:36:59.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:59.342+0000 7f2f2ba3f640 1 -- start start 2024-05-09T06:36:59.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:59.343+0000 7f2f2ba3f640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2f240d1130 0x7f2f240cc420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:59.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:59.343+0000 7f2f2ba3f640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2f240d4c40 0x7f2f241572a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:59.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:59.343+0000 7f2f2ba3f640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2f2415e5c0 0x7f2f240b6ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:59.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:59.343+0000 7f2f2ba3f640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f2f240b42c0 con 0x7f2f2415e5c0 2024-05-09T06:36:59.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:59.343+0000 7f2f2ba3f640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f2f240b3fe0 con 0x7f2f240d1130 2024-05-09T06:36:59.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:59.343+0000 7f2f2ba3f640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f2f240b57f0 con 0x7f2f240d4c40 2024-05-09T06:36:59.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:59.343+0000 7f2f29fdc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2f2415e5c0 0x7f2f240b6ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:59.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:59.343+0000 7f2f29fdc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2f2415e5c0 0x7f2f240b6ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:56852/0 (socket says 172.21.15.3:56852) 2024-05-09T06:36:59.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:59.343+0000 7f2f29fdc640 1 -- 172.21.15.3:0/742437623 learned_addr learned my addr 172.21.15.3:0/742437623 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:36:59.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:59.343+0000 7f2f297db640 1 --2- 172.21.15.3:0/742437623 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2f240d1130 0x7f2f240cc420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:59.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:59.343+0000 7f2f28fda640 1 --2- 172.21.15.3:0/742437623 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2f240d4c40 0x7f2f241572a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:59.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:59.343+0000 7f2f29fdc640 1 -- 172.21.15.3:0/742437623 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2f240d4c40 msgr2=0x7f2f241572a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:59.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:59.343+0000 7f2f29fdc640 1 --2- 172.21.15.3:0/742437623 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2f240d4c40 0x7f2f241572a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:59.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:59.343+0000 7f2f29fdc640 1 -- 172.21.15.3:0/742437623 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2f240d1130 msgr2=0x7f2f240cc420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:36:59.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:59.343+0000 7f2f29fdc640 1 --2- 172.21.15.3:0/742437623 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2f240d1130 0x7f2f240cc420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:36:59.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:59.343+0000 7f2f29fdc640 1 -- 172.21.15.3:0/742437623 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f240b7410 con 0x7f2f2415e5c0 2024-05-09T06:36:59.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:59.343+0000 7f2f29fdc640 1 --2- 172.21.15.3:0/742437623 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2f2415e5c0 0x7f2f240b6ed0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f2f1c00cb40 tx=0x7f2f1c00a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:59.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:59.344+0000 7f2f19ffb640 1 -- 172.21.15.3:0/742437623 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f1c017070 con 0x7f2f2415e5c0 2024-05-09T06:36:59.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:59.344+0000 7f2f2ba3f640 1 -- 172.21.15.3:0/742437623 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f240b82a0 con 0x7f2f2415e5c0 2024-05-09T06:36:59.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:59.344+0000 7f2f19ffb640 1 -- 172.21.15.3:0/742437623 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f1c00aca0 con 0x7f2f2415e5c0 2024-05-09T06:36:59.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:59.344+0000 7f2f19ffb640 1 -- 172.21.15.3:0/742437623 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f1c012620 con 0x7f2f2415e5c0 2024-05-09T06:36:59.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:59.344+0000 7f2f2ba3f640 1 -- 172.21.15.3:0/742437623 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f240b7c50 con 0x7f2f2415e5c0 2024-05-09T06:36:59.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:59.345+0000 7f2f2ba3f640 1 -- 172.21.15.3:0/742437623 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f24069550 con 0x7f2f2415e5c0 2024-05-09T06:36:59.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:59.345+0000 7f2f19ffb640 1 -- 172.21.15.3:0/742437623 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2f1c01a070 con 0x7f2f2415e5c0 2024-05-09T06:36:59.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:59.346+0000 7f2f19ffb640 1 --2- 172.21.15.3:0/742437623 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f2f0806be60 0x7f2f0806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:36:59.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:59.346+0000 7f2f19ffb640 1 -- 172.21.15.3:0/742437623 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f2f1c08abf0 con 0x7f2f2415e5c0 2024-05-09T06:36:59.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:59.346+0000 7f2f297db640 1 --2- 172.21.15.3:0/742437623 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f2f0806be60 0x7f2f0806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:36:59.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:59.347+0000 7f2f297db640 1 --2- 172.21.15.3:0/742437623 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f2f0806be60 0x7f2f0806e320 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f2f20009560 tx=0x7f2f20009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:36:59.331 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:59.350+0000 7f2f19ffb640 1 -- 172.21.15.3:0/742437623 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f2f1c058f60 con 0x7f2f2415e5c0 2024-05-09T06:36:59.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:36:59.574+0000 7f2f2ba3f640 1 -- 172.21.15.3:0/742437623 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f2f240ccd00 con 0x7f2f2415e5c0 2024-05-09T06:37:00.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:00.575+0000 7f2f19ffb640 1 -- 172.21.15.3:0/742437623 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v16) ==== 118+0+0 (secure 0 0 0) 0x7f2f1c05d260 con 0x7f2f2415e5c0 2024-05-09T06:37:00.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:00.577+0000 7f2f2ba3f640 1 -- 172.21.15.3:0/742437623 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f2f0806be60 msgr2=0x7f2f0806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:00.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:00.577+0000 7f2f2ba3f640 1 --2- 172.21.15.3:0/742437623 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f2f0806be60 0x7f2f0806e320 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f2f20009560 tx=0x7f2f20009190 comp rx=0 tx=0).stop 2024-05-09T06:37:00.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:00.577+0000 7f2f2ba3f640 1 -- 172.21.15.3:0/742437623 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2f2415e5c0 msgr2=0x7f2f240b6ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:00.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:00.578+0000 7f2f2ba3f640 1 --2- 172.21.15.3:0/742437623 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2f2415e5c0 0x7f2f240b6ed0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f2f1c00cb40 tx=0x7f2f1c00a060 comp rx=0 tx=0).stop 2024-05-09T06:37:00.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:00.578+0000 7f2f2ba3f640 1 -- 172.21.15.3:0/742437623 shutdown_connections 2024-05-09T06:37:00.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:00.578+0000 7f2f2ba3f640 1 --2- 172.21.15.3:0/742437623 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f2f0806be60 0x7f2f0806e320 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:00.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:00.578+0000 7f2f2ba3f640 1 --2- 172.21.15.3:0/742437623 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2f2415e5c0 0x7f2f240b6ed0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:00.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:00.578+0000 7f2f2ba3f640 1 --2- 172.21.15.3:0/742437623 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2f240d4c40 0x7f2f241572a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:00.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:00.578+0000 7f2f2ba3f640 1 --2- 172.21.15.3:0/742437623 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2f240d1130 0x7f2f240cc420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:00.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:00.578+0000 7f2f2ba3f640 1 -- 172.21.15.3:0/742437623 >> 172.21.15.3:0/742437623 conn(0x7f2f2409d500 msgr2=0x7f2f2415d770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:00.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:00.578+0000 7f2f2ba3f640 1 -- 172.21.15.3:0/742437623 shutdown_connections 2024-05-09T06:37:00.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:00.578+0000 7f2f2ba3f640 1 -- 172.21.15.3:0/742437623 wait complete. 2024-05-09T06:37:00.572 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-05-09T06:37:00.572 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-05-09T06:37:00.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:00.706+0000 7f8cec25d640 1 Processor -- start 2024-05-09T06:37:00.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:00.706+0000 7f8cec25d640 1 -- start start 2024-05-09T06:37:00.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:00.707+0000 7f8cec25d640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8ce415c0f0 0x7f8ce4160730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:00.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:00.707+0000 7f8cec25d640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f8ce40b4e30 0x7f8ce406e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:00.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:00.707+0000 7f8cec25d640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8ce4160c70 con 0x7f8ce406e790 2024-05-09T06:37:00.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:00.707+0000 7f8cec25d640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f8ce406ec00 con 0x7f8ce40b4e30 2024-05-09T06:37:00.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:00.707+0000 7f8cec25d640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f8ce40b40f0 con 0x7f8ce415c0f0 2024-05-09T06:37:00.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:00.707+0000 7f8ce97f8640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f8ce40b4e30 0x7f8ce406e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:00.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:00.707+0000 7f8ce9ff9640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8ce415c0f0 0x7f8ce4160730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:00.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:00.707+0000 7f8ce97f8640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f8ce40b4e30 0x7f8ce406e250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.3:51776/0 (socket says 172.21.15.3:51776) 2024-05-09T06:37:00.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:00.707+0000 7f8ce9ff9640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8ce415c0f0 0x7f8ce4160730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.3:42532/0 (socket says 172.21.15.3:42532) 2024-05-09T06:37:00.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:00.707+0000 7f8ce97f8640 1 -- 172.21.15.3:0/2728135907 learned_addr learned my addr 172.21.15.3:0/2728135907 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:37:00.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:00.707+0000 7f8cea7fa640 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7f8ce406e790 0x7f8ce40cd5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.3:37650/0 (socket says 172.21.15.3:37650) 2024-05-09T06:37:00.690 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:00.708+0000 7f8ce97f8640 1 -- 172.21.15.3:0/2728135907 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8ce415c0f0 msgr2=0x7f8ce4160730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:37:00.690 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:00.708+0000 7f8ce97f8640 1 --2- 172.21.15.3:0/2728135907 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8ce415c0f0 0x7f8ce4160730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:00.690 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:00.708+0000 7f8ce97f8640 1 -- 172.21.15.3:0/2728135907 >> v1:172.21.15.190:6789/0 conn(0x7f8ce406e790 legacy=0x7f8ce40cd5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:00.690 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:00.708+0000 7f8ce97f8640 1 -- 172.21.15.3:0/2728135907 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ce40b3050 con 0x7f8ce40b4e30 2024-05-09T06:37:00.690 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:00.708+0000 7f8ce9ff9640 1 --2- 172.21.15.3:0/2728135907 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8ce415c0f0 0x7f8ce4160730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T06:37:00.690 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:00.708+0000 7f8ce97f8640 1 --2- 172.21.15.3:0/2728135907 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8ce40b4e30 0x7f8ce406e250 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f8cd4009870 tx=0x7f8cd402a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fdd31fbd6848fbc6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:00.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:00.709+0000 7f8ce8ff7640 1 -- 172.21.15.3:0/2728135907 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cd4037070 con 0x7f8ce40b4e30 2024-05-09T06:37:00.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:00.709+0000 7f8ce8ff7640 1 -- 172.21.15.3:0/2728135907 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8cd4032440 con 0x7f8ce40b4e30 2024-05-09T06:37:00.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:00.709+0000 7f8ce8ff7640 1 -- 172.21.15.3:0/2728135907 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cd4032810 con 0x7f8ce40b4e30 2024-05-09T06:37:00.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:00.709+0000 7f8cec25d640 1 -- 172.21.15.3:0/2728135907 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8ce40b4e30 msgr2=0x7f8ce406e250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:00.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:00.709+0000 7f8cec25d640 1 --2- 172.21.15.3:0/2728135907 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8ce40b4e30 0x7f8ce406e250 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f8cd4009870 tx=0x7f8cd402a1c0 comp rx=0 tx=0).stop 2024-05-09T06:37:00.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:00.709+0000 7f8cec25d640 1 -- 172.21.15.3:0/2728135907 shutdown_connections 2024-05-09T06:37:00.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:00.709+0000 7f8cec25d640 1 --2- 172.21.15.3:0/2728135907 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8ce40b4e30 0x7f8ce406e250 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:00.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:00.709+0000 7f8cec25d640 1 --2- 172.21.15.3:0/2728135907 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8ce415c0f0 0x7f8ce4160730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:00.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:00.709+0000 7f8cec25d640 1 -- 172.21.15.3:0/2728135907 >> 172.21.15.3:0/2728135907 conn(0x7f8ce40b1c20 msgr2=0x7f8ce40b2030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:00.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:00.709+0000 7f8cec25d640 1 -- 172.21.15.3:0/2728135907 shutdown_connections 2024-05-09T06:37:00.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:00.709+0000 7f8cec25d640 1 -- 172.21.15.3:0/2728135907 wait complete. 2024-05-09T06:37:00.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:00.711+0000 7f8cec25d640 1 Processor -- start 2024-05-09T06:37:00.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:00.711+0000 7f8cec25d640 1 -- start start 2024-05-09T06:37:00.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:00.711+0000 7f8cec25d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8ce406e790 0x7f8ce4156160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:00.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:00.711+0000 7f8cec25d640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8ce40b4e30 0x7f8ce41f5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:00.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:00.711+0000 7f8cec25d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8ce415c0f0 0x7f8ce41f5ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:00.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:00.711+0000 7f8cec25d640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f8ce40b38f0 con 0x7f8ce415c0f0 2024-05-09T06:37:00.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:00.711+0000 7f8cec25d640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f8ce40b3610 con 0x7f8ce406e790 2024-05-09T06:37:00.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:00.711+0000 7f8cec25d640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f8ce40b3bd0 con 0x7f8ce40b4e30 2024-05-09T06:37:00.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:00.711+0000 7f8cea7fa640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8ce415c0f0 0x7f8ce41f5ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:00.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:00.712+0000 7f8cea7fa640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8ce415c0f0 0x7f8ce41f5ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:39374/0 (socket says 172.21.15.3:39374) 2024-05-09T06:37:00.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:00.712+0000 7f8cea7fa640 1 -- 172.21.15.3:0/1563286393 learned_addr learned my addr 172.21.15.3:0/1563286393 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:37:00.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:00.712+0000 7f8ce97f8640 1 --2- 172.21.15.3:0/1563286393 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8ce40b4e30 0x7f8ce41f5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:00.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:00.712+0000 7f8ce9ff9640 1 --2- 172.21.15.3:0/1563286393 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8ce406e790 0x7f8ce4156160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:00.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:00.712+0000 7f8cea7fa640 1 -- 172.21.15.3:0/1563286393 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8ce40b4e30 msgr2=0x7f8ce41f5580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:00.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:00.712+0000 7f8cea7fa640 1 --2- 172.21.15.3:0/1563286393 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8ce40b4e30 0x7f8ce41f5580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:00.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:00.712+0000 7f8cea7fa640 1 -- 172.21.15.3:0/1563286393 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8ce406e790 msgr2=0x7f8ce4156160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:00.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:00.712+0000 7f8cea7fa640 1 --2- 172.21.15.3:0/1563286393 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8ce406e790 0x7f8ce4156160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:00.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:00.712+0000 7f8cea7fa640 1 -- 172.21.15.3:0/1563286393 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ce4150880 con 0x7f8ce415c0f0 2024-05-09T06:37:00.695 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:00.712+0000 7f8ce9ff9640 1 --2- 172.21.15.3:0/1563286393 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8ce406e790 0x7f8ce4156160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:37:00.695 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:00.712+0000 7f8cea7fa640 1 --2- 172.21.15.3:0/1563286393 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8ce415c0f0 0x7f8ce41f5ac0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f8cdc00c7b0 tx=0x7f8cdc00cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:00.695 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:00.712+0000 7f8cda7fc640 1 -- 172.21.15.3:0/1563286393 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cdc017070 con 0x7f8ce415c0f0 2024-05-09T06:37:00.695 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:00.712+0000 7f8cda7fc640 1 -- 172.21.15.3:0/1563286393 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8cdc00a640 con 0x7f8ce415c0f0 2024-05-09T06:37:00.695 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:00.712+0000 7f8cec25d640 1 -- 172.21.15.3:0/1563286393 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ce4150b60 con 0x7f8ce415c0f0 2024-05-09T06:37:00.695 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:00.712+0000 7f8cda7fc640 1 -- 172.21.15.3:0/1563286393 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cdc00a9f0 con 0x7f8ce415c0f0 2024-05-09T06:37:00.695 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:00.712+0000 7f8cec25d640 1 -- 172.21.15.3:0/1563286393 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ce4151120 con 0x7f8ce415c0f0 2024-05-09T06:37:00.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:00.713+0000 7f8cec25d640 1 -- 172.21.15.3:0/1563286393 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ce4069550 con 0x7f8ce415c0f0 2024-05-09T06:37:00.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:00.714+0000 7f8cda7fc640 1 -- 172.21.15.3:0/1563286393 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8cdc01a070 con 0x7f8ce415c0f0 2024-05-09T06:37:00.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:00.715+0000 7f8cda7fc640 1 --2- 172.21.15.3:0/1563286393 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f8cc806be30 0x7f8cc806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:00.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:00.715+0000 7f8cda7fc640 1 -- 172.21.15.3:0/1563286393 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f8cdc08a320 con 0x7f8ce415c0f0 2024-05-09T06:37:00.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:00.715+0000 7f8ce9ff9640 1 --2- 172.21.15.3:0/1563286393 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f8cc806be30 0x7f8cc806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:00.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:00.715+0000 7f8ce9ff9640 1 --2- 172.21.15.3:0/1563286393 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f8cc806be30 0x7f8cc806e2f0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f8ce0009af0 tx=0x7f8ce0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:00.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:00.719+0000 7f8cda7fc640 1 -- 172.21.15.3:0/1563286393 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f8ce4160c70 con 0x7f8ce415c0f0 2024-05-09T06:37:00.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:00.940+0000 7f8cec25d640 1 -- 172.21.15.3:0/1563286393 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f8ce40b42e0 con 0x7f8ce415c0f0 2024-05-09T06:37:01.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:01.579+0000 7f8cda7fc640 1 -- 172.21.15.3:0/1563286393 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v17) ==== 122+0+0 (secure 0 0 0) 0x7f8cdc058710 con 0x7f8ce415c0f0 2024-05-09T06:37:01.564 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:01.583+0000 7f8cec25d640 1 -- 172.21.15.3:0/1563286393 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f8cc806be30 msgr2=0x7f8cc806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:01.564 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:01.583+0000 7f8cec25d640 1 --2- 172.21.15.3:0/1563286393 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f8cc806be30 0x7f8cc806e2f0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f8ce0009af0 tx=0x7f8ce0008040 comp rx=0 tx=0).stop 2024-05-09T06:37:01.564 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:01.583+0000 7f8cec25d640 1 -- 172.21.15.3:0/1563286393 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8ce415c0f0 msgr2=0x7f8ce41f5ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:01.564 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:01.583+0000 7f8cec25d640 1 --2- 172.21.15.3:0/1563286393 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8ce415c0f0 0x7f8ce41f5ac0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f8cdc00c7b0 tx=0x7f8cdc00cc70 comp rx=0 tx=0).stop 2024-05-09T06:37:01.564 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:01.583+0000 7f8cec25d640 1 -- 172.21.15.3:0/1563286393 shutdown_connections 2024-05-09T06:37:01.564 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:01.583+0000 7f8cec25d640 1 --2- 172.21.15.3:0/1563286393 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f8cc806be30 0x7f8cc806e2f0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:01.564 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:01.583+0000 7f8cec25d640 1 --2- 172.21.15.3:0/1563286393 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8ce415c0f0 0x7f8ce41f5ac0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:01.564 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:01.583+0000 7f8cec25d640 1 --2- 172.21.15.3:0/1563286393 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8ce40b4e30 0x7f8ce41f5580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:01.564 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:01.583+0000 7f8cec25d640 1 --2- 172.21.15.3:0/1563286393 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8ce406e790 0x7f8ce4156160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:01.564 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:01.583+0000 7f8cec25d640 1 -- 172.21.15.3:0/1563286393 >> 172.21.15.3:0/1563286393 conn(0x7f8ce40b1c20 msgr2=0x7f8ce4154e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:01.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:01.583+0000 7f8cec25d640 1 -- 172.21.15.3:0/1563286393 shutdown_connections 2024-05-09T06:37:01.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:01.583+0000 7f8cec25d640 1 -- 172.21.15.3:0/1563286393 wait complete. 2024-05-09T06:37:01.577 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-05-09T06:37:01.577 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T06:37:01.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:01.708+0000 7fab2b94f640 1 Processor -- start 2024-05-09T06:37:01.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:01.708+0000 7fab2b94f640 1 -- start start 2024-05-09T06:37:01.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:01.709+0000 7fab2b94f640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fab2413e0c0 0x7fab240cf720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:01.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:01.709+0000 7fab2b94f640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fab240cc870 0x7fab240ccd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:01.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:01.709+0000 7fab2b94f640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fab2413e4a0 con 0x7fab241408f0 2024-05-09T06:37:01.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:01.709+0000 7fab2b94f640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fab240cd240 con 0x7fab2413e0c0 2024-05-09T06:37:01.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:01.709+0000 7fab2b94f640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7fab24141410 con 0x7fab240cc870 2024-05-09T06:37:01.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:01.709+0000 7fab296eb640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fab2413e0c0 0x7fab240cf720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:01.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:01.709+0000 7fab296eb640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fab2413e0c0 0x7fab240cf720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:39378/0 (socket says 172.21.15.3:39378) 2024-05-09T06:37:01.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:01.709+0000 7fab296eb640 1 -- 172.21.15.3:0/1668111284 learned_addr learned my addr 172.21.15.3:0/1668111284 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:37:01.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:01.709+0000 7fab28eea640 1 --2- 172.21.15.3:0/1668111284 >> v2:172.21.15.190:3300/0 conn(0x7fab240cc870 0x7fab240ccd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:01.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:01.709+0000 7fab296eb640 1 -- 172.21.15.3:0/1668111284 >> v2:172.21.15.190:3300/0 conn(0x7fab240cc870 msgr2=0x7fab240ccd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:37:01.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:01.709+0000 7fab296eb640 1 --2- 172.21.15.3:0/1668111284 >> v2:172.21.15.190:3300/0 conn(0x7fab240cc870 0x7fab240ccd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:01.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:01.709+0000 7fab296eb640 1 -- 172.21.15.3:0/1668111284 >> v1:172.21.15.190:6789/0 conn(0x7fab241408f0 legacy=0x7fab24140d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:37:01.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:01.709+0000 7fab296eb640 1 -- 172.21.15.3:0/1668111284 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab240ce960 con 0x7fab2413e0c0 2024-05-09T06:37:01.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:01.709+0000 7fab28eea640 1 --2- 172.21.15.3:0/1668111284 >> v2:172.21.15.190:3300/0 conn(0x7fab240cc870 0x7fab240ccd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:37:01.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:01.710+0000 7fab296eb640 1 --2- 172.21.15.3:0/1668111284 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fab2413e0c0 0x7fab240cf720 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fab24072db0 tx=0x7fab1402a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb232be1b7a2748a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:01.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:01.710+0000 7fab23fff640 1 -- 172.21.15.3:0/1668111284 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab14036070 con 0x7fab2413e0c0 2024-05-09T06:37:01.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:01.710+0000 7fab23fff640 1 -- 172.21.15.3:0/1668111284 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab14031600 con 0x7fab2413e0c0 2024-05-09T06:37:01.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:01.710+0000 7fab23fff640 1 -- 172.21.15.3:0/1668111284 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab140319d0 con 0x7fab2413e0c0 2024-05-09T06:37:01.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:01.711+0000 7fab2b94f640 1 -- 172.21.15.3:0/1668111284 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fab2413e0c0 msgr2=0x7fab240cf720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:01.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:01.711+0000 7fab2b94f640 1 --2- 172.21.15.3:0/1668111284 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fab2413e0c0 0x7fab240cf720 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fab24072db0 tx=0x7fab1402a6f0 comp rx=0 tx=0).stop 2024-05-09T06:37:01.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:01.711+0000 7fab2b94f640 1 -- 172.21.15.3:0/1668111284 shutdown_connections 2024-05-09T06:37:01.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:01.711+0000 7fab2b94f640 1 --2- 172.21.15.3:0/1668111284 >> v2:172.21.15.190:3300/0 conn(0x7fab240cc870 0x7fab240ccd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:01.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:01.711+0000 7fab2b94f640 1 --2- 172.21.15.3:0/1668111284 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fab2413e0c0 0x7fab240cf720 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:01.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:01.711+0000 7fab2b94f640 1 -- 172.21.15.3:0/1668111284 >> 172.21.15.3:0/1668111284 conn(0x7fab2406b430 msgr2=0x7fab2406b810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:01.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:01.711+0000 7fab2b94f640 1 -- 172.21.15.3:0/1668111284 shutdown_connections 2024-05-09T06:37:01.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:01.711+0000 7fab2b94f640 1 -- 172.21.15.3:0/1668111284 wait complete. 2024-05-09T06:37:01.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:01.712+0000 7fab2b94f640 1 Processor -- start 2024-05-09T06:37:01.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:01.712+0000 7fab2b94f640 1 -- start start 2024-05-09T06:37:01.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:01.713+0000 7fab2b94f640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fab240cc870 0x7fab2413d680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:01.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:01.713+0000 7fab2b94f640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fab2413e0c0 0x7fab240b3d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:01.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:01.713+0000 7fab2b94f640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fab241408f0 0x7fab240b4280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:01.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:01.713+0000 7fab2b94f640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fab240cf200 con 0x7fab240cc870 2024-05-09T06:37:01.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:01.713+0000 7fab2b94f640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fab240cef20 con 0x7fab2413e0c0 2024-05-09T06:37:01.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:01.713+0000 7fab2b94f640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fab24141660 con 0x7fab241408f0 2024-05-09T06:37:01.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:01.713+0000 7fab296eb640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fab240cc870 0x7fab2413d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:01.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:01.713+0000 7fab296eb640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fab240cc870 0x7fab2413d680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:39388/0 (socket says 172.21.15.3:39388) 2024-05-09T06:37:01.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:01.713+0000 7fab296eb640 1 -- 172.21.15.3:0/3321668330 learned_addr learned my addr 172.21.15.3:0/3321668330 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:37:01.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:01.713+0000 7fab28eea640 1 --2- 172.21.15.3:0/3321668330 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fab2413e0c0 0x7fab240b3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:01.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:01.713+0000 7fab29eec640 1 --2- 172.21.15.3:0/3321668330 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fab241408f0 0x7fab240b4280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:01.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:01.713+0000 7fab296eb640 1 -- 172.21.15.3:0/3321668330 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fab241408f0 msgr2=0x7fab240b4280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:01.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:01.713+0000 7fab296eb640 1 --2- 172.21.15.3:0/3321668330 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fab241408f0 0x7fab240b4280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:01.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:01.713+0000 7fab296eb640 1 -- 172.21.15.3:0/3321668330 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fab2413e0c0 msgr2=0x7fab240b3d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:01.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:01.713+0000 7fab296eb640 1 --2- 172.21.15.3:0/3321668330 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fab2413e0c0 0x7fab240b3d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:01.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:01.713+0000 7fab296eb640 1 -- 172.21.15.3:0/3321668330 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab240b2a30 con 0x7fab240cc870 2024-05-09T06:37:01.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:01.713+0000 7fab28eea640 1 --2- 172.21.15.3:0/3321668330 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fab2413e0c0 0x7fab240b3d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:37:01.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:01.713+0000 7fab29eec640 1 --2- 172.21.15.3:0/3321668330 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fab241408f0 0x7fab240b4280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:37:01.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:01.713+0000 7fab296eb640 1 --2- 172.21.15.3:0/3321668330 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fab240cc870 0x7fab2413d680 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fab14034040 tx=0x7fab14004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:01.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:01.714+0000 7fab21ffb640 1 -- 172.21.15.3:0/3321668330 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab14036050 con 0x7fab240cc870 2024-05-09T06:37:01.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:01.714+0000 7fab21ffb640 1 -- 172.21.15.3:0/3321668330 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab14042070 con 0x7fab240cc870 2024-05-09T06:37:01.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:01.714+0000 7fab2b94f640 1 -- 172.21.15.3:0/3321668330 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab2413e720 con 0x7fab240cc870 2024-05-09T06:37:01.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:01.714+0000 7fab21ffb640 1 -- 172.21.15.3:0/3321668330 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab14009510 con 0x7fab240cc870 2024-05-09T06:37:01.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:01.714+0000 7fab2b94f640 1 -- 172.21.15.3:0/3321668330 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab2413ece0 con 0x7fab240cc870 2024-05-09T06:37:01.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:01.715+0000 7fab2b94f640 1 -- 172.21.15.3:0/3321668330 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab2404f660 con 0x7fab240cc870 2024-05-09T06:37:01.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:01.715+0000 7fab21ffb640 1 -- 172.21.15.3:0/3321668330 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fab14007c50 con 0x7fab240cc870 2024-05-09T06:37:01.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:01.716+0000 7fab21ffb640 1 --2- 172.21.15.3:0/3321668330 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fab1006be30 0x7fab1006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:01.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:01.716+0000 7fab21ffb640 1 -- 172.21.15.3:0/3321668330 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fab140abe50 con 0x7fab240cc870 2024-05-09T06:37:01.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:01.716+0000 7fab28eea640 1 --2- 172.21.15.3:0/3321668330 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fab1006be30 0x7fab1006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:01.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:01.717+0000 7fab28eea640 1 --2- 172.21.15.3:0/3321668330 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fab1006be30 0x7fab1006e2f0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fab241e63b0 tx=0x7fab18008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:01.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:01.721+0000 7fab21ffb640 1 -- 172.21.15.3:0/3321668330 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fab2413e4a0 con 0x7fab240cc870 2024-05-09T06:37:01.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:01.940+0000 7fab2b94f640 1 -- 172.21.15.3:0/3321668330 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fab240730a0 con 0x7fab240cc870 2024-05-09T06:37:01.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:01.941+0000 7fab21ffb640 1 -- 172.21.15.3:0/3321668330 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5181 (secure 0 0 0) 0x7fab14045360 con 0x7fab240cc870 2024-05-09T06:37:01.922 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-09T06:37:01.922 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4912,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6836/2400645940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":2400645940},{"type":"v1","addr":"172.21.15.190:6836","nonce":2400645940}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4915,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6837/3849729576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6835","nonce":3849729576},{"type":"v1","addr":"172.21.15.190:6837","nonce":3849729576}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4918,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6834/2647782276","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":2647782276},{"type":"v1","addr":"172.21.15.3:6834","nonce":2647782276}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-09T06:36:57.929421+0000","modified":"2024-05-09T06:37:00.941894+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5103},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5103":{"gid":5103,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.3:6835/2138710335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6833","nonce":2138710335},{"type":"v1","addr":"172.21.15.3:6835","nonce":2138710335}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5103,"qdb_cluster":[5103]},"id":2}]} 2024-05-09T06:37:01.922 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 17 2024-05-09T06:37:01.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:01.943+0000 7fab2b94f640 1 -- 172.21.15.3:0/3321668330 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fab1006be30 msgr2=0x7fab1006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:01.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:01.943+0000 7fab2b94f640 1 --2- 172.21.15.3:0/3321668330 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fab1006be30 0x7fab1006e2f0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fab241e63b0 tx=0x7fab18008040 comp rx=0 tx=0).stop 2024-05-09T06:37:01.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:01.943+0000 7fab2b94f640 1 -- 172.21.15.3:0/3321668330 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fab240cc870 msgr2=0x7fab2413d680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:01.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:01.943+0000 7fab2b94f640 1 --2- 172.21.15.3:0/3321668330 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fab240cc870 0x7fab2413d680 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fab14034040 tx=0x7fab14004a40 comp rx=0 tx=0).stop 2024-05-09T06:37:01.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:01.943+0000 7fab2b94f640 1 -- 172.21.15.3:0/3321668330 shutdown_connections 2024-05-09T06:37:01.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:01.943+0000 7fab2b94f640 1 --2- 172.21.15.3:0/3321668330 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fab1006be30 0x7fab1006e2f0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:01.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:01.943+0000 7fab2b94f640 1 --2- 172.21.15.3:0/3321668330 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fab241408f0 0x7fab240b4280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:01.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:01.943+0000 7fab2b94f640 1 --2- 172.21.15.3:0/3321668330 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fab2413e0c0 0x7fab240b3d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:01.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:01.943+0000 7fab2b94f640 1 --2- 172.21.15.3:0/3321668330 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fab240cc870 0x7fab2413d680 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:01.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:01.943+0000 7fab2b94f640 1 -- 172.21.15.3:0/3321668330 >> 172.21.15.3:0/3321668330 conn(0x7fab2406b430 msgr2=0x7fab24156ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:01.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:01.943+0000 7fab2b94f640 1 -- 172.21.15.3:0/3321668330 shutdown_connections 2024-05-09T06:37:01.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:01.943+0000 7fab2b94f640 1 -- 172.21.15.3:0/3321668330 wait complete. 2024-05-09T06:37:01.935 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T06:37:02.039 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.058+0000 7fa5430bd640 1 Processor -- start 2024-05-09T06:37:02.039 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.059+0000 7fa5430bd640 1 -- start start 2024-05-09T06:37:02.041 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.059+0000 7fa5430bd640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fa53c13e0c0 0x7fa53c0cf720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:02.041 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.059+0000 7fa5430bd640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa53c141000 0x7fa53c0ccd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:02.041 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.059+0000 7fa5430bd640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa53c13e4a0 con 0x7fa53c0cc870 2024-05-09T06:37:02.041 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.059+0000 7fa5430bd640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fa53c141410 con 0x7fa53c141000 2024-05-09T06:37:02.041 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.059+0000 7fa5430bd640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7fa53c0cd240 con 0x7fa53c13e0c0 2024-05-09T06:37:02.041 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.060+0000 7fa54165a640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa53c141000 0x7fa53c0ccd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:02.041 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.060+0000 7fa53bfff640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fa53c0cc870 0x7fa53c1408f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:58042/0 (socket says 172.21.15.3:58042) 2024-05-09T06:37:02.041 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.060+0000 7fa53bfff640 1 -- 172.21.15.3:0/1720943665 learned_addr learned my addr 172.21.15.3:0/1720943665 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-05-09T06:37:02.041 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.060+0000 7fa540e59640 1 --2- 172.21.15.3:0/1720943665 >> v2:172.21.15.190:3300/0 conn(0x7fa53c13e0c0 0x7fa53c0cf720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:02.041 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.060+0000 7fa54165a640 1 -- 172.21.15.3:0/1720943665 >> v2:172.21.15.190:3300/0 conn(0x7fa53c13e0c0 msgr2=0x7fa53c0cf720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:37:02.041 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.060+0000 7fa54165a640 1 --2- 172.21.15.3:0/1720943665 >> v2:172.21.15.190:3300/0 conn(0x7fa53c13e0c0 0x7fa53c0cf720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:02.041 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.060+0000 7fa54165a640 1 -- 172.21.15.3:0/1720943665 >> v1:172.21.15.3:6789/0 conn(0x7fa53c0cc870 legacy=0x7fa53c1408f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:02.042 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.060+0000 7fa54165a640 1 -- 172.21.15.3:0/1720943665 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa53c0ce960 con 0x7fa53c141000 2024-05-09T06:37:02.043 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.061+0000 7fa54165a640 1 --2- 172.21.15.3:0/1720943665 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa53c141000 0x7fa53c0ccd00 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fa53c06f010 tx=0x7fa52402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63c284f46044f91a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:02.043 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.061+0000 7fa53b7fe640 1 -- 172.21.15.3:0/1720943665 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa524037070 con 0x7fa53c141000 2024-05-09T06:37:02.043 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.061+0000 7fa53b7fe640 1 -- 172.21.15.3:0/1720943665 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa524032440 con 0x7fa53c141000 2024-05-09T06:37:02.043 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.061+0000 7fa53b7fe640 1 -- 172.21.15.3:0/1720943665 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa524032810 con 0x7fa53c141000 2024-05-09T06:37:02.043 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.061+0000 7fa5430bd640 1 -- 172.21.15.3:0/1720943665 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa53c141000 msgr2=0x7fa53c0ccd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:02.043 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.061+0000 7fa5430bd640 1 --2- 172.21.15.3:0/1720943665 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa53c141000 0x7fa53c0ccd00 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fa53c06f010 tx=0x7fa52402a1a0 comp rx=0 tx=0).stop 2024-05-09T06:37:02.043 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.061+0000 7fa5430bd640 1 -- 172.21.15.3:0/1720943665 shutdown_connections 2024-05-09T06:37:02.043 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.061+0000 7fa5430bd640 1 --2- 172.21.15.3:0/1720943665 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa53c141000 0x7fa53c0ccd00 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:02.043 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.061+0000 7fa5430bd640 1 --2- 172.21.15.3:0/1720943665 >> v2:172.21.15.190:3300/0 conn(0x7fa53c13e0c0 0x7fa53c0cf720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:02.043 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.061+0000 7fa5430bd640 1 -- 172.21.15.3:0/1720943665 >> 172.21.15.3:0/1720943665 conn(0x7fa53c06b430 msgr2=0x7fa53c06b810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:02.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.061+0000 7fa5430bd640 1 -- 172.21.15.3:0/1720943665 shutdown_connections 2024-05-09T06:37:02.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.061+0000 7fa5430bd640 1 -- 172.21.15.3:0/1720943665 wait complete. 2024-05-09T06:37:02.046 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.063+0000 7fa5430bd640 1 Processor -- start 2024-05-09T06:37:02.046 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.063+0000 7fa5430bd640 1 -- start start 2024-05-09T06:37:02.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.064+0000 7fa5430bd640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa53c0cc870 0x7fa53c148290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:02.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.064+0000 7fa5430bd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa53c13e0c0 0x7fa53c0aeda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:02.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.064+0000 7fa5430bd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa53c141000 0x7fa53c1f37c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:02.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.064+0000 7fa5430bd640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fa53c0cf200 con 0x7fa53c141000 2024-05-09T06:37:02.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.064+0000 7fa5430bd640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fa53c0cef20 con 0x7fa53c13e0c0 2024-05-09T06:37:02.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.064+0000 7fa5430bd640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fa53c141660 con 0x7fa53c0cc870 2024-05-09T06:37:02.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.064+0000 7fa54165a640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa53c141000 0x7fa53c1f37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:02.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.064+0000 7fa54165a640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa53c141000 0x7fa53c1f37c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:39400/0 (socket says 172.21.15.3:39400) 2024-05-09T06:37:02.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.064+0000 7fa54165a640 1 -- 172.21.15.3:0/2756475657 learned_addr learned my addr 172.21.15.3:0/2756475657 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:37:02.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.064+0000 7fa540e59640 1 --2- 172.21.15.3:0/2756475657 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa53c0cc870 0x7fa53c148290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:02.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.064+0000 7fa54165a640 1 -- 172.21.15.3:0/2756475657 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa53c0cc870 msgr2=0x7fa53c148290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:02.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.064+0000 7fa54165a640 1 --2- 172.21.15.3:0/2756475657 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa53c0cc870 0x7fa53c148290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:02.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.064+0000 7fa54165a640 1 -- 172.21.15.3:0/2756475657 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa53c13e0c0 msgr2=0x7fa53c0aeda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:02.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.064+0000 7fa53bfff640 1 --2- 172.21.15.3:0/2756475657 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa53c13e0c0 0x7fa53c0aeda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:02.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.064+0000 7fa54165a640 1 --2- 172.21.15.3:0/2756475657 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa53c13e0c0 0x7fa53c0aeda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:02.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.064+0000 7fa54165a640 1 -- 172.21.15.3:0/2756475657 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa53c148870 con 0x7fa53c141000 2024-05-09T06:37:02.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.064+0000 7fa540e59640 1 --2- 172.21.15.3:0/2756475657 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa53c0cc870 0x7fa53c148290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:37:02.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.064+0000 7fa53bfff640 1 --2- 172.21.15.3:0/2756475657 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa53c13e0c0 0x7fa53c0aeda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:37:02.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.064+0000 7fa54165a640 1 --2- 172.21.15.3:0/2756475657 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa53c141000 0x7fa53c1f37c0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fa524009890 tx=0x7fa5240046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:02.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.064+0000 7fa5397fa640 1 -- 172.21.15.3:0/2756475657 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa524037040 con 0x7fa53c141000 2024-05-09T06:37:02.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.065+0000 7fa5397fa640 1 -- 172.21.15.3:0/2756475657 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa524042070 con 0x7fa53c141000 2024-05-09T06:37:02.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.065+0000 7fa5430bd640 1 -- 172.21.15.3:0/2756475657 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa53c148b50 con 0x7fa53c141000 2024-05-09T06:37:02.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.065+0000 7fa5397fa640 1 -- 172.21.15.3:0/2756475657 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa524030b30 con 0x7fa53c141000 2024-05-09T06:37:02.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.065+0000 7fa5430bd640 1 -- 172.21.15.3:0/2756475657 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa53c149110 con 0x7fa53c141000 2024-05-09T06:37:02.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.065+0000 7fa5430bd640 1 -- 172.21.15.3:0/2756475657 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa53c149600 con 0x7fa53c141000 2024-05-09T06:37:02.052 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.066+0000 7fa5397fa640 1 -- 172.21.15.3:0/2756475657 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa524030cd0 con 0x7fa53c141000 2024-05-09T06:37:02.052 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.067+0000 7fa5397fa640 1 --2- 172.21.15.3:0/2756475657 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fa51c06be30 0x7fa51c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:02.052 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.067+0000 7fa5397fa640 1 -- 172.21.15.3:0/2756475657 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fa5240aba90 con 0x7fa53c141000 2024-05-09T06:37:02.052 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.067+0000 7fa540e59640 1 --2- 172.21.15.3:0/2756475657 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fa51c06be30 0x7fa51c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:02.052 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.067+0000 7fa540e59640 1 --2- 172.21.15.3:0/2756475657 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fa51c06be30 0x7fa51c06e2f0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fa534006fd0 tx=0x7fa534008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:02.052 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.071+0000 7fa5397fa640 1 -- 172.21.15.3:0/2756475657 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fa524032e20 con 0x7fa53c141000 2024-05-09T06:37:02.236 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.255+0000 7fa5430bd640 1 -- 172.21.15.3:0/2756475657 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa53c0730a0 con 0x7fa53c141000 2024-05-09T06:37:02.237 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.256+0000 7fa5397fa640 1 -- 172.21.15.3:0/2756475657 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) ==== 74+0+16585 (secure 0 0 0) 0x7fa524045360 con 0x7fa53c141000 2024-05-09T06:37:02.237 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-09T06:37:02.237 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":33,"fsid":"16ed737d-2665-431a-8ac6-ef242be3975a","created":"2024-05-09T06:35:42.541368+0000","modified":"2024-05-09T06:36:59.564366+0000","last_up_change":"2024-05-09T06:35:53.778274+0000","last_in_change":"2024-05-09T06:35:44.780586+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T06:35:54.819999+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-05-09T06:36:56.494700+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-05-09T06:36:56.900006+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fae3a8db-4826-4f3c-b33a-12c09bb5df6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":1897849827},{"type":"v1","addr":"172.21.15.3:6817","nonce":1897849827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":1897849827},{"type":"v1","addr":"172.21.15.3:6819","nonce":1897849827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":1897849827},{"type":"v1","addr":"172.21.15.3:6823","nonce":1897849827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":1897849827},{"type":"v1","addr":"172.21.15.3:6821","nonce":1897849827}]},"public_addr":"172.21.15.3:6817/1897849827","cluster_addr":"172.21.15.3:6819/1897849827","heartbeat_back_addr":"172.21.15.3:6823/1897849827","heartbeat_front_addr":"172.21.15.3:6821/1897849827","state":["exists","up"]},{"osd":1,"uuid":"646e882d-4ce4-4720-8c84-f71df563a35e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":926805498},{"type":"v1","addr":"172.21.15.3:6801","nonce":926805498}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":926805498},{"type":"v1","addr":"172.21.15.3:6803","nonce":926805498}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":926805498},{"type":"v1","addr":"172.21.15.3:6807","nonce":926805498}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":926805498},{"type":"v1","addr":"172.21.15.3:6805","nonce":926805498}]},"public_addr":"172.21.15.3:6801/926805498","cluster_addr":"172.21.15.3:6803/926805498","heartbeat_back_addr":"172.21.15.3:6807/926805498","heartbeat_front_addr":"172.21.15.3:6805/926805498","state":["exists","up"]},{"osd":2,"uuid":"1abbadb5-a570-4359-9643-acff3ede35a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":801295451},{"type":"v1","addr":"172.21.15.3:6809","nonce":801295451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":801295451},{"type":"v1","addr":"172.21.15.3:6811","nonce":801295451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":801295451},{"type":"v1","addr":"172.21.15.3:6815","nonce":801295451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":801295451},{"type":"v1","addr":"172.21.15.3:6813","nonce":801295451}]},"public_addr":"172.21.15.3:6809/801295451","cluster_addr":"172.21.15.3:6811/801295451","heartbeat_back_addr":"172.21.15.3:6815/801295451","heartbeat_front_addr":"172.21.15.3:6813/801295451","state":["exists","up"]},{"osd":3,"uuid":"62ac4dc1-2371-4dd2-acd3-e2dbb11aedf5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":1903389857},{"type":"v1","addr":"172.21.15.3:6825","nonce":1903389857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":1903389857},{"type":"v1","addr":"172.21.15.3:6827","nonce":1903389857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":1903389857},{"type":"v1","addr":"172.21.15.3:6831","nonce":1903389857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":1903389857},{"type":"v1","addr":"172.21.15.3:6829","nonce":1903389857}]},"public_addr":"172.21.15.3:6825/1903389857","cluster_addr":"172.21.15.3:6827/1903389857","heartbeat_back_addr":"172.21.15.3:6831/1903389857","heartbeat_front_addr":"172.21.15.3:6829/1903389857","state":["exists","up"]},{"osd":4,"uuid":"ae808398-8326-4d7c-9319-a2a0eafffb59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":2143624576},{"type":"v1","addr":"172.21.15.190:6801","nonce":2143624576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":2143624576},{"type":"v1","addr":"172.21.15.190:6803","nonce":2143624576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":2143624576},{"type":"v1","addr":"172.21.15.190:6807","nonce":2143624576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":2143624576},{"type":"v1","addr":"172.21.15.190:6805","nonce":2143624576}]},"public_addr":"172.21.15.190:6801/2143624576","cluster_addr":"172.21.15.190:6803/2143624576","heartbeat_back_addr":"172.21.15.190:6807/2143624576","heartbeat_front_addr":"172.21.15.190:6805/2143624576","state":["exists","up"]},{"osd":5,"uuid":"918adf50-c440-48ac-9a8d-4837562d7275","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":1766923126},{"type":"v1","addr":"172.21.15.190:6814","nonce":1766923126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":1766923126},{"type":"v1","addr":"172.21.15.190:6818","nonce":1766923126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":1766923126},{"type":"v1","addr":"172.21.15.190:6823","nonce":1766923126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":1766923126},{"type":"v1","addr":"172.21.15.190:6821","nonce":1766923126}]},"public_addr":"172.21.15.190:6814/1766923126","cluster_addr":"172.21.15.190:6818/1766923126","heartbeat_back_addr":"172.21.15.190:6823/1766923126","heartbeat_front_addr":"172.21.15.190:6821/1766923126","state":["exists","up"]},{"osd":6,"uuid":"ac2882bc-2a7c-463c-8f6c-76ff49828adc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":342414248},{"type":"v1","addr":"172.21.15.190:6825","nonce":342414248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":342414248},{"type":"v1","addr":"172.21.15.190:6827","nonce":342414248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":342414248},{"type":"v1","addr":"172.21.15.190:6831","nonce":342414248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":342414248},{"type":"v1","addr":"172.21.15.190:6829","nonce":342414248}]},"public_addr":"172.21.15.190:6825/342414248","cluster_addr":"172.21.15.190:6827/342414248","heartbeat_back_addr":"172.21.15.190:6831/342414248","heartbeat_front_addr":"172.21.15.190:6829/342414248","state":["exists","up"]},{"osd":7,"uuid":"dd4e767a-4730-4c70-8bd5-3f088140a5a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":537849891},{"type":"v1","addr":"172.21.15.190:6809","nonce":537849891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":537849891},{"type":"v1","addr":"172.21.15.190:6811","nonce":537849891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6817","nonce":537849891},{"type":"v1","addr":"172.21.15.190:6819","nonce":537849891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6813","nonce":537849891},{"type":"v1","addr":"172.21.15.190:6815","nonce":537849891}]},"public_addr":"172.21.15.190:6809/537849891","cluster_addr":"172.21.15.190:6811/537849891","heartbeat_back_addr":"172.21.15.190:6819/537849891","heartbeat_front_addr":"172.21.15.190:6815/537849891","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.889784+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.974002+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.973549+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.955396+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.928471+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.902418+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.958696+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.926474+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T06:37:02.239 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.258+0000 7fa5430bd640 1 -- 172.21.15.3:0/2756475657 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fa51c06be30 msgr2=0x7fa51c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:02.239 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.258+0000 7fa5430bd640 1 --2- 172.21.15.3:0/2756475657 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fa51c06be30 0x7fa51c06e2f0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fa534006fd0 tx=0x7fa534008040 comp rx=0 tx=0).stop 2024-05-09T06:37:02.239 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.258+0000 7fa5430bd640 1 -- 172.21.15.3:0/2756475657 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa53c141000 msgr2=0x7fa53c1f37c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:02.239 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.258+0000 7fa5430bd640 1 --2- 172.21.15.3:0/2756475657 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa53c141000 0x7fa53c1f37c0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fa524009890 tx=0x7fa5240046c0 comp rx=0 tx=0).stop 2024-05-09T06:37:02.239 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.258+0000 7fa5430bd640 1 -- 172.21.15.3:0/2756475657 shutdown_connections 2024-05-09T06:37:02.239 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.258+0000 7fa5430bd640 1 --2- 172.21.15.3:0/2756475657 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fa51c06be30 0x7fa51c06e2f0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:02.239 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.258+0000 7fa5430bd640 1 --2- 172.21.15.3:0/2756475657 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa53c141000 0x7fa53c1f37c0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:02.239 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.258+0000 7fa5430bd640 1 --2- 172.21.15.3:0/2756475657 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa53c13e0c0 0x7fa53c0aeda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:02.239 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.258+0000 7fa5430bd640 1 --2- 172.21.15.3:0/2756475657 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa53c0cc870 0x7fa53c148290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:02.239 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.258+0000 7fa5430bd640 1 -- 172.21.15.3:0/2756475657 >> 172.21.15.3:0/2756475657 conn(0x7fa53c06b430 msgr2=0x7fa53c15d7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:02.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.258+0000 7fa5430bd640 1 -- 172.21.15.3:0/2756475657 shutdown_connections 2024-05-09T06:37:02.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.258+0000 7fa5430bd640 1 -- 172.21.15.3:0/2756475657 wait complete. 2024-05-09T06:37:02.250 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-05-09T06:37:02.251 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-09T06:37:02.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.373+0000 7f31c3d61640 1 Processor -- start 2024-05-09T06:37:02.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.374+0000 7f31c3d61640 1 -- start start 2024-05-09T06:37:02.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.374+0000 7f31c3d61640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f31bc15e480 0x7f31bc0d1dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:02.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.374+0000 7f31c3d61640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f31bc0d2310 0x7f31bc0d27c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:02.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.374+0000 7f31c3d61640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f31bc0d2d00 0x7f31bc162070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:02.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.374+0000 7f31c3d61640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f31bc0d4e50 con 0x7f31bc0d2d00 2024-05-09T06:37:02.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.374+0000 7f31c3d61640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f31bc0d0270 con 0x7f31bc15e480 2024-05-09T06:37:02.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.374+0000 7f31c3d61640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f31bc0cf1d0 con 0x7f31bc0d2310 2024-05-09T06:37:02.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.375+0000 7f31c22fe640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f31bc0d2d00 0x7f31bc162070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:02.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.375+0000 7f31c22fe640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f31bc0d2d00 0x7f31bc162070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:39402/0 (socket says 172.21.15.3:39402) 2024-05-09T06:37:02.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.375+0000 7f31c22fe640 1 -- 172.21.15.3:0/3610246568 learned_addr learned my addr 172.21.15.3:0/3610246568 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:37:02.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.375+0000 7f31c1afd640 1 --2- 172.21.15.3:0/3610246568 >> v2:172.21.15.190:3300/0 conn(0x7f31bc15e480 0x7f31bc0d1dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:02.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.375+0000 7f31c12fc640 1 --2- 172.21.15.3:0/3610246568 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f31bc0d2310 0x7f31bc0d27c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:02.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.375+0000 7f31c22fe640 1 -- 172.21.15.3:0/3610246568 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f31bc0d2310 msgr2=0x7f31bc0d27c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:37:02.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.375+0000 7f31c22fe640 1 --2- 172.21.15.3:0/3610246568 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f31bc0d2310 0x7f31bc0d27c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:02.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.375+0000 7f31c22fe640 1 -- 172.21.15.3:0/3610246568 >> v2:172.21.15.190:3300/0 conn(0x7f31bc15e480 msgr2=0x7f31bc0d1dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:37:02.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.375+0000 7f31c22fe640 1 --2- 172.21.15.3:0/3610246568 >> v2:172.21.15.190:3300/0 conn(0x7f31bc15e480 0x7f31bc0d1dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:02.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.375+0000 7f31c22fe640 1 -- 172.21.15.3:0/3610246568 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31bc0cf4b0 con 0x7f31bc0d2d00 2024-05-09T06:37:02.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.376+0000 7f31c22fe640 1 --2- 172.21.15.3:0/3610246568 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f31bc0d2d00 0x7f31bc162070 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f31bc09da00 tx=0x7f31b402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=273c68f0c12a1290 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:02.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.376+0000 7f31c0afb640 1 -- 172.21.15.3:0/3610246568 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31b4036070 con 0x7f31bc0d2d00 2024-05-09T06:37:02.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.376+0000 7f31c0afb640 1 -- 172.21.15.3:0/3610246568 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f31b4031600 con 0x7f31bc0d2d00 2024-05-09T06:37:02.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.376+0000 7f31c0afb640 1 -- 172.21.15.3:0/3610246568 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31b40319d0 con 0x7f31bc0d2d00 2024-05-09T06:37:02.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.376+0000 7f31c3d61640 1 -- 172.21.15.3:0/3610246568 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f31bc0d2d00 msgr2=0x7f31bc162070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:02.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.376+0000 7f31c3d61640 1 --2- 172.21.15.3:0/3610246568 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f31bc0d2d00 0x7f31bc162070 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f31bc09da00 tx=0x7f31b402a6a0 comp rx=0 tx=0).stop 2024-05-09T06:37:02.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.376+0000 7f31c3d61640 1 -- 172.21.15.3:0/3610246568 shutdown_connections 2024-05-09T06:37:02.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.376+0000 7f31c3d61640 1 --2- 172.21.15.3:0/3610246568 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f31bc0d2d00 0x7f31bc162070 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:02.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.376+0000 7f31c3d61640 1 --2- 172.21.15.3:0/3610246568 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f31bc0d2310 0x7f31bc0d27c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:02.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.376+0000 7f31c3d61640 1 --2- 172.21.15.3:0/3610246568 >> v2:172.21.15.190:3300/0 conn(0x7f31bc15e480 0x7f31bc0d1dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:02.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.376+0000 7f31c3d61640 1 -- 172.21.15.3:0/3610246568 >> 172.21.15.3:0/3610246568 conn(0x7f31bc09d4d0 msgr2=0x7f31bc0d4200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:02.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.376+0000 7f31c3d61640 1 -- 172.21.15.3:0/3610246568 shutdown_connections 2024-05-09T06:37:02.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.376+0000 7f31c3d61640 1 -- 172.21.15.3:0/3610246568 wait complete. 2024-05-09T06:37:02.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.378+0000 7f31c3d61640 1 Processor -- start 2024-05-09T06:37:02.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.378+0000 7f31c3d61640 1 -- start start 2024-05-09T06:37:02.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.378+0000 7f31c3d61640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f31bc0d2310 0x7f31bc0b7800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:02.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.378+0000 7f31c3d61640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f31bc0d2d00 0x7f31bc1f5190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:02.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.378+0000 7f31c3d61640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f31bc15e480 0x7f31bc1585c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:02.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.378+0000 7f31c3d61640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f31bc0cfd50 con 0x7f31bc0d2d00 2024-05-09T06:37:02.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.378+0000 7f31c3d61640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f31bc0cfa70 con 0x7f31bc0d2310 2024-05-09T06:37:02.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.378+0000 7f31c3d61640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f31bc0b5ab0 con 0x7f31bc15e480 2024-05-09T06:37:02.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.378+0000 7f31c12fc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f31bc0d2d00 0x7f31bc1f5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:02.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.378+0000 7f31c12fc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f31bc0d2d00 0x7f31bc1f5190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:39408/0 (socket says 172.21.15.3:39408) 2024-05-09T06:37:02.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.378+0000 7f31c12fc640 1 -- 172.21.15.3:0/2469367982 learned_addr learned my addr 172.21.15.3:0/2469367982 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:37:02.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.379+0000 7f31c22fe640 1 --2- 172.21.15.3:0/2469367982 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f31bc15e480 0x7f31bc1585c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:02.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.379+0000 7f31c1afd640 1 --2- 172.21.15.3:0/2469367982 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f31bc0d2310 0x7f31bc0b7800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:02.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.379+0000 7f31c12fc640 1 -- 172.21.15.3:0/2469367982 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f31bc15e480 msgr2=0x7f31bc1585c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:02.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.379+0000 7f31c12fc640 1 --2- 172.21.15.3:0/2469367982 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f31bc15e480 0x7f31bc1585c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:02.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.379+0000 7f31c12fc640 1 -- 172.21.15.3:0/2469367982 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f31bc0d2310 msgr2=0x7f31bc0b7800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:02.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.379+0000 7f31c12fc640 1 --2- 172.21.15.3:0/2469367982 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f31bc0d2310 0x7f31bc0b7800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:02.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.379+0000 7f31c12fc640 1 -- 172.21.15.3:0/2469367982 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31bc158fc0 con 0x7f31bc0d2d00 2024-05-09T06:37:02.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.379+0000 7f31c12fc640 1 --2- 172.21.15.3:0/2469367982 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f31bc0d2d00 0x7f31bc1f5190 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f31ac00c570 tx=0x7f31ac00ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:02.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.379+0000 7f31b27fc640 1 -- 172.21.15.3:0/2469367982 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31ac015070 con 0x7f31bc0d2d00 2024-05-09T06:37:02.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.379+0000 7f31b27fc640 1 -- 172.21.15.3:0/2469367982 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f31ac004220 con 0x7f31bc0d2d00 2024-05-09T06:37:02.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.379+0000 7f31b27fc640 1 -- 172.21.15.3:0/2469367982 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31ac003610 con 0x7f31bc0d2d00 2024-05-09T06:37:02.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.379+0000 7f31c3d61640 1 -- 172.21.15.3:0/2469367982 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31bc1592a0 con 0x7f31bc0d2d00 2024-05-09T06:37:02.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.379+0000 7f31c3d61640 1 -- 172.21.15.3:0/2469367982 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31bc159860 con 0x7f31bc0d2d00 2024-05-09T06:37:02.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.380+0000 7f31c3d61640 1 -- 172.21.15.3:0/2469367982 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31bc159ca0 con 0x7f31bc0d2d00 2024-05-09T06:37:02.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.381+0000 7f31b27fc640 1 -- 172.21.15.3:0/2469367982 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f31ac0037b0 con 0x7f31bc0d2d00 2024-05-09T06:37:02.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.381+0000 7f31b27fc640 1 --2- 172.21.15.3:0/2469367982 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f31a006be60 0x7f31a006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:02.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.381+0000 7f31b27fc640 1 -- 172.21.15.3:0/2469367982 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f31ac08aa40 con 0x7f31bc0d2d00 2024-05-09T06:37:02.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.381+0000 7f31c1afd640 1 --2- 172.21.15.3:0/2469367982 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f31a006be60 0x7f31a006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:02.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.382+0000 7f31c1afd640 1 --2- 172.21.15.3:0/2469367982 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f31a006be60 0x7f31a006e320 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f31bc0b8690 tx=0x7f31b8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:02.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.385+0000 7f31b27fc640 1 -- 172.21.15.3:0/2469367982 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f31ac058e30 con 0x7f31bc0d2d00 2024-05-09T06:37:02.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.565+0000 7f31c3d61640 1 -- 172.21.15.3:0/2469367982 --> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f31bc0d18f0 con 0x7f31a006be60 2024-05-09T06:37:02.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.572+0000 7f31b27fc640 1 -- 172.21.15.3:0/2469367982 <== mgr.4105 v2:172.21.15.190:6832/975893321 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251880 (secure 0 0 0) 0x7f31bc0d18f0 con 0x7f31a006be60 2024-05-09T06:37:02.555 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-09T06:37:02.557 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2024-05-09T06:37:02.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.574+0000 7f31c3d61640 1 -- 172.21.15.3:0/2469367982 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f31a006be60 msgr2=0x7f31a006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:02.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.574+0000 7f31c3d61640 1 --2- 172.21.15.3:0/2469367982 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f31a006be60 0x7f31a006e320 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f31bc0b8690 tx=0x7f31b8008040 comp rx=0 tx=0).stop 2024-05-09T06:37:02.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.574+0000 7f31c3d61640 1 -- 172.21.15.3:0/2469367982 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f31bc0d2d00 msgr2=0x7f31bc1f5190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:02.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.574+0000 7f31c3d61640 1 --2- 172.21.15.3:0/2469367982 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f31bc0d2d00 0x7f31bc1f5190 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f31ac00c570 tx=0x7f31ac00ca30 comp rx=0 tx=0).stop 2024-05-09T06:37:02.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.575+0000 7f31c3d61640 1 -- 172.21.15.3:0/2469367982 shutdown_connections 2024-05-09T06:37:02.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.575+0000 7f31c3d61640 1 --2- 172.21.15.3:0/2469367982 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f31a006be60 0x7f31a006e320 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:02.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.575+0000 7f31c3d61640 1 --2- 172.21.15.3:0/2469367982 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f31bc15e480 0x7f31bc1585c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:02.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.575+0000 7f31c3d61640 1 --2- 172.21.15.3:0/2469367982 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f31bc0d2d00 0x7f31bc1f5190 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:02.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.575+0000 7f31c3d61640 1 --2- 172.21.15.3:0/2469367982 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f31bc0d2310 0x7f31bc0b7800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:02.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.575+0000 7f31c3d61640 1 -- 172.21.15.3:0/2469367982 >> 172.21.15.3:0/2469367982 conn(0x7f31bc09d4d0 msgr2=0x7f31bc0b64a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:02.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.575+0000 7f31c3d61640 1 -- 172.21.15.3:0/2469367982 shutdown_connections 2024-05-09T06:37:02.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.575+0000 7f31c3d61640 1 -- 172.21.15.3:0/2469367982 wait complete. 2024-05-09T06:37:02.567 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":60,"stamp":"2024-05-09T06:37:00.788264+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":45,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220116,"kb_used_data":3932,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754604,"statfs":{"total":773094113280,"available":772868714496,"internally_reserved":0,"allocated":4026368,"data_stored":2270109,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":28,"apply_latency_ms":28,"commit_latency_ns":28000000,"apply_latency_ns":28000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.240821"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T06:36:58.581052+0000","last_change":"2024-05-09T06:36:58.577302+0000","last_active":"2024-05-09T06:36:58.581052+0000","last_peered":"2024-05-09T06:36:58.581052+0000","last_clean":"2024-05-09T06:36:58.581052+0000","last_became_active":"2024-05-09T06:36:58.576999+0000","last_became_peered":"2024-05-09T06:36:58.576999+0000","last_unstale":"2024-05-09T06:36:58.581052+0000","last_undegraded":"2024-05-09T06:36:58.581052+0000","last_fullsized":"2024-05-09T06:36:58.581052+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_clean_scrub_stamp":"2024-05-09T06:36:57.547101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:47:37.035146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T06:36:58.586359+0000","last_change":"2024-05-09T06:36:58.573971+0000","last_active":"2024-05-09T06:36:58.586359+0000","last_peered":"2024-05-09T06:36:58.586359+0000","last_clean":"2024-05-09T06:36:58.586359+0000","last_became_active":"2024-05-09T06:36:58.573678+0000","last_became_peered":"2024-05-09T06:36:58.573678+0000","last_unstale":"2024-05-09T06:36:58.586359+0000","last_undegraded":"2024-05-09T06:36:58.586359+0000","last_fullsized":"2024-05-09T06:36:58.586359+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_clean_scrub_stamp":"2024-05-09T06:36:57.547101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:25:07.739454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T06:36:58.590807+0000","last_change":"2024-05-09T06:36:58.586251+0000","last_active":"2024-05-09T06:36:58.590807+0000","last_peered":"2024-05-09T06:36:58.590807+0000","last_clean":"2024-05-09T06:36:58.590807+0000","last_became_active":"2024-05-09T06:36:58.585982+0000","last_became_peered":"2024-05-09T06:36:58.585982+0000","last_unstale":"2024-05-09T06:36:58.590807+0000","last_undegraded":"2024-05-09T06:36:58.590807+0000","last_fullsized":"2024-05-09T06:36:58.590807+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_clean_scrub_stamp":"2024-05-09T06:36:57.547101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:16:33.467895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T06:36:58.585246+0000","last_change":"2024-05-09T06:36:58.579810+0000","last_active":"2024-05-09T06:36:58.585246+0000","last_peered":"2024-05-09T06:36:58.585246+0000","last_clean":"2024-05-09T06:36:58.585246+0000","last_became_active":"2024-05-09T06:36:58.579538+0000","last_became_peered":"2024-05-09T06:36:58.579538+0000","last_unstale":"2024-05-09T06:36:58.585246+0000","last_undegraded":"2024-05-09T06:36:58.585246+0000","last_fullsized":"2024-05-09T06:36:58.585246+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_clean_scrub_stamp":"2024-05-09T06:36:57.547101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:47:37.035146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T06:36:58.588201+0000","last_change":"2024-05-09T06:36:58.584786+0000","last_active":"2024-05-09T06:36:58.588201+0000","last_peered":"2024-05-09T06:36:58.588201+0000","last_clean":"2024-05-09T06:36:58.588201+0000","last_became_active":"2024-05-09T06:36:58.584545+0000","last_became_peered":"2024-05-09T06:36:58.584545+0000","last_unstale":"2024-05-09T06:36:58.588201+0000","last_undegraded":"2024-05-09T06:36:58.588201+0000","last_fullsized":"2024-05-09T06:36:58.588201+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_clean_scrub_stamp":"2024-05-09T06:36:57.547101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:45:01.358608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T06:36:58.586439+0000","last_change":"2024-05-09T06:36:58.577494+0000","last_active":"2024-05-09T06:36:58.586439+0000","last_peered":"2024-05-09T06:36:58.586439+0000","last_clean":"2024-05-09T06:36:58.586439+0000","last_became_active":"2024-05-09T06:36:58.577198+0000","last_became_peered":"2024-05-09T06:36:58.577198+0000","last_unstale":"2024-05-09T06:36:58.586439+0000","last_undegraded":"2024-05-09T06:36:58.586439+0000","last_fullsized":"2024-05-09T06:36:58.586439+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_clean_scrub_stamp":"2024-05-09T06:36:57.547101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:16:33.467895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T06:36:58.580892+0000","last_change":"2024-05-09T06:36:58.575409+0000","last_active":"2024-05-09T06:36:58.580892+0000","last_peered":"2024-05-09T06:36:58.580892+0000","last_clean":"2024-05-09T06:36:58.580892+0000","last_became_active":"2024-05-09T06:36:58.575049+0000","last_became_peered":"2024-05-09T06:36:58.575049+0000","last_unstale":"2024-05-09T06:36:58.580892+0000","last_undegraded":"2024-05-09T06:36:58.580892+0000","last_fullsized":"2024-05-09T06:36:58.580892+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_clean_scrub_stamp":"2024-05-09T06:36:57.547101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:17:59.484786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T06:36:58.580846+0000","last_change":"2024-05-09T06:36:58.576987+0000","last_active":"2024-05-09T06:36:58.580846+0000","last_peered":"2024-05-09T06:36:58.580846+0000","last_clean":"2024-05-09T06:36:58.580846+0000","last_became_active":"2024-05-09T06:36:58.576693+0000","last_became_peered":"2024-05-09T06:36:58.576693+0000","last_unstale":"2024-05-09T06:36:58.580846+0000","last_undegraded":"2024-05-09T06:36:58.580846+0000","last_fullsized":"2024-05-09T06:36:58.580846+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_clean_scrub_stamp":"2024-05-09T06:36:57.547101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:39:22.810622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T06:36:58.585592+0000","last_change":"2024-05-09T06:36:58.581421+0000","last_active":"2024-05-09T06:36:58.585592+0000","last_peered":"2024-05-09T06:36:58.585592+0000","last_clean":"2024-05-09T06:36:58.585592+0000","last_became_active":"2024-05-09T06:36:58.581028+0000","last_became_peered":"2024-05-09T06:36:58.581028+0000","last_unstale":"2024-05-09T06:36:58.585592+0000","last_undegraded":"2024-05-09T06:36:58.585592+0000","last_fullsized":"2024-05-09T06:36:58.585592+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_clean_scrub_stamp":"2024-05-09T06:36:57.547101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:03:45.503132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T06:36:58.593000+0000","last_change":"2024-05-09T06:36:58.589883+0000","last_active":"2024-05-09T06:36:58.593000+0000","last_peered":"2024-05-09T06:36:58.593000+0000","last_clean":"2024-05-09T06:36:58.593000+0000","last_became_active":"2024-05-09T06:36:58.589544+0000","last_became_peered":"2024-05-09T06:36:58.589544+0000","last_unstale":"2024-05-09T06:36:58.593000+0000","last_undegraded":"2024-05-09T06:36:58.593000+0000","last_fullsized":"2024-05-09T06:36:58.593000+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_clean_scrub_stamp":"2024-05-09T06:36:57.547101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:40:37.529479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T06:36:58.587999+0000","last_change":"2024-05-09T06:36:58.585279+0000","last_active":"2024-05-09T06:36:58.587999+0000","last_peered":"2024-05-09T06:36:58.587999+0000","last_clean":"2024-05-09T06:36:58.587999+0000","last_became_active":"2024-05-09T06:36:58.585002+0000","last_became_peered":"2024-05-09T06:36:58.585002+0000","last_unstale":"2024-05-09T06:36:58.587999+0000","last_undegraded":"2024-05-09T06:36:58.587999+0000","last_fullsized":"2024-05-09T06:36:58.587999+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_clean_scrub_stamp":"2024-05-09T06:36:57.547101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:35:21.215213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T06:36:58.577813+0000","last_change":"2024-05-09T06:36:58.573737+0000","last_active":"2024-05-09T06:36:58.577813+0000","last_peered":"2024-05-09T06:36:58.577813+0000","last_clean":"2024-05-09T06:36:58.577813+0000","last_became_active":"2024-05-09T06:36:58.573447+0000","last_became_peered":"2024-05-09T06:36:58.573447+0000","last_unstale":"2024-05-09T06:36:58.577813+0000","last_undegraded":"2024-05-09T06:36:58.577813+0000","last_fullsized":"2024-05-09T06:36:58.577813+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_clean_scrub_stamp":"2024-05-09T06:36:57.547101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:39:22.810622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T06:36:58.587570+0000","last_change":"2024-05-09T06:36:58.582303+0000","last_active":"2024-05-09T06:36:58.587570+0000","last_peered":"2024-05-09T06:36:58.587570+0000","last_clean":"2024-05-09T06:36:58.587570+0000","last_became_active":"2024-05-09T06:36:58.581882+0000","last_became_peered":"2024-05-09T06:36:58.581882+0000","last_unstale":"2024-05-09T06:36:58.587570+0000","last_undegraded":"2024-05-09T06:36:58.587570+0000","last_fullsized":"2024-05-09T06:36:58.587570+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_clean_scrub_stamp":"2024-05-09T06:36:57.547101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:08:22.332489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T06:36:58.574682+0000","last_change":"2024-05-09T06:36:58.566230+0000","last_active":"2024-05-09T06:36:58.574682+0000","last_peered":"2024-05-09T06:36:58.574682+0000","last_clean":"2024-05-09T06:36:58.574682+0000","last_became_active":"2024-05-09T06:36:58.565926+0000","last_became_peered":"2024-05-09T06:36:58.565926+0000","last_unstale":"2024-05-09T06:36:58.574682+0000","last_undegraded":"2024-05-09T06:36:58.574682+0000","last_fullsized":"2024-05-09T06:36:58.574682+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_clean_scrub_stamp":"2024-05-09T06:36:57.547101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:35:21.215213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T06:36:58.580038+0000","last_change":"2024-05-09T06:36:58.575650+0000","last_active":"2024-05-09T06:36:58.580038+0000","last_peered":"2024-05-09T06:36:58.580038+0000","last_clean":"2024-05-09T06:36:58.580038+0000","last_became_active":"2024-05-09T06:36:58.574909+0000","last_became_peered":"2024-05-09T06:36:58.574909+0000","last_unstale":"2024-05-09T06:36:58.580038+0000","last_undegraded":"2024-05-09T06:36:58.580038+0000","last_fullsized":"2024-05-09T06:36:58.580038+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_clean_scrub_stamp":"2024-05-09T06:36:57.547101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:50:46.567561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T06:36:58.593322+0000","last_change":"2024-05-09T06:36:58.588645+0000","last_active":"2024-05-09T06:36:58.593322+0000","last_peered":"2024-05-09T06:36:58.593322+0000","last_clean":"2024-05-09T06:36:58.593322+0000","last_became_active":"2024-05-09T06:36:58.588385+0000","last_became_peered":"2024-05-09T06:36:58.588385+0000","last_unstale":"2024-05-09T06:36:58.593322+0000","last_undegraded":"2024-05-09T06:36:58.593322+0000","last_fullsized":"2024-05-09T06:36:58.593322+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_clean_scrub_stamp":"2024-05-09T06:36:57.547101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:40:37.529479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T06:36:58.580453+0000","last_change":"2024-05-09T06:36:58.576747+0000","last_active":"2024-05-09T06:36:58.580453+0000","last_peered":"2024-05-09T06:36:58.580453+0000","last_clean":"2024-05-09T06:36:58.580453+0000","last_became_active":"2024-05-09T06:36:58.576462+0000","last_became_peered":"2024-05-09T06:36:58.576462+0000","last_unstale":"2024-05-09T06:36:58.580453+0000","last_undegraded":"2024-05-09T06:36:58.580453+0000","last_fullsized":"2024-05-09T06:36:58.580453+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_clean_scrub_stamp":"2024-05-09T06:36:57.547101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:02:32.394179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T06:36:58.587852+0000","last_change":"2024-05-09T06:36:58.581757+0000","last_active":"2024-05-09T06:36:58.587852+0000","last_peered":"2024-05-09T06:36:58.587852+0000","last_clean":"2024-05-09T06:36:58.587852+0000","last_became_active":"2024-05-09T06:36:58.581443+0000","last_became_peered":"2024-05-09T06:36:58.581443+0000","last_unstale":"2024-05-09T06:36:58.587852+0000","last_undegraded":"2024-05-09T06:36:58.587852+0000","last_fullsized":"2024-05-09T06:36:58.587852+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_clean_scrub_stamp":"2024-05-09T06:36:57.547101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:49:32.371985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T06:36:58.588384+0000","last_change":"2024-05-09T06:36:58.586844+0000","last_active":"2024-05-09T06:36:58.588384+0000","last_peered":"2024-05-09T06:36:58.588384+0000","last_clean":"2024-05-09T06:36:58.588384+0000","last_became_active":"2024-05-09T06:36:58.586560+0000","last_became_peered":"2024-05-09T06:36:58.586560+0000","last_unstale":"2024-05-09T06:36:58.588384+0000","last_undegraded":"2024-05-09T06:36:58.588384+0000","last_fullsized":"2024-05-09T06:36:58.588384+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_clean_scrub_stamp":"2024-05-09T06:36:57.547101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:52:57.675729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T06:36:58.584079+0000","last_change":"2024-05-09T06:36:58.580532+0000","last_active":"2024-05-09T06:36:58.584079+0000","last_peered":"2024-05-09T06:36:58.584079+0000","last_clean":"2024-05-09T06:36:58.584079+0000","last_became_active":"2024-05-09T06:36:58.580241+0000","last_became_peered":"2024-05-09T06:36:58.580241+0000","last_unstale":"2024-05-09T06:36:58.584079+0000","last_undegraded":"2024-05-09T06:36:58.584079+0000","last_fullsized":"2024-05-09T06:36:58.584079+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_clean_scrub_stamp":"2024-05-09T06:36:57.547101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:18:06.560834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T06:36:58.581363+0000","last_change":"2024-05-09T06:36:58.576651+0000","last_active":"2024-05-09T06:36:58.581363+0000","last_peered":"2024-05-09T06:36:58.581363+0000","last_clean":"2024-05-09T06:36:58.581363+0000","last_became_active":"2024-05-09T06:36:58.576273+0000","last_became_peered":"2024-05-09T06:36:58.576273+0000","last_unstale":"2024-05-09T06:36:58.581363+0000","last_undegraded":"2024-05-09T06:36:58.581363+0000","last_fullsized":"2024-05-09T06:36:58.581363+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_clean_scrub_stamp":"2024-05-09T06:36:57.547101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:59:53.598004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T06:36:58.567567+0000","last_change":"2024-05-09T06:36:58.564892+0000","last_active":"2024-05-09T06:36:58.567567+0000","last_peered":"2024-05-09T06:36:58.567567+0000","last_clean":"2024-05-09T06:36:58.567567+0000","last_became_active":"2024-05-09T06:36:58.564595+0000","last_became_peered":"2024-05-09T06:36:58.564595+0000","last_unstale":"2024-05-09T06:36:58.567567+0000","last_undegraded":"2024-05-09T06:36:58.567567+0000","last_fullsized":"2024-05-09T06:36:58.567567+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_clean_scrub_stamp":"2024-05-09T06:36:57.547101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:49:21.146962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T06:36:58.580235+0000","last_change":"2024-05-09T06:36:58.575688+0000","last_active":"2024-05-09T06:36:58.580235+0000","last_peered":"2024-05-09T06:36:58.580235+0000","last_clean":"2024-05-09T06:36:58.580235+0000","last_became_active":"2024-05-09T06:36:58.575397+0000","last_became_peered":"2024-05-09T06:36:58.575397+0000","last_unstale":"2024-05-09T06:36:58.580235+0000","last_undegraded":"2024-05-09T06:36:58.580235+0000","last_fullsized":"2024-05-09T06:36:58.580235+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_clean_scrub_stamp":"2024-05-09T06:36:57.547101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:02:32.394179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T06:36:58.577902+0000","last_change":"2024-05-09T06:36:58.574804+0000","last_active":"2024-05-09T06:36:58.577902+0000","last_peered":"2024-05-09T06:36:58.577902+0000","last_clean":"2024-05-09T06:36:58.577902+0000","last_became_active":"2024-05-09T06:36:58.574535+0000","last_became_peered":"2024-05-09T06:36:58.574535+0000","last_unstale":"2024-05-09T06:36:58.577902+0000","last_undegraded":"2024-05-09T06:36:58.577902+0000","last_fullsized":"2024-05-09T06:36:58.577902+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_clean_scrub_stamp":"2024-05-09T06:36:57.547101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:14:46.331201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T06:36:58.585486+0000","last_change":"2024-05-09T06:36:58.579820+0000","last_active":"2024-05-09T06:36:58.585486+0000","last_peered":"2024-05-09T06:36:58.585486+0000","last_clean":"2024-05-09T06:36:58.585486+0000","last_became_active":"2024-05-09T06:36:58.579582+0000","last_became_peered":"2024-05-09T06:36:58.579582+0000","last_unstale":"2024-05-09T06:36:58.585486+0000","last_undegraded":"2024-05-09T06:36:58.585486+0000","last_fullsized":"2024-05-09T06:36:58.585486+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_clean_scrub_stamp":"2024-05-09T06:36:57.547101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:02:32.394179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T06:36:58.587931+0000","last_change":"2024-05-09T06:36:58.582893+0000","last_active":"2024-05-09T06:36:58.587931+0000","last_peered":"2024-05-09T06:36:58.587931+0000","last_clean":"2024-05-09T06:36:58.587931+0000","last_became_active":"2024-05-09T06:36:58.582612+0000","last_became_peered":"2024-05-09T06:36:58.582612+0000","last_unstale":"2024-05-09T06:36:58.587931+0000","last_undegraded":"2024-05-09T06:36:58.587931+0000","last_fullsized":"2024-05-09T06:36:58.587931+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_clean_scrub_stamp":"2024-05-09T06:36:57.547101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:29:21.168673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T06:36:58.588056+0000","last_change":"2024-05-09T06:36:58.584681+0000","last_active":"2024-05-09T06:36:58.588056+0000","last_peered":"2024-05-09T06:36:58.588056+0000","last_clean":"2024-05-09T06:36:58.588056+0000","last_became_active":"2024-05-09T06:36:58.584252+0000","last_became_peered":"2024-05-09T06:36:58.584252+0000","last_unstale":"2024-05-09T06:36:58.588056+0000","last_undegraded":"2024-05-09T06:36:58.588056+0000","last_fullsized":"2024-05-09T06:36:58.588056+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_clean_scrub_stamp":"2024-05-09T06:36:57.547101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:50:46.567561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T06:36:58.579784+0000","last_change":"2024-05-09T06:36:58.576434+0000","last_active":"2024-05-09T06:36:58.579784+0000","last_peered":"2024-05-09T06:36:58.579784+0000","last_clean":"2024-05-09T06:36:58.579784+0000","last_became_active":"2024-05-09T06:36:58.575957+0000","last_became_peered":"2024-05-09T06:36:58.575957+0000","last_unstale":"2024-05-09T06:36:58.579784+0000","last_undegraded":"2024-05-09T06:36:58.579784+0000","last_fullsized":"2024-05-09T06:36:58.579784+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_clean_scrub_stamp":"2024-05-09T06:36:57.547101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:35:21.215213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T06:36:58.578371+0000","last_change":"2024-05-09T06:36:58.567313+0000","last_active":"2024-05-09T06:36:58.578371+0000","last_peered":"2024-05-09T06:36:58.578371+0000","last_clean":"2024-05-09T06:36:58.578371+0000","last_became_active":"2024-05-09T06:36:58.567039+0000","last_became_peered":"2024-05-09T06:36:58.567039+0000","last_unstale":"2024-05-09T06:36:58.578371+0000","last_undegraded":"2024-05-09T06:36:58.578371+0000","last_fullsized":"2024-05-09T06:36:58.578371+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_clean_scrub_stamp":"2024-05-09T06:36:57.547101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:54:47.768120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T06:36:58.580426+0000","last_change":"2024-05-09T06:36:58.576038+0000","last_active":"2024-05-09T06:36:58.580426+0000","last_peered":"2024-05-09T06:36:58.580426+0000","last_clean":"2024-05-09T06:36:58.580426+0000","last_became_active":"2024-05-09T06:36:58.575771+0000","last_became_peered":"2024-05-09T06:36:58.575771+0000","last_unstale":"2024-05-09T06:36:58.580426+0000","last_undegraded":"2024-05-09T06:36:58.580426+0000","last_fullsized":"2024-05-09T06:36:58.580426+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_clean_scrub_stamp":"2024-05-09T06:36:57.547101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:47:37.035146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T06:36:58.580632+0000","last_change":"2024-05-09T06:36:58.574578+0000","last_active":"2024-05-09T06:36:58.580632+0000","last_peered":"2024-05-09T06:36:58.580632+0000","last_clean":"2024-05-09T06:36:58.580632+0000","last_became_active":"2024-05-09T06:36:58.574241+0000","last_became_peered":"2024-05-09T06:36:58.574241+0000","last_unstale":"2024-05-09T06:36:58.580632+0000","last_undegraded":"2024-05-09T06:36:58.580632+0000","last_fullsized":"2024-05-09T06:36:58.580632+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_clean_scrub_stamp":"2024-05-09T06:36:57.547101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:17:59.484786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T06:36:58.585461+0000","last_change":"2024-05-09T06:36:58.581463+0000","last_active":"2024-05-09T06:36:58.585461+0000","last_peered":"2024-05-09T06:36:58.585461+0000","last_clean":"2024-05-09T06:36:58.585461+0000","last_became_active":"2024-05-09T06:36:58.581171+0000","last_became_peered":"2024-05-09T06:36:58.581171+0000","last_unstale":"2024-05-09T06:36:58.585461+0000","last_undegraded":"2024-05-09T06:36:58.585461+0000","last_fullsized":"2024-05-09T06:36:58.585461+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_clean_scrub_stamp":"2024-05-09T06:36:57.547101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:52:21.177837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T06:36:58.580302+0000","last_change":"2024-05-09T06:36:58.575579+0000","last_active":"2024-05-09T06:36:58.580302+0000","last_peered":"2024-05-09T06:36:58.580302+0000","last_clean":"2024-05-09T06:36:58.580302+0000","last_became_active":"2024-05-09T06:36:58.575290+0000","last_became_peered":"2024-05-09T06:36:58.575290+0000","last_unstale":"2024-05-09T06:36:58.580302+0000","last_undegraded":"2024-05-09T06:36:58.580302+0000","last_fullsized":"2024-05-09T06:36:58.580302+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_clean_scrub_stamp":"2024-05-09T06:36:57.547101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:47:37.035146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"32'4","reported_seq":16,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T06:36:58.582987+0000","last_change":"2024-05-09T06:36:57.573551+0000","last_active":"2024-05-09T06:36:58.582987+0000","last_peered":"2024-05-09T06:36:58.582987+0000","last_clean":"2024-05-09T06:36:58.582987+0000","last_became_active":"2024-05-09T06:36:57.573003+0000","last_became_peered":"2024-05-09T06:36:57.573003+0000","last_unstale":"2024-05-09T06:36:58.582987+0000","last_undegraded":"2024-05-09T06:36:58.582987+0000","last_fullsized":"2024-05-09T06:36:58.582987+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:56.544873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:56.544873+0000","last_clean_scrub_stamp":"2024-05-09T06:36:56.544873+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:59:52.595776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T06:36:58.573623+0000","last_change":"2024-05-09T06:36:58.571321+0000","last_active":"2024-05-09T06:36:58.573623+0000","last_peered":"2024-05-09T06:36:58.573623+0000","last_clean":"2024-05-09T06:36:58.573623+0000","last_became_active":"2024-05-09T06:36:58.570982+0000","last_became_peered":"2024-05-09T06:36:58.570982+0000","last_unstale":"2024-05-09T06:36:58.573623+0000","last_undegraded":"2024-05-09T06:36:58.573623+0000","last_fullsized":"2024-05-09T06:36:58.573623+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_clean_scrub_stamp":"2024-05-09T06:36:57.547101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:03:45.503132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"32'1","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T06:36:58.578002+0000","last_change":"2024-05-09T06:36:57.562690+0000","last_active":"2024-05-09T06:36:58.578002+0000","last_peered":"2024-05-09T06:36:58.578002+0000","last_clean":"2024-05-09T06:36:58.578002+0000","last_became_active":"2024-05-09T06:36:57.562354+0000","last_became_peered":"2024-05-09T06:36:57.562354+0000","last_unstale":"2024-05-09T06:36:58.578002+0000","last_undegraded":"2024-05-09T06:36:58.578002+0000","last_fullsized":"2024-05-09T06:36:58.578002+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:56.544873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:56.544873+0000","last_clean_scrub_stamp":"2024-05-09T06:36:56.544873+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:59:52.595776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T06:36:58.572625+0000","last_change":"2024-05-09T06:36:58.566855+0000","last_active":"2024-05-09T06:36:58.572625+0000","last_peered":"2024-05-09T06:36:58.572625+0000","last_clean":"2024-05-09T06:36:58.572625+0000","last_became_active":"2024-05-09T06:36:58.566571+0000","last_became_peered":"2024-05-09T06:36:58.566571+0000","last_unstale":"2024-05-09T06:36:58.572625+0000","last_undegraded":"2024-05-09T06:36:58.572625+0000","last_fullsized":"2024-05-09T06:36:58.572625+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_clean_scrub_stamp":"2024-05-09T06:36:57.547101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:11:08.902382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"32'2","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T06:36:58.588675+0000","last_change":"2024-05-09T06:36:57.579970+0000","last_active":"2024-05-09T06:36:58.588675+0000","last_peered":"2024-05-09T06:36:58.588675+0000","last_clean":"2024-05-09T06:36:58.588675+0000","last_became_active":"2024-05-09T06:36:57.579660+0000","last_became_peered":"2024-05-09T06:36:57.579660+0000","last_unstale":"2024-05-09T06:36:58.588675+0000","last_undegraded":"2024-05-09T06:36:58.588675+0000","last_fullsized":"2024-05-09T06:36:58.588675+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:56.544873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:56.544873+0000","last_clean_scrub_stamp":"2024-05-09T06:36:56.544873+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:14:45.328973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T06:36:58.587578+0000","last_change":"2024-05-09T06:36:58.583966+0000","last_active":"2024-05-09T06:36:58.587578+0000","last_peered":"2024-05-09T06:36:58.587578+0000","last_clean":"2024-05-09T06:36:58.587578+0000","last_became_active":"2024-05-09T06:36:58.583393+0000","last_became_peered":"2024-05-09T06:36:58.583393+0000","last_unstale":"2024-05-09T06:36:58.587578+0000","last_undegraded":"2024-05-09T06:36:58.587578+0000","last_fullsized":"2024-05-09T06:36:58.587578+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_clean_scrub_stamp":"2024-05-09T06:36:57.547101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:52:21.177837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"32'2","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T06:36:58.575278+0000","last_change":"2024-05-09T06:36:57.576761+0000","last_active":"2024-05-09T06:36:58.575278+0000","last_peered":"2024-05-09T06:36:58.575278+0000","last_clean":"2024-05-09T06:36:58.575278+0000","last_became_active":"2024-05-09T06:36:57.576487+0000","last_became_peered":"2024-05-09T06:36:57.576487+0000","last_unstale":"2024-05-09T06:36:58.575278+0000","last_undegraded":"2024-05-09T06:36:58.575278+0000","last_fullsized":"2024-05-09T06:36:58.575278+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:56.544873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:56.544873+0000","last_clean_scrub_stamp":"2024-05-09T06:36:56.544873+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:25:12.896515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T06:36:58.585455+0000","last_change":"2024-05-09T06:36:58.580707+0000","last_active":"2024-05-09T06:36:58.585455+0000","last_peered":"2024-05-09T06:36:58.585455+0000","last_clean":"2024-05-09T06:36:58.585455+0000","last_became_active":"2024-05-09T06:36:58.580283+0000","last_became_peered":"2024-05-09T06:36:58.580283+0000","last_unstale":"2024-05-09T06:36:58.585455+0000","last_undegraded":"2024-05-09T06:36:58.585455+0000","last_fullsized":"2024-05-09T06:36:58.585455+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_clean_scrub_stamp":"2024-05-09T06:36:57.547101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:14:46.331201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"32'1","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T06:36:58.588103+0000","last_change":"2024-05-09T06:36:57.572231+0000","last_active":"2024-05-09T06:36:58.588103+0000","last_peered":"2024-05-09T06:36:58.588103+0000","last_clean":"2024-05-09T06:36:58.588103+0000","last_became_active":"2024-05-09T06:36:57.571937+0000","last_became_peered":"2024-05-09T06:36:57.571937+0000","last_unstale":"2024-05-09T06:36:58.588103+0000","last_undegraded":"2024-05-09T06:36:58.588103+0000","last_fullsized":"2024-05-09T06:36:58.588103+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:56.544873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:56.544873+0000","last_clean_scrub_stamp":"2024-05-09T06:36:56.544873+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:44:53.136881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T06:36:58.579731+0000","last_change":"2024-05-09T06:36:58.575508+0000","last_active":"2024-05-09T06:36:58.579731+0000","last_peered":"2024-05-09T06:36:58.579731+0000","last_clean":"2024-05-09T06:36:58.579731+0000","last_became_active":"2024-05-09T06:36:58.575256+0000","last_became_peered":"2024-05-09T06:36:58.575256+0000","last_unstale":"2024-05-09T06:36:58.579731+0000","last_undegraded":"2024-05-09T06:36:58.579731+0000","last_fullsized":"2024-05-09T06:36:58.579731+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_clean_scrub_stamp":"2024-05-09T06:36:57.547101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:52:21.177837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"32'1","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T06:36:58.588064+0000","last_change":"2024-05-09T06:36:57.572722+0000","last_active":"2024-05-09T06:36:58.588064+0000","last_peered":"2024-05-09T06:36:58.588064+0000","last_clean":"2024-05-09T06:36:58.588064+0000","last_became_active":"2024-05-09T06:36:57.572242+0000","last_became_peered":"2024-05-09T06:36:57.572242+0000","last_unstale":"2024-05-09T06:36:58.588064+0000","last_undegraded":"2024-05-09T06:36:58.588064+0000","last_fullsized":"2024-05-09T06:36:58.588064+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:56.544873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:56.544873+0000","last_clean_scrub_stamp":"2024-05-09T06:36:56.544873+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:31:45.233338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T06:36:58.578328+0000","last_change":"2024-05-09T06:36:58.568074+0000","last_active":"2024-05-09T06:36:58.578328+0000","last_peered":"2024-05-09T06:36:58.578328+0000","last_clean":"2024-05-09T06:36:58.578328+0000","last_became_active":"2024-05-09T06:36:58.567787+0000","last_became_peered":"2024-05-09T06:36:58.567787+0000","last_unstale":"2024-05-09T06:36:58.578328+0000","last_undegraded":"2024-05-09T06:36:58.578328+0000","last_fullsized":"2024-05-09T06:36:58.578328+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_clean_scrub_stamp":"2024-05-09T06:36:57.547101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:05:38.369597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T06:36:58.558272+0000","last_change":"2024-05-09T06:36:57.577513+0000","last_active":"2024-05-09T06:36:58.558272+0000","last_peered":"2024-05-09T06:36:58.558272+0000","last_clean":"2024-05-09T06:36:58.558272+0000","last_became_active":"2024-05-09T06:36:57.577217+0000","last_became_peered":"2024-05-09T06:36:57.577217+0000","last_unstale":"2024-05-09T06:36:58.558272+0000","last_undegraded":"2024-05-09T06:36:58.558272+0000","last_fullsized":"2024-05-09T06:36:58.558272+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:56.544873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:56.544873+0000","last_clean_scrub_stamp":"2024-05-09T06:36:56.544873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:02:31.391951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T06:36:58.585339+0000","last_change":"2024-05-09T06:36:58.582556+0000","last_active":"2024-05-09T06:36:58.585339+0000","last_peered":"2024-05-09T06:36:58.585339+0000","last_clean":"2024-05-09T06:36:58.585339+0000","last_became_active":"2024-05-09T06:36:58.582256+0000","last_became_peered":"2024-05-09T06:36:58.582256+0000","last_unstale":"2024-05-09T06:36:58.585339+0000","last_undegraded":"2024-05-09T06:36:58.585339+0000","last_fullsized":"2024-05-09T06:36:58.585339+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_clean_scrub_stamp":"2024-05-09T06:36:57.547101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:39:22.810622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T06:36:58.558678+0000","last_change":"2024-05-09T06:36:57.571350+0000","last_active":"2024-05-09T06:36:58.558678+0000","last_peered":"2024-05-09T06:36:58.558678+0000","last_clean":"2024-05-09T06:36:58.558678+0000","last_became_active":"2024-05-09T06:36:57.570918+0000","last_became_peered":"2024-05-09T06:36:57.570918+0000","last_unstale":"2024-05-09T06:36:58.558678+0000","last_undegraded":"2024-05-09T06:36:58.558678+0000","last_fullsized":"2024-05-09T06:36:58.558678+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:56.544873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:56.544873+0000","last_clean_scrub_stamp":"2024-05-09T06:36:56.544873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:18:05.558606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T06:36:58.575869+0000","last_change":"2024-05-09T06:36:58.570138+0000","last_active":"2024-05-09T06:36:58.575869+0000","last_peered":"2024-05-09T06:36:58.575869+0000","last_clean":"2024-05-09T06:36:58.575869+0000","last_became_active":"2024-05-09T06:36:58.569862+0000","last_became_peered":"2024-05-09T06:36:58.569862+0000","last_unstale":"2024-05-09T06:36:58.575869+0000","last_undegraded":"2024-05-09T06:36:58.575869+0000","last_fullsized":"2024-05-09T06:36:58.575869+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_clean_scrub_stamp":"2024-05-09T06:36:57.547101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:02:32.394179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T06:36:58.558490+0000","last_change":"2024-05-09T06:36:57.576687+0000","last_active":"2024-05-09T06:36:58.558490+0000","last_peered":"2024-05-09T06:36:58.558490+0000","last_clean":"2024-05-09T06:36:58.558490+0000","last_became_active":"2024-05-09T06:36:57.576390+0000","last_became_peered":"2024-05-09T06:36:57.576390+0000","last_unstale":"2024-05-09T06:36:58.558490+0000","last_undegraded":"2024-05-09T06:36:58.558490+0000","last_fullsized":"2024-05-09T06:36:58.558490+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:56.544873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:56.544873+0000","last_clean_scrub_stamp":"2024-05-09T06:36:56.544873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:39:21.808394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T06:36:58.588002+0000","last_change":"2024-05-09T06:36:58.583652+0000","last_active":"2024-05-09T06:36:58.588002+0000","last_peered":"2024-05-09T06:36:58.588002+0000","last_clean":"2024-05-09T06:36:58.588002+0000","last_became_active":"2024-05-09T06:36:58.583365+0000","last_became_peered":"2024-05-09T06:36:58.583365+0000","last_unstale":"2024-05-09T06:36:58.588002+0000","last_undegraded":"2024-05-09T06:36:58.588002+0000","last_fullsized":"2024-05-09T06:36:58.588002+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_clean_scrub_stamp":"2024-05-09T06:36:57.547101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:30:41.385698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T06:36:58.570107+0000","last_change":"2024-05-09T06:36:57.579767+0000","last_active":"2024-05-09T06:36:58.570107+0000","last_peered":"2024-05-09T06:36:58.570107+0000","last_clean":"2024-05-09T06:36:58.570107+0000","last_became_active":"2024-05-09T06:36:57.579451+0000","last_became_peered":"2024-05-09T06:36:57.579451+0000","last_unstale":"2024-05-09T06:36:58.570107+0000","last_undegraded":"2024-05-09T06:36:58.570107+0000","last_fullsized":"2024-05-09T06:36:58.570107+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:56.544873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:56.544873+0000","last_clean_scrub_stamp":"2024-05-09T06:36:56.544873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:03:44.500904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T06:36:58.588128+0000","last_change":"2024-05-09T06:36:58.585258+0000","last_active":"2024-05-09T06:36:58.588128+0000","last_peered":"2024-05-09T06:36:58.588128+0000","last_clean":"2024-05-09T06:36:58.588128+0000","last_became_active":"2024-05-09T06:36:58.584564+0000","last_became_peered":"2024-05-09T06:36:58.584564+0000","last_unstale":"2024-05-09T06:36:58.588128+0000","last_undegraded":"2024-05-09T06:36:58.588128+0000","last_fullsized":"2024-05-09T06:36:58.588128+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_clean_scrub_stamp":"2024-05-09T06:36:57.547101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:00:45.954062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"32'3","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T06:36:58.581840+0000","last_change":"2024-05-09T06:36:57.575448+0000","last_active":"2024-05-09T06:36:58.581840+0000","last_peered":"2024-05-09T06:36:58.581840+0000","last_clean":"2024-05-09T06:36:58.581840+0000","last_became_active":"2024-05-09T06:36:57.575191+0000","last_became_peered":"2024-05-09T06:36:57.575191+0000","last_unstale":"2024-05-09T06:36:58.581840+0000","last_undegraded":"2024-05-09T06:36:58.581840+0000","last_fullsized":"2024-05-09T06:36:58.581840+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:56.544873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:56.544873+0000","last_clean_scrub_stamp":"2024-05-09T06:36:56.544873+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:17:58.482558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T06:36:58.590831+0000","last_change":"2024-05-09T06:36:58.586260+0000","last_active":"2024-05-09T06:36:58.590831+0000","last_peered":"2024-05-09T06:36:58.590831+0000","last_clean":"2024-05-09T06:36:58.590831+0000","last_became_active":"2024-05-09T06:36:58.586003+0000","last_became_peered":"2024-05-09T06:36:58.586003+0000","last_unstale":"2024-05-09T06:36:58.590831+0000","last_undegraded":"2024-05-09T06:36:58.590831+0000","last_fullsized":"2024-05-09T06:36:58.590831+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_clean_scrub_stamp":"2024-05-09T06:36:57.547101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:49:32.371985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"32'6","reported_seq":18,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T06:36:58.595551+0000","last_change":"2024-05-09T06:36:57.579927+0000","last_active":"2024-05-09T06:36:58.595551+0000","last_peered":"2024-05-09T06:36:58.595551+0000","last_clean":"2024-05-09T06:36:58.595551+0000","last_became_active":"2024-05-09T06:36:57.579612+0000","last_became_peered":"2024-05-09T06:36:57.579612+0000","last_unstale":"2024-05-09T06:36:58.595551+0000","last_undegraded":"2024-05-09T06:36:58.595551+0000","last_fullsized":"2024-05-09T06:36:58.595551+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:56.544873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:56.544873+0000","last_clean_scrub_stamp":"2024-05-09T06:36:56.544873+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:50:45.565333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T06:36:58.585540+0000","last_change":"2024-05-09T06:36:58.582611+0000","last_active":"2024-05-09T06:36:58.585540+0000","last_peered":"2024-05-09T06:36:58.585540+0000","last_clean":"2024-05-09T06:36:58.585540+0000","last_became_active":"2024-05-09T06:36:58.582331+0000","last_became_peered":"2024-05-09T06:36:58.582331+0000","last_unstale":"2024-05-09T06:36:58.585540+0000","last_undegraded":"2024-05-09T06:36:58.585540+0000","last_fullsized":"2024-05-09T06:36:58.585540+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_clean_scrub_stamp":"2024-05-09T06:36:57.547101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:14:46.331201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"32'2","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T06:36:58.580836+0000","last_change":"2024-05-09T06:36:57.573842+0000","last_active":"2024-05-09T06:36:58.580836+0000","last_peered":"2024-05-09T06:36:58.580836+0000","last_clean":"2024-05-09T06:36:58.580836+0000","last_became_active":"2024-05-09T06:36:57.573569+0000","last_became_peered":"2024-05-09T06:36:57.573569+0000","last_unstale":"2024-05-09T06:36:58.580836+0000","last_undegraded":"2024-05-09T06:36:58.580836+0000","last_fullsized":"2024-05-09T06:36:58.580836+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:56.544873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:56.544873+0000","last_clean_scrub_stamp":"2024-05-09T06:36:56.544873+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:59:17.959203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T06:36:58.578741+0000","last_change":"2024-05-09T06:36:58.573239+0000","last_active":"2024-05-09T06:36:58.578741+0000","last_peered":"2024-05-09T06:36:58.578741+0000","last_clean":"2024-05-09T06:36:58.578741+0000","last_became_active":"2024-05-09T06:36:58.572737+0000","last_became_peered":"2024-05-09T06:36:58.572737+0000","last_unstale":"2024-05-09T06:36:58.578741+0000","last_undegraded":"2024-05-09T06:36:58.578741+0000","last_fullsized":"2024-05-09T06:36:58.578741+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_clean_scrub_stamp":"2024-05-09T06:36:57.547101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:40:37.529479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T06:36:58.570497+0000","last_change":"2024-05-09T06:36:57.568508+0000","last_active":"2024-05-09T06:36:58.570497+0000","last_peered":"2024-05-09T06:36:58.570497+0000","last_clean":"2024-05-09T06:36:58.570497+0000","last_became_active":"2024-05-09T06:36:57.568191+0000","last_became_peered":"2024-05-09T06:36:57.568191+0000","last_unstale":"2024-05-09T06:36:58.570497+0000","last_undegraded":"2024-05-09T06:36:58.570497+0000","last_fullsized":"2024-05-09T06:36:58.570497+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:56.544873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:56.544873+0000","last_clean_scrub_stamp":"2024-05-09T06:36:56.544873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:59:52.595776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T06:36:58.585479+0000","last_change":"2024-05-09T06:36:58.582085+0000","last_active":"2024-05-09T06:36:58.585479+0000","last_peered":"2024-05-09T06:36:58.585479+0000","last_clean":"2024-05-09T06:36:58.585479+0000","last_became_active":"2024-05-09T06:36:58.581809+0000","last_became_peered":"2024-05-09T06:36:58.581809+0000","last_unstale":"2024-05-09T06:36:58.585479+0000","last_undegraded":"2024-05-09T06:36:58.585479+0000","last_fullsized":"2024-05-09T06:36:58.585479+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_clean_scrub_stamp":"2024-05-09T06:36:57.547101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:50:46.567561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T06:36:58.557958+0000","last_change":"2024-05-09T06:36:57.575641+0000","last_active":"2024-05-09T06:36:58.557958+0000","last_peered":"2024-05-09T06:36:58.557958+0000","last_clean":"2024-05-09T06:36:58.557958+0000","last_became_active":"2024-05-09T06:36:57.575216+0000","last_became_peered":"2024-05-09T06:36:57.575216+0000","last_unstale":"2024-05-09T06:36:58.557958+0000","last_undegraded":"2024-05-09T06:36:58.557958+0000","last_fullsized":"2024-05-09T06:36:58.557958+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:56.544873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:56.544873+0000","last_clean_scrub_stamp":"2024-05-09T06:36:56.544873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:17:58.482558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T06:36:58.574432+0000","last_change":"2024-05-09T06:36:58.569346+0000","last_active":"2024-05-09T06:36:58.574432+0000","last_peered":"2024-05-09T06:36:58.574432+0000","last_clean":"2024-05-09T06:36:58.574432+0000","last_became_active":"2024-05-09T06:36:58.568493+0000","last_became_peered":"2024-05-09T06:36:58.568493+0000","last_unstale":"2024-05-09T06:36:58.574432+0000","last_undegraded":"2024-05-09T06:36:58.574432+0000","last_fullsized":"2024-05-09T06:36:58.574432+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_clean_scrub_stamp":"2024-05-09T06:36:57.547101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:02:41.385043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"32'8","reported_seq":20,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T06:36:58.588996+0000","last_change":"2024-05-09T06:36:57.580235+0000","last_active":"2024-05-09T06:36:58.588996+0000","last_peered":"2024-05-09T06:36:58.588996+0000","last_clean":"2024-05-09T06:36:58.588996+0000","last_became_active":"2024-05-09T06:36:57.579644+0000","last_became_peered":"2024-05-09T06:36:57.579644+0000","last_unstale":"2024-05-09T06:36:58.588996+0000","last_undegraded":"2024-05-09T06:36:58.588996+0000","last_fullsized":"2024-05-09T06:36:58.588996+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:56.544873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:56.544873+0000","last_clean_scrub_stamp":"2024-05-09T06:36:56.544873+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:52:20.175609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T06:36:58.580785+0000","last_change":"2024-05-09T06:36:58.577792+0000","last_active":"2024-05-09T06:36:58.580785+0000","last_peered":"2024-05-09T06:36:58.580785+0000","last_clean":"2024-05-09T06:36:58.580785+0000","last_became_active":"2024-05-09T06:36:58.577443+0000","last_became_peered":"2024-05-09T06:36:58.577443+0000","last_unstale":"2024-05-09T06:36:58.580785+0000","last_undegraded":"2024-05-09T06:36:58.580785+0000","last_fullsized":"2024-05-09T06:36:58.580785+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_clean_scrub_stamp":"2024-05-09T06:36:57.547101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:02:41.385043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"32'1","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T06:36:58.589522+0000","last_change":"2024-05-09T06:36:57.575115+0000","last_active":"2024-05-09T06:36:58.589522+0000","last_peered":"2024-05-09T06:36:58.589522+0000","last_clean":"2024-05-09T06:36:58.589522+0000","last_became_active":"2024-05-09T06:36:57.574770+0000","last_became_peered":"2024-05-09T06:36:57.574770+0000","last_unstale":"2024-05-09T06:36:58.589522+0000","last_undegraded":"2024-05-09T06:36:58.589522+0000","last_fullsized":"2024-05-09T06:36:58.589522+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:56.544873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:56.544873+0000","last_clean_scrub_stamp":"2024-05-09T06:36:56.544873+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:17:58.482558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T06:36:58.582605+0000","last_change":"2024-05-09T06:36:58.578899+0000","last_active":"2024-05-09T06:36:58.582605+0000","last_peered":"2024-05-09T06:36:58.582605+0000","last_clean":"2024-05-09T06:36:58.582605+0000","last_became_active":"2024-05-09T06:36:58.578611+0000","last_became_peered":"2024-05-09T06:36:58.578611+0000","last_unstale":"2024-05-09T06:36:58.582605+0000","last_undegraded":"2024-05-09T06:36:58.582605+0000","last_fullsized":"2024-05-09T06:36:58.582605+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_clean_scrub_stamp":"2024-05-09T06:36:57.547101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:40:37.529479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"32'2","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T06:36:58.578688+0000","last_change":"2024-05-09T06:36:57.573885+0000","last_active":"2024-05-09T06:36:58.578688+0000","last_peered":"2024-05-09T06:36:58.578688+0000","last_clean":"2024-05-09T06:36:58.578688+0000","last_became_active":"2024-05-09T06:36:57.573554+0000","last_became_peered":"2024-05-09T06:36:57.573554+0000","last_unstale":"2024-05-09T06:36:58.578688+0000","last_undegraded":"2024-05-09T06:36:58.578688+0000","last_fullsized":"2024-05-09T06:36:58.578688+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:56.544873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:56.544873+0000","last_clean_scrub_stamp":"2024-05-09T06:36:56.544873+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:08:21.330262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T06:36:58.588464+0000","last_change":"2024-05-09T06:36:58.586396+0000","last_active":"2024-05-09T06:36:58.588464+0000","last_peered":"2024-05-09T06:36:58.588464+0000","last_clean":"2024-05-09T06:36:58.588464+0000","last_became_active":"2024-05-09T06:36:58.586103+0000","last_became_peered":"2024-05-09T06:36:58.586103+0000","last_unstale":"2024-05-09T06:36:58.588464+0000","last_undegraded":"2024-05-09T06:36:58.588464+0000","last_fullsized":"2024-05-09T06:36:58.588464+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_clean_scrub_stamp":"2024-05-09T06:36:57.547101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:35:21.215213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"32'1","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T06:36:58.584306+0000","last_change":"2024-05-09T06:36:57.572543+0000","last_active":"2024-05-09T06:36:58.584306+0000","last_peered":"2024-05-09T06:36:58.584306+0000","last_clean":"2024-05-09T06:36:58.584306+0000","last_became_active":"2024-05-09T06:36:57.572287+0000","last_became_peered":"2024-05-09T06:36:57.572287+0000","last_unstale":"2024-05-09T06:36:58.584306+0000","last_undegraded":"2024-05-09T06:36:58.584306+0000","last_fullsized":"2024-05-09T06:36:58.584306+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:56.544873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:56.544873+0000","last_clean_scrub_stamp":"2024-05-09T06:36:56.544873+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:22:40.811061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T06:36:58.581990+0000","last_change":"2024-05-09T06:36:58.576676+0000","last_active":"2024-05-09T06:36:58.581990+0000","last_peered":"2024-05-09T06:36:58.581990+0000","last_clean":"2024-05-09T06:36:58.581990+0000","last_became_active":"2024-05-09T06:36:58.576282+0000","last_became_peered":"2024-05-09T06:36:58.576282+0000","last_unstale":"2024-05-09T06:36:58.581990+0000","last_undegraded":"2024-05-09T06:36:58.581990+0000","last_fullsized":"2024-05-09T06:36:58.581990+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_clean_scrub_stamp":"2024-05-09T06:36:57.547101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:41:10.976320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T06:36:58.559460+0000","last_change":"2024-05-09T06:36:57.571629+0000","last_active":"2024-05-09T06:36:58.559460+0000","last_peered":"2024-05-09T06:36:58.559460+0000","last_clean":"2024-05-09T06:36:58.559460+0000","last_became_active":"2024-05-09T06:36:57.571349+0000","last_became_peered":"2024-05-09T06:36:57.571349+0000","last_unstale":"2024-05-09T06:36:58.559460+0000","last_undegraded":"2024-05-09T06:36:58.559460+0000","last_fullsized":"2024-05-09T06:36:58.559460+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:56.544873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:56.544873+0000","last_clean_scrub_stamp":"2024-05-09T06:36:56.544873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:25:29.082214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T06:36:58.569500+0000","last_change":"2024-05-09T06:36:58.568515+0000","last_active":"2024-05-09T06:36:58.569500+0000","last_peered":"2024-05-09T06:36:58.569500+0000","last_clean":"2024-05-09T06:36:58.569500+0000","last_became_active":"2024-05-09T06:36:58.568217+0000","last_became_peered":"2024-05-09T06:36:58.568217+0000","last_unstale":"2024-05-09T06:36:58.569500+0000","last_undegraded":"2024-05-09T06:36:58.569500+0000","last_fullsized":"2024-05-09T06:36:58.569500+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_clean_scrub_stamp":"2024-05-09T06:36:57.547101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:11:18.718993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"32'2","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T06:36:58.585701+0000","last_change":"2024-05-09T06:36:57.571198+0000","last_active":"2024-05-09T06:36:58.585701+0000","last_peered":"2024-05-09T06:36:58.585701+0000","last_clean":"2024-05-09T06:36:58.585701+0000","last_became_active":"2024-05-09T06:36:57.570902+0000","last_became_peered":"2024-05-09T06:36:57.570902+0000","last_unstale":"2024-05-09T06:36:58.585701+0000","last_undegraded":"2024-05-09T06:36:58.585701+0000","last_fullsized":"2024-05-09T06:36:58.585701+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:56.544873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:56.544873+0000","last_clean_scrub_stamp":"2024-05-09T06:36:56.544873+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:13:48.003027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T06:36:58.577070+0000","last_change":"2024-05-09T06:36:58.573013+0000","last_active":"2024-05-09T06:36:58.577070+0000","last_peered":"2024-05-09T06:36:58.577070+0000","last_clean":"2024-05-09T06:36:58.577070+0000","last_became_active":"2024-05-09T06:36:58.572718+0000","last_became_peered":"2024-05-09T06:36:58.572718+0000","last_unstale":"2024-05-09T06:36:58.577070+0000","last_undegraded":"2024-05-09T06:36:58.577070+0000","last_fullsized":"2024-05-09T06:36:58.577070+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_clean_scrub_stamp":"2024-05-09T06:36:57.547101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:18:09.799782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T06:36:58.563676+0000","last_change":"2024-05-09T06:36:57.571413+0000","last_active":"2024-05-09T06:36:58.563676+0000","last_peered":"2024-05-09T06:36:58.563676+0000","last_clean":"2024-05-09T06:36:58.563676+0000","last_became_active":"2024-05-09T06:36:57.570893+0000","last_became_peered":"2024-05-09T06:36:57.570893+0000","last_unstale":"2024-05-09T06:36:58.563676+0000","last_undegraded":"2024-05-09T06:36:58.563676+0000","last_fullsized":"2024-05-09T06:36:58.563676+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:56.544873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:56.544873+0000","last_clean_scrub_stamp":"2024-05-09T06:36:56.544873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:54:46.765892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T06:36:58.581921+0000","last_change":"2024-05-09T06:36:58.574149+0000","last_active":"2024-05-09T06:36:58.581921+0000","last_peered":"2024-05-09T06:36:58.581921+0000","last_clean":"2024-05-09T06:36:58.581921+0000","last_became_active":"2024-05-09T06:36:58.573789+0000","last_became_peered":"2024-05-09T06:36:58.573789+0000","last_unstale":"2024-05-09T06:36:58.581921+0000","last_undegraded":"2024-05-09T06:36:58.581921+0000","last_fullsized":"2024-05-09T06:36:58.581921+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_clean_scrub_stamp":"2024-05-09T06:36:57.547101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:45:52.389606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T06:36:58.558676+0000","last_change":"2024-05-09T06:36:57.576882+0000","last_active":"2024-05-09T06:36:58.558676+0000","last_peered":"2024-05-09T06:36:58.558676+0000","last_clean":"2024-05-09T06:36:58.558676+0000","last_became_active":"2024-05-09T06:36:57.576586+0000","last_became_peered":"2024-05-09T06:36:57.576586+0000","last_unstale":"2024-05-09T06:36:58.558676+0000","last_undegraded":"2024-05-09T06:36:58.558676+0000","last_fullsized":"2024-05-09T06:36:58.558676+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:56.544873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:56.544873+0000","last_clean_scrub_stamp":"2024-05-09T06:36:56.544873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:05:16.941219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T06:36:58.580873+0000","last_change":"2024-05-09T06:36:58.574529+0000","last_active":"2024-05-09T06:36:58.580873+0000","last_peered":"2024-05-09T06:36:58.580873+0000","last_clean":"2024-05-09T06:36:58.580873+0000","last_became_active":"2024-05-09T06:36:58.574272+0000","last_became_peered":"2024-05-09T06:36:58.574272+0000","last_unstale":"2024-05-09T06:36:58.580873+0000","last_undegraded":"2024-05-09T06:36:58.580873+0000","last_fullsized":"2024-05-09T06:36:58.580873+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_clean_scrub_stamp":"2024-05-09T06:36:57.547101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:13:49.005255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T06:36:58.566319+0000","last_change":"2024-05-09T06:36:57.575260+0000","last_active":"2024-05-09T06:36:58.566319+0000","last_peered":"2024-05-09T06:36:58.566319+0000","last_clean":"2024-05-09T06:36:58.566319+0000","last_became_active":"2024-05-09T06:36:57.574970+0000","last_became_peered":"2024-05-09T06:36:57.574970+0000","last_unstale":"2024-05-09T06:36:58.566319+0000","last_undegraded":"2024-05-09T06:36:58.566319+0000","last_fullsized":"2024-05-09T06:36:58.566319+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:56.544873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:56.544873+0000","last_clean_scrub_stamp":"2024-05-09T06:36:56.544873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:47:36.032918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T06:36:58.584127+0000","last_change":"2024-05-09T06:36:58.578342+0000","last_active":"2024-05-09T06:36:58.584127+0000","last_peered":"2024-05-09T06:36:58.584127+0000","last_clean":"2024-05-09T06:36:58.584127+0000","last_became_active":"2024-05-09T06:36:58.578063+0000","last_became_peered":"2024-05-09T06:36:58.578063+0000","last_unstale":"2024-05-09T06:36:58.584127+0000","last_undegraded":"2024-05-09T06:36:58.584127+0000","last_fullsized":"2024-05-09T06:36:58.584127+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_clean_scrub_stamp":"2024-05-09T06:36:57.547101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:05:17.943447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"32'2","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T06:36:58.587478+0000","last_change":"2024-05-09T06:36:57.575847+0000","last_active":"2024-05-09T06:36:58.587478+0000","last_peered":"2024-05-09T06:36:58.587478+0000","last_clean":"2024-05-09T06:36:58.587478+0000","last_became_active":"2024-05-09T06:36:57.575472+0000","last_became_peered":"2024-05-09T06:36:57.575472+0000","last_unstale":"2024-05-09T06:36:58.587478+0000","last_undegraded":"2024-05-09T06:36:58.587478+0000","last_fullsized":"2024-05-09T06:36:58.587478+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:56.544873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:56.544873+0000","last_clean_scrub_stamp":"2024-05-09T06:36:56.544873+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:13:48.003027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T06:36:58.588590+0000","last_change":"2024-05-09T06:36:58.585339+0000","last_active":"2024-05-09T06:36:58.588590+0000","last_peered":"2024-05-09T06:36:58.588590+0000","last_clean":"2024-05-09T06:36:58.588590+0000","last_became_active":"2024-05-09T06:36:58.585018+0000","last_became_peered":"2024-05-09T06:36:58.585018+0000","last_unstale":"2024-05-09T06:36:58.588590+0000","last_undegraded":"2024-05-09T06:36:58.588590+0000","last_fullsized":"2024-05-09T06:36:58.588590+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_clean_scrub_stamp":"2024-05-09T06:36:57.547101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:25:07.739454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T06:36:58.569942+0000","last_change":"2024-05-09T06:36:57.571388+0000","last_active":"2024-05-09T06:36:58.569942+0000","last_peered":"2024-05-09T06:36:58.569942+0000","last_clean":"2024-05-09T06:36:58.569942+0000","last_became_active":"2024-05-09T06:36:57.570865+0000","last_became_peered":"2024-05-09T06:36:57.570865+0000","last_unstale":"2024-05-09T06:36:58.569942+0000","last_undegraded":"2024-05-09T06:36:58.569942+0000","last_fullsized":"2024-05-09T06:36:58.569942+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:56.544873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:56.544873+0000","last_clean_scrub_stamp":"2024-05-09T06:36:56.544873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:02:40.382815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T06:36:58.583993+0000","last_change":"2024-05-09T06:36:58.583093+0000","last_active":"2024-05-09T06:36:58.583993+0000","last_peered":"2024-05-09T06:36:58.583993+0000","last_clean":"2024-05-09T06:36:58.583993+0000","last_became_active":"2024-05-09T06:36:58.582823+0000","last_became_peered":"2024-05-09T06:36:58.582823+0000","last_unstale":"2024-05-09T06:36:58.583993+0000","last_undegraded":"2024-05-09T06:36:58.583993+0000","last_fullsized":"2024-05-09T06:36:58.583993+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_clean_scrub_stamp":"2024-05-09T06:36:57.547101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:14:46.331201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T06:36:58.573381+0000","last_change":"2024-05-09T06:36:57.573519+0000","last_active":"2024-05-09T06:36:58.573381+0000","last_peered":"2024-05-09T06:36:58.573381+0000","last_clean":"2024-05-09T06:36:58.573381+0000","last_became_active":"2024-05-09T06:36:57.573157+0000","last_became_peered":"2024-05-09T06:36:57.573157+0000","last_unstale":"2024-05-09T06:36:58.573381+0000","last_undegraded":"2024-05-09T06:36:58.573381+0000","last_fullsized":"2024-05-09T06:36:58.573381+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:56.544873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:56.544873+0000","last_clean_scrub_stamp":"2024-05-09T06:36:56.544873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:03:44.500904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T06:36:58.579954+0000","last_change":"2024-05-09T06:36:58.572755+0000","last_active":"2024-05-09T06:36:58.579954+0000","last_peered":"2024-05-09T06:36:58.579954+0000","last_clean":"2024-05-09T06:36:58.579954+0000","last_became_active":"2024-05-09T06:36:58.572481+0000","last_became_peered":"2024-05-09T06:36:58.572481+0000","last_unstale":"2024-05-09T06:36:58.579954+0000","last_undegraded":"2024-05-09T06:36:58.579954+0000","last_fullsized":"2024-05-09T06:36:58.579954+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_clean_scrub_stamp":"2024-05-09T06:36:57.547101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:13:49.005255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T06:36:58.570538+0000","last_change":"2024-05-09T06:36:57.572806+0000","last_active":"2024-05-09T06:36:58.570538+0000","last_peered":"2024-05-09T06:36:58.570538+0000","last_clean":"2024-05-09T06:36:58.570538+0000","last_became_active":"2024-05-09T06:36:57.572446+0000","last_became_peered":"2024-05-09T06:36:57.572446+0000","last_unstale":"2024-05-09T06:36:58.570538+0000","last_undegraded":"2024-05-09T06:36:58.570538+0000","last_fullsized":"2024-05-09T06:36:58.570538+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:56.544873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:56.544873+0000","last_clean_scrub_stamp":"2024-05-09T06:36:56.544873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:17:58.482558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T06:36:58.584030+0000","last_change":"2024-05-09T06:36:58.581217+0000","last_active":"2024-05-09T06:36:58.584030+0000","last_peered":"2024-05-09T06:36:58.584030+0000","last_clean":"2024-05-09T06:36:58.584030+0000","last_became_active":"2024-05-09T06:36:58.580927+0000","last_became_peered":"2024-05-09T06:36:58.580927+0000","last_unstale":"2024-05-09T06:36:58.584030+0000","last_undegraded":"2024-05-09T06:36:58.584030+0000","last_fullsized":"2024-05-09T06:36:58.584030+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_clean_scrub_stamp":"2024-05-09T06:36:57.547101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:39:22.810622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T06:36:58.572558+0000","last_change":"2024-05-09T06:36:57.572648+0000","last_active":"2024-05-09T06:36:58.572558+0000","last_peered":"2024-05-09T06:36:58.572558+0000","last_clean":"2024-05-09T06:36:58.572558+0000","last_became_active":"2024-05-09T06:36:57.572365+0000","last_became_peered":"2024-05-09T06:36:57.572365+0000","last_unstale":"2024-05-09T06:36:58.572558+0000","last_undegraded":"2024-05-09T06:36:58.572558+0000","last_fullsized":"2024-05-09T06:36:58.572558+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:56.544873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:56.544873+0000","last_clean_scrub_stamp":"2024-05-09T06:36:56.544873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:02:40.382815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T06:36:58.588004+0000","last_change":"2024-05-09T06:36:58.582957+0000","last_active":"2024-05-09T06:36:58.588004+0000","last_peered":"2024-05-09T06:36:58.588004+0000","last_clean":"2024-05-09T06:36:58.588004+0000","last_became_active":"2024-05-09T06:36:58.582674+0000","last_became_peered":"2024-05-09T06:36:58.582674+0000","last_unstale":"2024-05-09T06:36:58.588004+0000","last_undegraded":"2024-05-09T06:36:58.588004+0000","last_fullsized":"2024-05-09T06:36:58.588004+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_clean_scrub_stamp":"2024-05-09T06:36:57.547101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:44:53.864242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T06:36:58.558908+0000","last_change":"2024-05-09T06:36:57.572999+0000","last_active":"2024-05-09T06:36:58.558908+0000","last_peered":"2024-05-09T06:36:58.558908+0000","last_clean":"2024-05-09T06:36:58.558908+0000","last_became_active":"2024-05-09T06:36:57.572327+0000","last_became_peered":"2024-05-09T06:36:57.572327+0000","last_unstale":"2024-05-09T06:36:58.558908+0000","last_undegraded":"2024-05-09T06:36:58.558908+0000","last_fullsized":"2024-05-09T06:36:58.558908+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:56.544873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:56.544873+0000","last_clean_scrub_stamp":"2024-05-09T06:36:56.544873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:45:00.356381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T06:36:58.573408+0000","last_change":"2024-05-09T06:36:58.565289+0000","last_active":"2024-05-09T06:36:58.573408+0000","last_peered":"2024-05-09T06:36:58.573408+0000","last_clean":"2024-05-09T06:36:58.573408+0000","last_became_active":"2024-05-09T06:36:58.564823+0000","last_became_peered":"2024-05-09T06:36:58.564823+0000","last_unstale":"2024-05-09T06:36:58.573408+0000","last_undegraded":"2024-05-09T06:36:58.573408+0000","last_fullsized":"2024-05-09T06:36:58.573408+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_clean_scrub_stamp":"2024-05-09T06:36:57.547101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:47:54.992336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T06:36:58.558147+0000","last_change":"2024-05-09T06:36:57.576383+0000","last_active":"2024-05-09T06:36:58.558147+0000","last_peered":"2024-05-09T06:36:58.558147+0000","last_clean":"2024-05-09T06:36:58.558147+0000","last_became_active":"2024-05-09T06:36:57.576046+0000","last_became_peered":"2024-05-09T06:36:57.576046+0000","last_unstale":"2024-05-09T06:36:58.558147+0000","last_undegraded":"2024-05-09T06:36:58.558147+0000","last_fullsized":"2024-05-09T06:36:58.558147+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:56.544873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:56.544873+0000","last_clean_scrub_stamp":"2024-05-09T06:36:56.544873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:47:36.032918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T06:36:58.579469+0000","last_change":"2024-05-09T06:36:58.574222+0000","last_active":"2024-05-09T06:36:58.579469+0000","last_peered":"2024-05-09T06:36:58.579469+0000","last_clean":"2024-05-09T06:36:58.579469+0000","last_became_active":"2024-05-09T06:36:58.573947+0000","last_became_peered":"2024-05-09T06:36:58.573947+0000","last_unstale":"2024-05-09T06:36:58.579469+0000","last_undegraded":"2024-05-09T06:36:58.579469+0000","last_fullsized":"2024-05-09T06:36:58.579469+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_clean_scrub_stamp":"2024-05-09T06:36:57.547101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:03:45.503132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T06:36:58.563918+0000","last_change":"2024-05-09T06:36:57.569766+0000","last_active":"2024-05-09T06:36:58.563918+0000","last_peered":"2024-05-09T06:36:58.563918+0000","last_clean":"2024-05-09T06:36:58.563918+0000","last_became_active":"2024-05-09T06:36:57.569458+0000","last_became_peered":"2024-05-09T06:36:57.569458+0000","last_unstale":"2024-05-09T06:36:58.563918+0000","last_undegraded":"2024-05-09T06:36:58.563918+0000","last_fullsized":"2024-05-09T06:36:58.563918+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:56.544873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:56.544873+0000","last_clean_scrub_stamp":"2024-05-09T06:36:56.544873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:05:37.367369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":191,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T06:36:58.570131+0000","last_change":"2024-05-09T06:35:55.895214+0000","last_active":"2024-05-09T06:36:58.570131+0000","last_peered":"2024-05-09T06:36:58.570131+0000","last_clean":"2024-05-09T06:36:58.570131+0000","last_became_active":"2024-05-09T06:35:55.894087+0000","last_became_peered":"2024-05-09T06:35:55.894087+0000","last_unstale":"2024-05-09T06:36:58.570131+0000","last_undegraded":"2024-05-09T06:36:58.570131+0000","last_fullsized":"2024-05-09T06:36:58.570131+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:35:54.820102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:35:54.820102+0000","last_clean_scrub_stamp":"2024-05-09T06:35:54.820102+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:29:20.408099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574865,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27944,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343896,"statfs":{"total":96636764160,"available":96608149504,"internally_reserved":0,"allocated":942080,"data_stored":723909,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574865,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27380,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344460,"statfs":{"total":96636764160,"available":96608727040,"internally_reserved":0,"allocated":368640,"data_stored":140932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574864,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27364,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344476,"statfs":{"total":96636764160,"available":96608743424,"internally_reserved":0,"allocated":348160,"data_stored":133541,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574864,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27380,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344460,"statfs":{"total":96636764160,"available":96608727040,"internally_reserved":0,"allocated":368640,"data_stored":140420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574865,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27364,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344476,"statfs":{"total":96636764160,"available":96608743424,"internally_reserved":0,"allocated":348160,"data_stored":133541,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574864,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27364,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344476,"statfs":{"total":96636764160,"available":96608743424,"internally_reserved":0,"allocated":348160,"data_stored":133541,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574864,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27364,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344476,"statfs":{"total":96636764160,"available":96608743424,"internally_reserved":0,"allocated":348160,"data_stored":133541,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574865,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27956,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343884,"statfs":{"total":96636764160,"available":96608137216,"internally_reserved":0,"allocated":954368,"data_stored":730684,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-09T06:37:02.575 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-09T06:37:02.712 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.731+0000 7f0bb0342640 1 Processor -- start 2024-05-09T06:37:02.712 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.731+0000 7f0bb0342640 1 -- start start 2024-05-09T06:37:02.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.732+0000 7f0bb0342640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f0ba815e4e0 0x7f0ba806ec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:02.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.732+0000 7f0bb0342640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0ba806f290 con 0x7f0ba80d0120 2024-05-09T06:37:02.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.732+0000 7f0bb0342640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0ba80ce3d0 con 0x7f0ba80af720 2024-05-09T06:37:02.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.732+0000 7f0bb0342640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f0ba806f570 con 0x7f0ba815e4e0 2024-05-09T06:37:02.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.732+0000 7f0bae8df640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f0ba80d0120 0x7f0ba80d0530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:58052/0 (socket says 172.21.15.3:58052) 2024-05-09T06:37:02.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.732+0000 7f0bae8df640 1 -- 172.21.15.3:0/2948893108 learned_addr learned my addr 172.21.15.3:0/2948893108 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-05-09T06:37:02.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.732+0000 7f0bae0de640 1 --2- 172.21.15.3:0/2948893108 >> v2:172.21.15.190:3300/0 conn(0x7f0ba815e4e0 0x7f0ba806ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:02.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.732+0000 7f0bad0dc640 1 -- 172.21.15.3:0/2948893108 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1464868650 0 0) 0x7f0ba806f290 con 0x7f0ba80d0120 2024-05-09T06:37:02.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.732+0000 7f0bad0dc640 1 -- 172.21.15.3:0/2948893108 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0b94003520 con 0x7f0ba80d0120 2024-05-09T06:37:02.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.732+0000 7f0bad0dc640 1 -- 172.21.15.3:0/2948893108 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1960169826 0 0) 0x7f0b94003520 con 0x7f0ba80d0120 2024-05-09T06:37:02.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.732+0000 7f0bad0dc640 1 -- 172.21.15.3:0/2948893108 >> v2:172.21.15.190:3300/0 conn(0x7f0ba815e4e0 msgr2=0x7f0ba806ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:37:02.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.732+0000 7f0bad0dc640 1 --2- 172.21.15.3:0/2948893108 >> v2:172.21.15.190:3300/0 conn(0x7f0ba815e4e0 0x7f0ba806ec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:02.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.732+0000 7f0bad0dc640 1 -- 172.21.15.3:0/2948893108 >> v1:172.21.15.190:6789/0 conn(0x7f0ba80af720 legacy=0x7f0ba80cc7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:02.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.732+0000 7f0bad0dc640 1 -- 172.21.15.3:0/2948893108 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ba80ce6b0 con 0x7f0ba80d0120 2024-05-09T06:37:02.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.732+0000 7f0bad0dc640 1 -- 172.21.15.3:0/2948893108 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 563312031 0 0) 0x7f0ba00019f0 con 0x7f0ba80d0120 2024-05-09T06:37:02.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.733+0000 7f0bad0dc640 1 -- 172.21.15.3:0/2948893108 >> v1:172.21.15.3:6789/0 conn(0x7f0ba80d0120 legacy=0x7f0ba80d0530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:02.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.733+0000 7f0bad0dc640 1 --2- 172.21.15.3:0/2948893108 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0b940039b0 0x7f0b94003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:02.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.733+0000 7f0bad0dc640 1 --2- 172.21.15.3:0/2948893108 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0b94004800 0x7f0b94004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:02.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.733+0000 7f0bad0dc640 1 --2- 172.21.15.3:0/2948893108 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0b940052e0 0x7f0b9400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:02.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.733+0000 7f0bad0dc640 1 -- 172.21.15.3:0/2948893108 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f0ba806f570 con 0x7f0b940052e0 2024-05-09T06:37:02.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.733+0000 7f0bad0dc640 1 -- 172.21.15.3:0/2948893108 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f0b9400c1b0 con 0x7f0b940039b0 2024-05-09T06:37:02.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.733+0000 7f0bad0dc640 1 -- 172.21.15.3:0/2948893108 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f0b9400c3b0 con 0x7f0b94004800 2024-05-09T06:37:02.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.733+0000 7f0bae0de640 1 --2- 172.21.15.3:0/2948893108 >> v2:172.21.15.190:3300/0 conn(0x7f0ba815e4e0 0x7f0ba806ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T06:37:02.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.733+0000 7f0bae8df640 1 --2- 172.21.15.3:0/2948893108 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0b940052e0 0x7f0b9400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:02.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.733+0000 7f0bad8dd640 1 --2- 172.21.15.3:0/2948893108 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0b94004800 0x7f0b94004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:02.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.733+0000 7f0bae0de640 1 --2- 172.21.15.3:0/2948893108 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0b940039b0 0x7f0b94003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:02.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.733+0000 7f0bae8df640 1 -- 172.21.15.3:0/2948893108 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0b94004800 msgr2=0x7f0b94004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:37:02.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.733+0000 7f0bae8df640 1 --2- 172.21.15.3:0/2948893108 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0b94004800 0x7f0b94004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:02.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.733+0000 7f0bae8df640 1 -- 172.21.15.3:0/2948893108 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0b940039b0 msgr2=0x7f0b94003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:37:02.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.733+0000 7f0bae8df640 1 --2- 172.21.15.3:0/2948893108 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0b940039b0 0x7f0b94003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:02.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.733+0000 7f0bae8df640 1 -- 172.21.15.3:0/2948893108 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b94003520 con 0x7f0b940052e0 2024-05-09T06:37:02.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.734+0000 7f0bae8df640 1 -- 172.21.15.3:0/2948893108 reap_dead start 2024-05-09T06:37:02.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.734+0000 7f0bae8df640 1 --2- 172.21.15.3:0/2948893108 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0b940052e0 0x7f0b9400b9c0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f0ba000a830 tx=0x7f0b94005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ded4e524dacfa0c1 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:02.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.734+0000 7f0bad0dc640 1 -- 172.21.15.3:0/2948893108 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ba000a480 con 0x7f0b940052e0 2024-05-09T06:37:02.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.734+0000 7f0bad0dc640 1 -- 172.21.15.3:0/2948893108 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ba002f070 con 0x7f0b940052e0 2024-05-09T06:37:02.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.734+0000 7f0bad0dc640 1 -- 172.21.15.3:0/2948893108 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ba002bc70 con 0x7f0b940052e0 2024-05-09T06:37:02.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.734+0000 7f0bb0342640 1 -- 172.21.15.3:0/2948893108 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0b940052e0 msgr2=0x7f0b9400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:02.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.734+0000 7f0bb0342640 1 --2- 172.21.15.3:0/2948893108 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0b940052e0 0x7f0b9400b9c0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f0ba000a830 tx=0x7f0b94005830 comp rx=0 tx=0).stop 2024-05-09T06:37:02.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.734+0000 7f0bb0342640 1 -- 172.21.15.3:0/2948893108 shutdown_connections 2024-05-09T06:37:02.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.734+0000 7f0bb0342640 1 --2- 172.21.15.3:0/2948893108 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0b940052e0 0x7f0b9400b9c0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:02.719 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.734+0000 7f0bb0342640 1 -- 172.21.15.3:0/2948893108 >> 172.21.15.3:0/2948893108 conn(0x7f0ba8072870 msgr2=0x7f0ba8072c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:02.719 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.735+0000 7f0bb0342640 1 -- 172.21.15.3:0/2948893108 shutdown_connections 2024-05-09T06:37:02.719 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.735+0000 7f0bb0342640 1 -- 172.21.15.3:0/2948893108 wait complete. 2024-05-09T06:37:02.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.736+0000 7f0bb0342640 1 Processor -- start 2024-05-09T06:37:02.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.736+0000 7f0bb0342640 1 -- start start 2024-05-09T06:37:02.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.736+0000 7f0bb0342640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0b940052e0 0x7f0ba8146690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:02.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.736+0000 7f0bb0342640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0ba80ccd90 0x7f0ba80c93c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:02.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.736+0000 7f0bb0342640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0ba8146db0 0x7f0ba8147230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:02.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.736+0000 7f0bb0342640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f0ba80cea20 con 0x7f0b940052e0 2024-05-09T06:37:02.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.736+0000 7f0bb0342640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f0ba815ead0 con 0x7f0ba80ccd90 2024-05-09T06:37:02.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.736+0000 7f0bb0342640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f0ba80736b0 con 0x7f0ba8146db0 2024-05-09T06:37:02.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.737+0000 7f0bae0de640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0b940052e0 0x7f0ba8146690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:02.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.737+0000 7f0bae0de640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0b940052e0 0x7f0ba8146690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:39416/0 (socket says 172.21.15.3:39416) 2024-05-09T06:37:02.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.737+0000 7f0bae0de640 1 -- 172.21.15.3:0/1530094896 learned_addr learned my addr 172.21.15.3:0/1530094896 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:37:02.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.737+0000 7f0bad8dd640 1 --2- 172.21.15.3:0/1530094896 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0ba80ccd90 0x7f0ba80c93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:02.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.737+0000 7f0bae8df640 1 --2- 172.21.15.3:0/1530094896 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0ba8146db0 0x7f0ba8147230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:02.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.737+0000 7f0bae0de640 1 -- 172.21.15.3:0/1530094896 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0ba8146db0 msgr2=0x7f0ba8147230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:02.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.737+0000 7f0bae0de640 1 --2- 172.21.15.3:0/1530094896 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0ba8146db0 0x7f0ba8147230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:02.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.737+0000 7f0bae0de640 1 -- 172.21.15.3:0/1530094896 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0ba80ccd90 msgr2=0x7f0ba80c93c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:02.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.737+0000 7f0bae0de640 1 --2- 172.21.15.3:0/1530094896 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0ba80ccd90 0x7f0ba80c93c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:02.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.737+0000 7f0bae0de640 1 -- 172.21.15.3:0/1530094896 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ba80c9b30 con 0x7f0b940052e0 2024-05-09T06:37:02.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.737+0000 7f0bad8dd640 1 --2- 172.21.15.3:0/1530094896 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0ba80ccd90 0x7f0ba80c93c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:37:02.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.737+0000 7f0bae0de640 1 --2- 172.21.15.3:0/1530094896 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0b940052e0 0x7f0ba8146690 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f0ba40116f0 tx=0x7f0ba4011bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:02.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.737+0000 7f0b9effd640 1 -- 172.21.15.3:0/1530094896 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ba400f9c0 con 0x7f0b940052e0 2024-05-09T06:37:02.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.737+0000 7f0b9effd640 1 -- 172.21.15.3:0/1530094896 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ba400fb60 con 0x7f0b940052e0 2024-05-09T06:37:02.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.737+0000 7f0bb0342640 1 -- 172.21.15.3:0/1530094896 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ba80c9e10 con 0x7f0b940052e0 2024-05-09T06:37:02.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.737+0000 7f0b9effd640 1 -- 172.21.15.3:0/1530094896 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ba4015460 con 0x7f0b940052e0 2024-05-09T06:37:02.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.737+0000 7f0bb0342640 1 -- 172.21.15.3:0/1530094896 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ba80ca3d0 con 0x7f0b940052e0 2024-05-09T06:37:02.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.738+0000 7f0bb0342640 1 -- 172.21.15.3:0/1530094896 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ba804f660 con 0x7f0b940052e0 2024-05-09T06:37:02.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.739+0000 7f0b9effd640 1 -- 172.21.15.3:0/1530094896 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0ba4015600 con 0x7f0b940052e0 2024-05-09T06:37:02.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.739+0000 7f0b9effd640 1 --2- 172.21.15.3:0/1530094896 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f0b8c06be30 0x7f0b8c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:02.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.739+0000 7f0b9effd640 1 -- 172.21.15.3:0/1530094896 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f0ba408b980 con 0x7f0b940052e0 2024-05-09T06:37:02.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.739+0000 7f0bad8dd640 1 --2- 172.21.15.3:0/1530094896 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f0b8c06be30 0x7f0b8c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:02.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.740+0000 7f0bad8dd640 1 --2- 172.21.15.3:0/1530094896 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f0b8c06be30 0x7f0b8c06e2f0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f0ba815b110 tx=0x7f0b98009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:02.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.744+0000 7f0b9effd640 1 -- 172.21.15.3:0/1530094896 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f0ba4019030 con 0x7f0b940052e0 2024-05-09T06:37:02.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.923+0000 7f0bb0342640 1 -- 172.21.15.3:0/1530094896 --> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f0ba815dd00 con 0x7f0b8c06be30 2024-05-09T06:37:02.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.929+0000 7f0b9effd640 1 -- 172.21.15.3:0/1530094896 <== mgr.4105 v2:172.21.15.190:6832/975893321 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251880 (secure 0 0 0) 0x7f0ba815dd00 con 0x7f0b8c06be30 2024-05-09T06:37:02.912 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-09T06:37:02.913 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2024-05-09T06:37:02.915 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.932+0000 7f0bb0342640 1 -- 172.21.15.3:0/1530094896 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f0b8c06be30 msgr2=0x7f0b8c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:02.915 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.932+0000 7f0bb0342640 1 --2- 172.21.15.3:0/1530094896 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f0b8c06be30 0x7f0b8c06e2f0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f0ba815b110 tx=0x7f0b98009040 comp rx=0 tx=0).stop 2024-05-09T06:37:02.915 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.932+0000 7f0bb0342640 1 -- 172.21.15.3:0/1530094896 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0b940052e0 msgr2=0x7f0ba8146690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:02.915 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.932+0000 7f0bb0342640 1 --2- 172.21.15.3:0/1530094896 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0b940052e0 0x7f0ba8146690 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f0ba40116f0 tx=0x7f0ba4011bb0 comp rx=0 tx=0).stop 2024-05-09T06:37:02.915 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.932+0000 7f0bb0342640 1 -- 172.21.15.3:0/1530094896 shutdown_connections 2024-05-09T06:37:02.915 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.932+0000 7f0bb0342640 1 --2- 172.21.15.3:0/1530094896 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f0b8c06be30 0x7f0b8c06e2f0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:02.915 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.932+0000 7f0bb0342640 1 --2- 172.21.15.3:0/1530094896 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0ba8146db0 0x7f0ba8147230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:02.915 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.932+0000 7f0bb0342640 1 --2- 172.21.15.3:0/1530094896 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0ba80ccd90 0x7f0ba80c93c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:02.915 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.932+0000 7f0bb0342640 1 --2- 172.21.15.3:0/1530094896 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0b940052e0 0x7f0ba8146690 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:02.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.932+0000 7f0bb0342640 1 -- 172.21.15.3:0/1530094896 >> 172.21.15.3:0/1530094896 conn(0x7f0ba8072870 msgr2=0x7f0ba80b25d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:02.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.932+0000 7f0bb0342640 1 -- 172.21.15.3:0/1530094896 shutdown_connections 2024-05-09T06:37:02.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:02.932+0000 7f0bb0342640 1 -- 172.21.15.3:0/1530094896 wait complete. 2024-05-09T06:37:02.924 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":61,"stamp":"2024-05-09T06:37:02.788745+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":45,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220116,"kb_used_data":3932,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754604,"statfs":{"total":773094113280,"available":772868714496,"internally_reserved":0,"allocated":4026368,"data_stored":2270109,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":28,"apply_latency_ms":28,"commit_latency_ns":28000000,"apply_latency_ns":28000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001263"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T06:36:58.581052+0000","last_change":"2024-05-09T06:36:58.577302+0000","last_active":"2024-05-09T06:36:58.581052+0000","last_peered":"2024-05-09T06:36:58.581052+0000","last_clean":"2024-05-09T06:36:58.581052+0000","last_became_active":"2024-05-09T06:36:58.576999+0000","last_became_peered":"2024-05-09T06:36:58.576999+0000","last_unstale":"2024-05-09T06:36:58.581052+0000","last_undegraded":"2024-05-09T06:36:58.581052+0000","last_fullsized":"2024-05-09T06:36:58.581052+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_clean_scrub_stamp":"2024-05-09T06:36:57.547101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:47:37.035146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T06:36:58.586359+0000","last_change":"2024-05-09T06:36:58.573971+0000","last_active":"2024-05-09T06:36:58.586359+0000","last_peered":"2024-05-09T06:36:58.586359+0000","last_clean":"2024-05-09T06:36:58.586359+0000","last_became_active":"2024-05-09T06:36:58.573678+0000","last_became_peered":"2024-05-09T06:36:58.573678+0000","last_unstale":"2024-05-09T06:36:58.586359+0000","last_undegraded":"2024-05-09T06:36:58.586359+0000","last_fullsized":"2024-05-09T06:36:58.586359+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_clean_scrub_stamp":"2024-05-09T06:36:57.547101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:25:07.739454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T06:36:58.590807+0000","last_change":"2024-05-09T06:36:58.586251+0000","last_active":"2024-05-09T06:36:58.590807+0000","last_peered":"2024-05-09T06:36:58.590807+0000","last_clean":"2024-05-09T06:36:58.590807+0000","last_became_active":"2024-05-09T06:36:58.585982+0000","last_became_peered":"2024-05-09T06:36:58.585982+0000","last_unstale":"2024-05-09T06:36:58.590807+0000","last_undegraded":"2024-05-09T06:36:58.590807+0000","last_fullsized":"2024-05-09T06:36:58.590807+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_clean_scrub_stamp":"2024-05-09T06:36:57.547101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:16:33.467895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T06:36:58.585246+0000","last_change":"2024-05-09T06:36:58.579810+0000","last_active":"2024-05-09T06:36:58.585246+0000","last_peered":"2024-05-09T06:36:58.585246+0000","last_clean":"2024-05-09T06:36:58.585246+0000","last_became_active":"2024-05-09T06:36:58.579538+0000","last_became_peered":"2024-05-09T06:36:58.579538+0000","last_unstale":"2024-05-09T06:36:58.585246+0000","last_undegraded":"2024-05-09T06:36:58.585246+0000","last_fullsized":"2024-05-09T06:36:58.585246+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_clean_scrub_stamp":"2024-05-09T06:36:57.547101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:47:37.035146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T06:36:58.588201+0000","last_change":"2024-05-09T06:36:58.584786+0000","last_active":"2024-05-09T06:36:58.588201+0000","last_peered":"2024-05-09T06:36:58.588201+0000","last_clean":"2024-05-09T06:36:58.588201+0000","last_became_active":"2024-05-09T06:36:58.584545+0000","last_became_peered":"2024-05-09T06:36:58.584545+0000","last_unstale":"2024-05-09T06:36:58.588201+0000","last_undegraded":"2024-05-09T06:36:58.588201+0000","last_fullsized":"2024-05-09T06:36:58.588201+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_clean_scrub_stamp":"2024-05-09T06:36:57.547101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:45:01.358608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T06:36:58.586439+0000","last_change":"2024-05-09T06:36:58.577494+0000","last_active":"2024-05-09T06:36:58.586439+0000","last_peered":"2024-05-09T06:36:58.586439+0000","last_clean":"2024-05-09T06:36:58.586439+0000","last_became_active":"2024-05-09T06:36:58.577198+0000","last_became_peered":"2024-05-09T06:36:58.577198+0000","last_unstale":"2024-05-09T06:36:58.586439+0000","last_undegraded":"2024-05-09T06:36:58.586439+0000","last_fullsized":"2024-05-09T06:36:58.586439+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_clean_scrub_stamp":"2024-05-09T06:36:57.547101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:16:33.467895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T06:36:58.580892+0000","last_change":"2024-05-09T06:36:58.575409+0000","last_active":"2024-05-09T06:36:58.580892+0000","last_peered":"2024-05-09T06:36:58.580892+0000","last_clean":"2024-05-09T06:36:58.580892+0000","last_became_active":"2024-05-09T06:36:58.575049+0000","last_became_peered":"2024-05-09T06:36:58.575049+0000","last_unstale":"2024-05-09T06:36:58.580892+0000","last_undegraded":"2024-05-09T06:36:58.580892+0000","last_fullsized":"2024-05-09T06:36:58.580892+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_clean_scrub_stamp":"2024-05-09T06:36:57.547101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:17:59.484786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T06:36:58.580846+0000","last_change":"2024-05-09T06:36:58.576987+0000","last_active":"2024-05-09T06:36:58.580846+0000","last_peered":"2024-05-09T06:36:58.580846+0000","last_clean":"2024-05-09T06:36:58.580846+0000","last_became_active":"2024-05-09T06:36:58.576693+0000","last_became_peered":"2024-05-09T06:36:58.576693+0000","last_unstale":"2024-05-09T06:36:58.580846+0000","last_undegraded":"2024-05-09T06:36:58.580846+0000","last_fullsized":"2024-05-09T06:36:58.580846+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_clean_scrub_stamp":"2024-05-09T06:36:57.547101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:39:22.810622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T06:36:58.585592+0000","last_change":"2024-05-09T06:36:58.581421+0000","last_active":"2024-05-09T06:36:58.585592+0000","last_peered":"2024-05-09T06:36:58.585592+0000","last_clean":"2024-05-09T06:36:58.585592+0000","last_became_active":"2024-05-09T06:36:58.581028+0000","last_became_peered":"2024-05-09T06:36:58.581028+0000","last_unstale":"2024-05-09T06:36:58.585592+0000","last_undegraded":"2024-05-09T06:36:58.585592+0000","last_fullsized":"2024-05-09T06:36:58.585592+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_clean_scrub_stamp":"2024-05-09T06:36:57.547101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:03:45.503132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T06:36:58.593000+0000","last_change":"2024-05-09T06:36:58.589883+0000","last_active":"2024-05-09T06:36:58.593000+0000","last_peered":"2024-05-09T06:36:58.593000+0000","last_clean":"2024-05-09T06:36:58.593000+0000","last_became_active":"2024-05-09T06:36:58.589544+0000","last_became_peered":"2024-05-09T06:36:58.589544+0000","last_unstale":"2024-05-09T06:36:58.593000+0000","last_undegraded":"2024-05-09T06:36:58.593000+0000","last_fullsized":"2024-05-09T06:36:58.593000+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_clean_scrub_stamp":"2024-05-09T06:36:57.547101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:40:37.529479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T06:36:58.587999+0000","last_change":"2024-05-09T06:36:58.585279+0000","last_active":"2024-05-09T06:36:58.587999+0000","last_peered":"2024-05-09T06:36:58.587999+0000","last_clean":"2024-05-09T06:36:58.587999+0000","last_became_active":"2024-05-09T06:36:58.585002+0000","last_became_peered":"2024-05-09T06:36:58.585002+0000","last_unstale":"2024-05-09T06:36:58.587999+0000","last_undegraded":"2024-05-09T06:36:58.587999+0000","last_fullsized":"2024-05-09T06:36:58.587999+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_clean_scrub_stamp":"2024-05-09T06:36:57.547101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:35:21.215213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T06:36:58.577813+0000","last_change":"2024-05-09T06:36:58.573737+0000","last_active":"2024-05-09T06:36:58.577813+0000","last_peered":"2024-05-09T06:36:58.577813+0000","last_clean":"2024-05-09T06:36:58.577813+0000","last_became_active":"2024-05-09T06:36:58.573447+0000","last_became_peered":"2024-05-09T06:36:58.573447+0000","last_unstale":"2024-05-09T06:36:58.577813+0000","last_undegraded":"2024-05-09T06:36:58.577813+0000","last_fullsized":"2024-05-09T06:36:58.577813+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_clean_scrub_stamp":"2024-05-09T06:36:57.547101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:39:22.810622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T06:36:58.587570+0000","last_change":"2024-05-09T06:36:58.582303+0000","last_active":"2024-05-09T06:36:58.587570+0000","last_peered":"2024-05-09T06:36:58.587570+0000","last_clean":"2024-05-09T06:36:58.587570+0000","last_became_active":"2024-05-09T06:36:58.581882+0000","last_became_peered":"2024-05-09T06:36:58.581882+0000","last_unstale":"2024-05-09T06:36:58.587570+0000","last_undegraded":"2024-05-09T06:36:58.587570+0000","last_fullsized":"2024-05-09T06:36:58.587570+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_clean_scrub_stamp":"2024-05-09T06:36:57.547101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:08:22.332489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T06:36:58.574682+0000","last_change":"2024-05-09T06:36:58.566230+0000","last_active":"2024-05-09T06:36:58.574682+0000","last_peered":"2024-05-09T06:36:58.574682+0000","last_clean":"2024-05-09T06:36:58.574682+0000","last_became_active":"2024-05-09T06:36:58.565926+0000","last_became_peered":"2024-05-09T06:36:58.565926+0000","last_unstale":"2024-05-09T06:36:58.574682+0000","last_undegraded":"2024-05-09T06:36:58.574682+0000","last_fullsized":"2024-05-09T06:36:58.574682+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_clean_scrub_stamp":"2024-05-09T06:36:57.547101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:35:21.215213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T06:36:58.580038+0000","last_change":"2024-05-09T06:36:58.575650+0000","last_active":"2024-05-09T06:36:58.580038+0000","last_peered":"2024-05-09T06:36:58.580038+0000","last_clean":"2024-05-09T06:36:58.580038+0000","last_became_active":"2024-05-09T06:36:58.574909+0000","last_became_peered":"2024-05-09T06:36:58.574909+0000","last_unstale":"2024-05-09T06:36:58.580038+0000","last_undegraded":"2024-05-09T06:36:58.580038+0000","last_fullsized":"2024-05-09T06:36:58.580038+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_clean_scrub_stamp":"2024-05-09T06:36:57.547101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:50:46.567561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T06:36:58.593322+0000","last_change":"2024-05-09T06:36:58.588645+0000","last_active":"2024-05-09T06:36:58.593322+0000","last_peered":"2024-05-09T06:36:58.593322+0000","last_clean":"2024-05-09T06:36:58.593322+0000","last_became_active":"2024-05-09T06:36:58.588385+0000","last_became_peered":"2024-05-09T06:36:58.588385+0000","last_unstale":"2024-05-09T06:36:58.593322+0000","last_undegraded":"2024-05-09T06:36:58.593322+0000","last_fullsized":"2024-05-09T06:36:58.593322+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_clean_scrub_stamp":"2024-05-09T06:36:57.547101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:40:37.529479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T06:36:58.580453+0000","last_change":"2024-05-09T06:36:58.576747+0000","last_active":"2024-05-09T06:36:58.580453+0000","last_peered":"2024-05-09T06:36:58.580453+0000","last_clean":"2024-05-09T06:36:58.580453+0000","last_became_active":"2024-05-09T06:36:58.576462+0000","last_became_peered":"2024-05-09T06:36:58.576462+0000","last_unstale":"2024-05-09T06:36:58.580453+0000","last_undegraded":"2024-05-09T06:36:58.580453+0000","last_fullsized":"2024-05-09T06:36:58.580453+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_clean_scrub_stamp":"2024-05-09T06:36:57.547101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:02:32.394179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T06:36:58.587852+0000","last_change":"2024-05-09T06:36:58.581757+0000","last_active":"2024-05-09T06:36:58.587852+0000","last_peered":"2024-05-09T06:36:58.587852+0000","last_clean":"2024-05-09T06:36:58.587852+0000","last_became_active":"2024-05-09T06:36:58.581443+0000","last_became_peered":"2024-05-09T06:36:58.581443+0000","last_unstale":"2024-05-09T06:36:58.587852+0000","last_undegraded":"2024-05-09T06:36:58.587852+0000","last_fullsized":"2024-05-09T06:36:58.587852+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_clean_scrub_stamp":"2024-05-09T06:36:57.547101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:49:32.371985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T06:36:58.588384+0000","last_change":"2024-05-09T06:36:58.586844+0000","last_active":"2024-05-09T06:36:58.588384+0000","last_peered":"2024-05-09T06:36:58.588384+0000","last_clean":"2024-05-09T06:36:58.588384+0000","last_became_active":"2024-05-09T06:36:58.586560+0000","last_became_peered":"2024-05-09T06:36:58.586560+0000","last_unstale":"2024-05-09T06:36:58.588384+0000","last_undegraded":"2024-05-09T06:36:58.588384+0000","last_fullsized":"2024-05-09T06:36:58.588384+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_clean_scrub_stamp":"2024-05-09T06:36:57.547101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:52:57.675729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T06:36:58.584079+0000","last_change":"2024-05-09T06:36:58.580532+0000","last_active":"2024-05-09T06:36:58.584079+0000","last_peered":"2024-05-09T06:36:58.584079+0000","last_clean":"2024-05-09T06:36:58.584079+0000","last_became_active":"2024-05-09T06:36:58.580241+0000","last_became_peered":"2024-05-09T06:36:58.580241+0000","last_unstale":"2024-05-09T06:36:58.584079+0000","last_undegraded":"2024-05-09T06:36:58.584079+0000","last_fullsized":"2024-05-09T06:36:58.584079+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_clean_scrub_stamp":"2024-05-09T06:36:57.547101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:18:06.560834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T06:36:58.581363+0000","last_change":"2024-05-09T06:36:58.576651+0000","last_active":"2024-05-09T06:36:58.581363+0000","last_peered":"2024-05-09T06:36:58.581363+0000","last_clean":"2024-05-09T06:36:58.581363+0000","last_became_active":"2024-05-09T06:36:58.576273+0000","last_became_peered":"2024-05-09T06:36:58.576273+0000","last_unstale":"2024-05-09T06:36:58.581363+0000","last_undegraded":"2024-05-09T06:36:58.581363+0000","last_fullsized":"2024-05-09T06:36:58.581363+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_clean_scrub_stamp":"2024-05-09T06:36:57.547101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:59:53.598004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T06:36:58.567567+0000","last_change":"2024-05-09T06:36:58.564892+0000","last_active":"2024-05-09T06:36:58.567567+0000","last_peered":"2024-05-09T06:36:58.567567+0000","last_clean":"2024-05-09T06:36:58.567567+0000","last_became_active":"2024-05-09T06:36:58.564595+0000","last_became_peered":"2024-05-09T06:36:58.564595+0000","last_unstale":"2024-05-09T06:36:58.567567+0000","last_undegraded":"2024-05-09T06:36:58.567567+0000","last_fullsized":"2024-05-09T06:36:58.567567+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_clean_scrub_stamp":"2024-05-09T06:36:57.547101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:49:21.146962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T06:36:58.580235+0000","last_change":"2024-05-09T06:36:58.575688+0000","last_active":"2024-05-09T06:36:58.580235+0000","last_peered":"2024-05-09T06:36:58.580235+0000","last_clean":"2024-05-09T06:36:58.580235+0000","last_became_active":"2024-05-09T06:36:58.575397+0000","last_became_peered":"2024-05-09T06:36:58.575397+0000","last_unstale":"2024-05-09T06:36:58.580235+0000","last_undegraded":"2024-05-09T06:36:58.580235+0000","last_fullsized":"2024-05-09T06:36:58.580235+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_clean_scrub_stamp":"2024-05-09T06:36:57.547101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:02:32.394179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T06:36:58.577902+0000","last_change":"2024-05-09T06:36:58.574804+0000","last_active":"2024-05-09T06:36:58.577902+0000","last_peered":"2024-05-09T06:36:58.577902+0000","last_clean":"2024-05-09T06:36:58.577902+0000","last_became_active":"2024-05-09T06:36:58.574535+0000","last_became_peered":"2024-05-09T06:36:58.574535+0000","last_unstale":"2024-05-09T06:36:58.577902+0000","last_undegraded":"2024-05-09T06:36:58.577902+0000","last_fullsized":"2024-05-09T06:36:58.577902+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_clean_scrub_stamp":"2024-05-09T06:36:57.547101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:14:46.331201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T06:36:58.585486+0000","last_change":"2024-05-09T06:36:58.579820+0000","last_active":"2024-05-09T06:36:58.585486+0000","last_peered":"2024-05-09T06:36:58.585486+0000","last_clean":"2024-05-09T06:36:58.585486+0000","last_became_active":"2024-05-09T06:36:58.579582+0000","last_became_peered":"2024-05-09T06:36:58.579582+0000","last_unstale":"2024-05-09T06:36:58.585486+0000","last_undegraded":"2024-05-09T06:36:58.585486+0000","last_fullsized":"2024-05-09T06:36:58.585486+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_clean_scrub_stamp":"2024-05-09T06:36:57.547101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:02:32.394179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T06:36:58.587931+0000","last_change":"2024-05-09T06:36:58.582893+0000","last_active":"2024-05-09T06:36:58.587931+0000","last_peered":"2024-05-09T06:36:58.587931+0000","last_clean":"2024-05-09T06:36:58.587931+0000","last_became_active":"2024-05-09T06:36:58.582612+0000","last_became_peered":"2024-05-09T06:36:58.582612+0000","last_unstale":"2024-05-09T06:36:58.587931+0000","last_undegraded":"2024-05-09T06:36:58.587931+0000","last_fullsized":"2024-05-09T06:36:58.587931+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_clean_scrub_stamp":"2024-05-09T06:36:57.547101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:29:21.168673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T06:36:58.588056+0000","last_change":"2024-05-09T06:36:58.584681+0000","last_active":"2024-05-09T06:36:58.588056+0000","last_peered":"2024-05-09T06:36:58.588056+0000","last_clean":"2024-05-09T06:36:58.588056+0000","last_became_active":"2024-05-09T06:36:58.584252+0000","last_became_peered":"2024-05-09T06:36:58.584252+0000","last_unstale":"2024-05-09T06:36:58.588056+0000","last_undegraded":"2024-05-09T06:36:58.588056+0000","last_fullsized":"2024-05-09T06:36:58.588056+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_clean_scrub_stamp":"2024-05-09T06:36:57.547101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:50:46.567561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T06:36:58.579784+0000","last_change":"2024-05-09T06:36:58.576434+0000","last_active":"2024-05-09T06:36:58.579784+0000","last_peered":"2024-05-09T06:36:58.579784+0000","last_clean":"2024-05-09T06:36:58.579784+0000","last_became_active":"2024-05-09T06:36:58.575957+0000","last_became_peered":"2024-05-09T06:36:58.575957+0000","last_unstale":"2024-05-09T06:36:58.579784+0000","last_undegraded":"2024-05-09T06:36:58.579784+0000","last_fullsized":"2024-05-09T06:36:58.579784+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_clean_scrub_stamp":"2024-05-09T06:36:57.547101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:35:21.215213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T06:36:58.578371+0000","last_change":"2024-05-09T06:36:58.567313+0000","last_active":"2024-05-09T06:36:58.578371+0000","last_peered":"2024-05-09T06:36:58.578371+0000","last_clean":"2024-05-09T06:36:58.578371+0000","last_became_active":"2024-05-09T06:36:58.567039+0000","last_became_peered":"2024-05-09T06:36:58.567039+0000","last_unstale":"2024-05-09T06:36:58.578371+0000","last_undegraded":"2024-05-09T06:36:58.578371+0000","last_fullsized":"2024-05-09T06:36:58.578371+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_clean_scrub_stamp":"2024-05-09T06:36:57.547101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:54:47.768120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T06:36:58.580426+0000","last_change":"2024-05-09T06:36:58.576038+0000","last_active":"2024-05-09T06:36:58.580426+0000","last_peered":"2024-05-09T06:36:58.580426+0000","last_clean":"2024-05-09T06:36:58.580426+0000","last_became_active":"2024-05-09T06:36:58.575771+0000","last_became_peered":"2024-05-09T06:36:58.575771+0000","last_unstale":"2024-05-09T06:36:58.580426+0000","last_undegraded":"2024-05-09T06:36:58.580426+0000","last_fullsized":"2024-05-09T06:36:58.580426+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_clean_scrub_stamp":"2024-05-09T06:36:57.547101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:47:37.035146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T06:36:58.580632+0000","last_change":"2024-05-09T06:36:58.574578+0000","last_active":"2024-05-09T06:36:58.580632+0000","last_peered":"2024-05-09T06:36:58.580632+0000","last_clean":"2024-05-09T06:36:58.580632+0000","last_became_active":"2024-05-09T06:36:58.574241+0000","last_became_peered":"2024-05-09T06:36:58.574241+0000","last_unstale":"2024-05-09T06:36:58.580632+0000","last_undegraded":"2024-05-09T06:36:58.580632+0000","last_fullsized":"2024-05-09T06:36:58.580632+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_clean_scrub_stamp":"2024-05-09T06:36:57.547101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:17:59.484786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T06:36:58.585461+0000","last_change":"2024-05-09T06:36:58.581463+0000","last_active":"2024-05-09T06:36:58.585461+0000","last_peered":"2024-05-09T06:36:58.585461+0000","last_clean":"2024-05-09T06:36:58.585461+0000","last_became_active":"2024-05-09T06:36:58.581171+0000","last_became_peered":"2024-05-09T06:36:58.581171+0000","last_unstale":"2024-05-09T06:36:58.585461+0000","last_undegraded":"2024-05-09T06:36:58.585461+0000","last_fullsized":"2024-05-09T06:36:58.585461+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_clean_scrub_stamp":"2024-05-09T06:36:57.547101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:52:21.177837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T06:36:58.580302+0000","last_change":"2024-05-09T06:36:58.575579+0000","last_active":"2024-05-09T06:36:58.580302+0000","last_peered":"2024-05-09T06:36:58.580302+0000","last_clean":"2024-05-09T06:36:58.580302+0000","last_became_active":"2024-05-09T06:36:58.575290+0000","last_became_peered":"2024-05-09T06:36:58.575290+0000","last_unstale":"2024-05-09T06:36:58.580302+0000","last_undegraded":"2024-05-09T06:36:58.580302+0000","last_fullsized":"2024-05-09T06:36:58.580302+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_clean_scrub_stamp":"2024-05-09T06:36:57.547101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:47:37.035146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"32'4","reported_seq":16,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T06:36:58.582987+0000","last_change":"2024-05-09T06:36:57.573551+0000","last_active":"2024-05-09T06:36:58.582987+0000","last_peered":"2024-05-09T06:36:58.582987+0000","last_clean":"2024-05-09T06:36:58.582987+0000","last_became_active":"2024-05-09T06:36:57.573003+0000","last_became_peered":"2024-05-09T06:36:57.573003+0000","last_unstale":"2024-05-09T06:36:58.582987+0000","last_undegraded":"2024-05-09T06:36:58.582987+0000","last_fullsized":"2024-05-09T06:36:58.582987+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:56.544873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:56.544873+0000","last_clean_scrub_stamp":"2024-05-09T06:36:56.544873+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:59:52.595776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T06:36:58.573623+0000","last_change":"2024-05-09T06:36:58.571321+0000","last_active":"2024-05-09T06:36:58.573623+0000","last_peered":"2024-05-09T06:36:58.573623+0000","last_clean":"2024-05-09T06:36:58.573623+0000","last_became_active":"2024-05-09T06:36:58.570982+0000","last_became_peered":"2024-05-09T06:36:58.570982+0000","last_unstale":"2024-05-09T06:36:58.573623+0000","last_undegraded":"2024-05-09T06:36:58.573623+0000","last_fullsized":"2024-05-09T06:36:58.573623+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_clean_scrub_stamp":"2024-05-09T06:36:57.547101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:03:45.503132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"32'1","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T06:36:58.578002+0000","last_change":"2024-05-09T06:36:57.562690+0000","last_active":"2024-05-09T06:36:58.578002+0000","last_peered":"2024-05-09T06:36:58.578002+0000","last_clean":"2024-05-09T06:36:58.578002+0000","last_became_active":"2024-05-09T06:36:57.562354+0000","last_became_peered":"2024-05-09T06:36:57.562354+0000","last_unstale":"2024-05-09T06:36:58.578002+0000","last_undegraded":"2024-05-09T06:36:58.578002+0000","last_fullsized":"2024-05-09T06:36:58.578002+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:56.544873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:56.544873+0000","last_clean_scrub_stamp":"2024-05-09T06:36:56.544873+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:59:52.595776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T06:36:58.572625+0000","last_change":"2024-05-09T06:36:58.566855+0000","last_active":"2024-05-09T06:36:58.572625+0000","last_peered":"2024-05-09T06:36:58.572625+0000","last_clean":"2024-05-09T06:36:58.572625+0000","last_became_active":"2024-05-09T06:36:58.566571+0000","last_became_peered":"2024-05-09T06:36:58.566571+0000","last_unstale":"2024-05-09T06:36:58.572625+0000","last_undegraded":"2024-05-09T06:36:58.572625+0000","last_fullsized":"2024-05-09T06:36:58.572625+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_clean_scrub_stamp":"2024-05-09T06:36:57.547101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:11:08.902382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"32'2","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T06:36:58.588675+0000","last_change":"2024-05-09T06:36:57.579970+0000","last_active":"2024-05-09T06:36:58.588675+0000","last_peered":"2024-05-09T06:36:58.588675+0000","last_clean":"2024-05-09T06:36:58.588675+0000","last_became_active":"2024-05-09T06:36:57.579660+0000","last_became_peered":"2024-05-09T06:36:57.579660+0000","last_unstale":"2024-05-09T06:36:58.588675+0000","last_undegraded":"2024-05-09T06:36:58.588675+0000","last_fullsized":"2024-05-09T06:36:58.588675+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:56.544873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:56.544873+0000","last_clean_scrub_stamp":"2024-05-09T06:36:56.544873+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:14:45.328973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T06:36:58.587578+0000","last_change":"2024-05-09T06:36:58.583966+0000","last_active":"2024-05-09T06:36:58.587578+0000","last_peered":"2024-05-09T06:36:58.587578+0000","last_clean":"2024-05-09T06:36:58.587578+0000","last_became_active":"2024-05-09T06:36:58.583393+0000","last_became_peered":"2024-05-09T06:36:58.583393+0000","last_unstale":"2024-05-09T06:36:58.587578+0000","last_undegraded":"2024-05-09T06:36:58.587578+0000","last_fullsized":"2024-05-09T06:36:58.587578+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_clean_scrub_stamp":"2024-05-09T06:36:57.547101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:52:21.177837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"32'2","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T06:36:58.575278+0000","last_change":"2024-05-09T06:36:57.576761+0000","last_active":"2024-05-09T06:36:58.575278+0000","last_peered":"2024-05-09T06:36:58.575278+0000","last_clean":"2024-05-09T06:36:58.575278+0000","last_became_active":"2024-05-09T06:36:57.576487+0000","last_became_peered":"2024-05-09T06:36:57.576487+0000","last_unstale":"2024-05-09T06:36:58.575278+0000","last_undegraded":"2024-05-09T06:36:58.575278+0000","last_fullsized":"2024-05-09T06:36:58.575278+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:56.544873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:56.544873+0000","last_clean_scrub_stamp":"2024-05-09T06:36:56.544873+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:25:12.896515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T06:36:58.585455+0000","last_change":"2024-05-09T06:36:58.580707+0000","last_active":"2024-05-09T06:36:58.585455+0000","last_peered":"2024-05-09T06:36:58.585455+0000","last_clean":"2024-05-09T06:36:58.585455+0000","last_became_active":"2024-05-09T06:36:58.580283+0000","last_became_peered":"2024-05-09T06:36:58.580283+0000","last_unstale":"2024-05-09T06:36:58.585455+0000","last_undegraded":"2024-05-09T06:36:58.585455+0000","last_fullsized":"2024-05-09T06:36:58.585455+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_clean_scrub_stamp":"2024-05-09T06:36:57.547101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:14:46.331201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"32'1","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T06:36:58.588103+0000","last_change":"2024-05-09T06:36:57.572231+0000","last_active":"2024-05-09T06:36:58.588103+0000","last_peered":"2024-05-09T06:36:58.588103+0000","last_clean":"2024-05-09T06:36:58.588103+0000","last_became_active":"2024-05-09T06:36:57.571937+0000","last_became_peered":"2024-05-09T06:36:57.571937+0000","last_unstale":"2024-05-09T06:36:58.588103+0000","last_undegraded":"2024-05-09T06:36:58.588103+0000","last_fullsized":"2024-05-09T06:36:58.588103+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:56.544873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:56.544873+0000","last_clean_scrub_stamp":"2024-05-09T06:36:56.544873+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:44:53.136881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T06:36:58.579731+0000","last_change":"2024-05-09T06:36:58.575508+0000","last_active":"2024-05-09T06:36:58.579731+0000","last_peered":"2024-05-09T06:36:58.579731+0000","last_clean":"2024-05-09T06:36:58.579731+0000","last_became_active":"2024-05-09T06:36:58.575256+0000","last_became_peered":"2024-05-09T06:36:58.575256+0000","last_unstale":"2024-05-09T06:36:58.579731+0000","last_undegraded":"2024-05-09T06:36:58.579731+0000","last_fullsized":"2024-05-09T06:36:58.579731+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_clean_scrub_stamp":"2024-05-09T06:36:57.547101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:52:21.177837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"32'1","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T06:36:58.588064+0000","last_change":"2024-05-09T06:36:57.572722+0000","last_active":"2024-05-09T06:36:58.588064+0000","last_peered":"2024-05-09T06:36:58.588064+0000","last_clean":"2024-05-09T06:36:58.588064+0000","last_became_active":"2024-05-09T06:36:57.572242+0000","last_became_peered":"2024-05-09T06:36:57.572242+0000","last_unstale":"2024-05-09T06:36:58.588064+0000","last_undegraded":"2024-05-09T06:36:58.588064+0000","last_fullsized":"2024-05-09T06:36:58.588064+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:56.544873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:56.544873+0000","last_clean_scrub_stamp":"2024-05-09T06:36:56.544873+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:31:45.233338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T06:36:58.578328+0000","last_change":"2024-05-09T06:36:58.568074+0000","last_active":"2024-05-09T06:36:58.578328+0000","last_peered":"2024-05-09T06:36:58.578328+0000","last_clean":"2024-05-09T06:36:58.578328+0000","last_became_active":"2024-05-09T06:36:58.567787+0000","last_became_peered":"2024-05-09T06:36:58.567787+0000","last_unstale":"2024-05-09T06:36:58.578328+0000","last_undegraded":"2024-05-09T06:36:58.578328+0000","last_fullsized":"2024-05-09T06:36:58.578328+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_clean_scrub_stamp":"2024-05-09T06:36:57.547101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:05:38.369597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T06:36:58.558272+0000","last_change":"2024-05-09T06:36:57.577513+0000","last_active":"2024-05-09T06:36:58.558272+0000","last_peered":"2024-05-09T06:36:58.558272+0000","last_clean":"2024-05-09T06:36:58.558272+0000","last_became_active":"2024-05-09T06:36:57.577217+0000","last_became_peered":"2024-05-09T06:36:57.577217+0000","last_unstale":"2024-05-09T06:36:58.558272+0000","last_undegraded":"2024-05-09T06:36:58.558272+0000","last_fullsized":"2024-05-09T06:36:58.558272+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:56.544873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:56.544873+0000","last_clean_scrub_stamp":"2024-05-09T06:36:56.544873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:02:31.391951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T06:36:58.585339+0000","last_change":"2024-05-09T06:36:58.582556+0000","last_active":"2024-05-09T06:36:58.585339+0000","last_peered":"2024-05-09T06:36:58.585339+0000","last_clean":"2024-05-09T06:36:58.585339+0000","last_became_active":"2024-05-09T06:36:58.582256+0000","last_became_peered":"2024-05-09T06:36:58.582256+0000","last_unstale":"2024-05-09T06:36:58.585339+0000","last_undegraded":"2024-05-09T06:36:58.585339+0000","last_fullsized":"2024-05-09T06:36:58.585339+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_clean_scrub_stamp":"2024-05-09T06:36:57.547101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:39:22.810622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T06:36:58.558678+0000","last_change":"2024-05-09T06:36:57.571350+0000","last_active":"2024-05-09T06:36:58.558678+0000","last_peered":"2024-05-09T06:36:58.558678+0000","last_clean":"2024-05-09T06:36:58.558678+0000","last_became_active":"2024-05-09T06:36:57.570918+0000","last_became_peered":"2024-05-09T06:36:57.570918+0000","last_unstale":"2024-05-09T06:36:58.558678+0000","last_undegraded":"2024-05-09T06:36:58.558678+0000","last_fullsized":"2024-05-09T06:36:58.558678+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:56.544873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:56.544873+0000","last_clean_scrub_stamp":"2024-05-09T06:36:56.544873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:18:05.558606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T06:36:58.575869+0000","last_change":"2024-05-09T06:36:58.570138+0000","last_active":"2024-05-09T06:36:58.575869+0000","last_peered":"2024-05-09T06:36:58.575869+0000","last_clean":"2024-05-09T06:36:58.575869+0000","last_became_active":"2024-05-09T06:36:58.569862+0000","last_became_peered":"2024-05-09T06:36:58.569862+0000","last_unstale":"2024-05-09T06:36:58.575869+0000","last_undegraded":"2024-05-09T06:36:58.575869+0000","last_fullsized":"2024-05-09T06:36:58.575869+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_clean_scrub_stamp":"2024-05-09T06:36:57.547101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:02:32.394179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T06:36:58.558490+0000","last_change":"2024-05-09T06:36:57.576687+0000","last_active":"2024-05-09T06:36:58.558490+0000","last_peered":"2024-05-09T06:36:58.558490+0000","last_clean":"2024-05-09T06:36:58.558490+0000","last_became_active":"2024-05-09T06:36:57.576390+0000","last_became_peered":"2024-05-09T06:36:57.576390+0000","last_unstale":"2024-05-09T06:36:58.558490+0000","last_undegraded":"2024-05-09T06:36:58.558490+0000","last_fullsized":"2024-05-09T06:36:58.558490+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:56.544873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:56.544873+0000","last_clean_scrub_stamp":"2024-05-09T06:36:56.544873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:39:21.808394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T06:36:58.588002+0000","last_change":"2024-05-09T06:36:58.583652+0000","last_active":"2024-05-09T06:36:58.588002+0000","last_peered":"2024-05-09T06:36:58.588002+0000","last_clean":"2024-05-09T06:36:58.588002+0000","last_became_active":"2024-05-09T06:36:58.583365+0000","last_became_peered":"2024-05-09T06:36:58.583365+0000","last_unstale":"2024-05-09T06:36:58.588002+0000","last_undegraded":"2024-05-09T06:36:58.588002+0000","last_fullsized":"2024-05-09T06:36:58.588002+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_clean_scrub_stamp":"2024-05-09T06:36:57.547101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:30:41.385698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T06:36:58.570107+0000","last_change":"2024-05-09T06:36:57.579767+0000","last_active":"2024-05-09T06:36:58.570107+0000","last_peered":"2024-05-09T06:36:58.570107+0000","last_clean":"2024-05-09T06:36:58.570107+0000","last_became_active":"2024-05-09T06:36:57.579451+0000","last_became_peered":"2024-05-09T06:36:57.579451+0000","last_unstale":"2024-05-09T06:36:58.570107+0000","last_undegraded":"2024-05-09T06:36:58.570107+0000","last_fullsized":"2024-05-09T06:36:58.570107+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:56.544873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:56.544873+0000","last_clean_scrub_stamp":"2024-05-09T06:36:56.544873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:03:44.500904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T06:36:58.588128+0000","last_change":"2024-05-09T06:36:58.585258+0000","last_active":"2024-05-09T06:36:58.588128+0000","last_peered":"2024-05-09T06:36:58.588128+0000","last_clean":"2024-05-09T06:36:58.588128+0000","last_became_active":"2024-05-09T06:36:58.584564+0000","last_became_peered":"2024-05-09T06:36:58.584564+0000","last_unstale":"2024-05-09T06:36:58.588128+0000","last_undegraded":"2024-05-09T06:36:58.588128+0000","last_fullsized":"2024-05-09T06:36:58.588128+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_clean_scrub_stamp":"2024-05-09T06:36:57.547101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:00:45.954062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"32'3","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T06:36:58.581840+0000","last_change":"2024-05-09T06:36:57.575448+0000","last_active":"2024-05-09T06:36:58.581840+0000","last_peered":"2024-05-09T06:36:58.581840+0000","last_clean":"2024-05-09T06:36:58.581840+0000","last_became_active":"2024-05-09T06:36:57.575191+0000","last_became_peered":"2024-05-09T06:36:57.575191+0000","last_unstale":"2024-05-09T06:36:58.581840+0000","last_undegraded":"2024-05-09T06:36:58.581840+0000","last_fullsized":"2024-05-09T06:36:58.581840+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:56.544873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:56.544873+0000","last_clean_scrub_stamp":"2024-05-09T06:36:56.544873+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:17:58.482558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T06:36:58.590831+0000","last_change":"2024-05-09T06:36:58.586260+0000","last_active":"2024-05-09T06:36:58.590831+0000","last_peered":"2024-05-09T06:36:58.590831+0000","last_clean":"2024-05-09T06:36:58.590831+0000","last_became_active":"2024-05-09T06:36:58.586003+0000","last_became_peered":"2024-05-09T06:36:58.586003+0000","last_unstale":"2024-05-09T06:36:58.590831+0000","last_undegraded":"2024-05-09T06:36:58.590831+0000","last_fullsized":"2024-05-09T06:36:58.590831+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_clean_scrub_stamp":"2024-05-09T06:36:57.547101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:49:32.371985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"32'6","reported_seq":18,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T06:36:58.595551+0000","last_change":"2024-05-09T06:36:57.579927+0000","last_active":"2024-05-09T06:36:58.595551+0000","last_peered":"2024-05-09T06:36:58.595551+0000","last_clean":"2024-05-09T06:36:58.595551+0000","last_became_active":"2024-05-09T06:36:57.579612+0000","last_became_peered":"2024-05-09T06:36:57.579612+0000","last_unstale":"2024-05-09T06:36:58.595551+0000","last_undegraded":"2024-05-09T06:36:58.595551+0000","last_fullsized":"2024-05-09T06:36:58.595551+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:56.544873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:56.544873+0000","last_clean_scrub_stamp":"2024-05-09T06:36:56.544873+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:50:45.565333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T06:36:58.585540+0000","last_change":"2024-05-09T06:36:58.582611+0000","last_active":"2024-05-09T06:36:58.585540+0000","last_peered":"2024-05-09T06:36:58.585540+0000","last_clean":"2024-05-09T06:36:58.585540+0000","last_became_active":"2024-05-09T06:36:58.582331+0000","last_became_peered":"2024-05-09T06:36:58.582331+0000","last_unstale":"2024-05-09T06:36:58.585540+0000","last_undegraded":"2024-05-09T06:36:58.585540+0000","last_fullsized":"2024-05-09T06:36:58.585540+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_clean_scrub_stamp":"2024-05-09T06:36:57.547101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:14:46.331201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"32'2","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T06:36:58.580836+0000","last_change":"2024-05-09T06:36:57.573842+0000","last_active":"2024-05-09T06:36:58.580836+0000","last_peered":"2024-05-09T06:36:58.580836+0000","last_clean":"2024-05-09T06:36:58.580836+0000","last_became_active":"2024-05-09T06:36:57.573569+0000","last_became_peered":"2024-05-09T06:36:57.573569+0000","last_unstale":"2024-05-09T06:36:58.580836+0000","last_undegraded":"2024-05-09T06:36:58.580836+0000","last_fullsized":"2024-05-09T06:36:58.580836+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:56.544873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:56.544873+0000","last_clean_scrub_stamp":"2024-05-09T06:36:56.544873+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:59:17.959203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T06:36:58.578741+0000","last_change":"2024-05-09T06:36:58.573239+0000","last_active":"2024-05-09T06:36:58.578741+0000","last_peered":"2024-05-09T06:36:58.578741+0000","last_clean":"2024-05-09T06:36:58.578741+0000","last_became_active":"2024-05-09T06:36:58.572737+0000","last_became_peered":"2024-05-09T06:36:58.572737+0000","last_unstale":"2024-05-09T06:36:58.578741+0000","last_undegraded":"2024-05-09T06:36:58.578741+0000","last_fullsized":"2024-05-09T06:36:58.578741+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_clean_scrub_stamp":"2024-05-09T06:36:57.547101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:40:37.529479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T06:36:58.570497+0000","last_change":"2024-05-09T06:36:57.568508+0000","last_active":"2024-05-09T06:36:58.570497+0000","last_peered":"2024-05-09T06:36:58.570497+0000","last_clean":"2024-05-09T06:36:58.570497+0000","last_became_active":"2024-05-09T06:36:57.568191+0000","last_became_peered":"2024-05-09T06:36:57.568191+0000","last_unstale":"2024-05-09T06:36:58.570497+0000","last_undegraded":"2024-05-09T06:36:58.570497+0000","last_fullsized":"2024-05-09T06:36:58.570497+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:56.544873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:56.544873+0000","last_clean_scrub_stamp":"2024-05-09T06:36:56.544873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:59:52.595776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T06:36:58.585479+0000","last_change":"2024-05-09T06:36:58.582085+0000","last_active":"2024-05-09T06:36:58.585479+0000","last_peered":"2024-05-09T06:36:58.585479+0000","last_clean":"2024-05-09T06:36:58.585479+0000","last_became_active":"2024-05-09T06:36:58.581809+0000","last_became_peered":"2024-05-09T06:36:58.581809+0000","last_unstale":"2024-05-09T06:36:58.585479+0000","last_undegraded":"2024-05-09T06:36:58.585479+0000","last_fullsized":"2024-05-09T06:36:58.585479+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_clean_scrub_stamp":"2024-05-09T06:36:57.547101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:50:46.567561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T06:36:58.557958+0000","last_change":"2024-05-09T06:36:57.575641+0000","last_active":"2024-05-09T06:36:58.557958+0000","last_peered":"2024-05-09T06:36:58.557958+0000","last_clean":"2024-05-09T06:36:58.557958+0000","last_became_active":"2024-05-09T06:36:57.575216+0000","last_became_peered":"2024-05-09T06:36:57.575216+0000","last_unstale":"2024-05-09T06:36:58.557958+0000","last_undegraded":"2024-05-09T06:36:58.557958+0000","last_fullsized":"2024-05-09T06:36:58.557958+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:56.544873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:56.544873+0000","last_clean_scrub_stamp":"2024-05-09T06:36:56.544873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:17:58.482558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T06:36:58.574432+0000","last_change":"2024-05-09T06:36:58.569346+0000","last_active":"2024-05-09T06:36:58.574432+0000","last_peered":"2024-05-09T06:36:58.574432+0000","last_clean":"2024-05-09T06:36:58.574432+0000","last_became_active":"2024-05-09T06:36:58.568493+0000","last_became_peered":"2024-05-09T06:36:58.568493+0000","last_unstale":"2024-05-09T06:36:58.574432+0000","last_undegraded":"2024-05-09T06:36:58.574432+0000","last_fullsized":"2024-05-09T06:36:58.574432+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_clean_scrub_stamp":"2024-05-09T06:36:57.547101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:02:41.385043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"32'8","reported_seq":20,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T06:36:58.588996+0000","last_change":"2024-05-09T06:36:57.580235+0000","last_active":"2024-05-09T06:36:58.588996+0000","last_peered":"2024-05-09T06:36:58.588996+0000","last_clean":"2024-05-09T06:36:58.588996+0000","last_became_active":"2024-05-09T06:36:57.579644+0000","last_became_peered":"2024-05-09T06:36:57.579644+0000","last_unstale":"2024-05-09T06:36:58.588996+0000","last_undegraded":"2024-05-09T06:36:58.588996+0000","last_fullsized":"2024-05-09T06:36:58.588996+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:56.544873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:56.544873+0000","last_clean_scrub_stamp":"2024-05-09T06:36:56.544873+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:52:20.175609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T06:36:58.580785+0000","last_change":"2024-05-09T06:36:58.577792+0000","last_active":"2024-05-09T06:36:58.580785+0000","last_peered":"2024-05-09T06:36:58.580785+0000","last_clean":"2024-05-09T06:36:58.580785+0000","last_became_active":"2024-05-09T06:36:58.577443+0000","last_became_peered":"2024-05-09T06:36:58.577443+0000","last_unstale":"2024-05-09T06:36:58.580785+0000","last_undegraded":"2024-05-09T06:36:58.580785+0000","last_fullsized":"2024-05-09T06:36:58.580785+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_clean_scrub_stamp":"2024-05-09T06:36:57.547101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:02:41.385043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"32'1","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T06:36:58.589522+0000","last_change":"2024-05-09T06:36:57.575115+0000","last_active":"2024-05-09T06:36:58.589522+0000","last_peered":"2024-05-09T06:36:58.589522+0000","last_clean":"2024-05-09T06:36:58.589522+0000","last_became_active":"2024-05-09T06:36:57.574770+0000","last_became_peered":"2024-05-09T06:36:57.574770+0000","last_unstale":"2024-05-09T06:36:58.589522+0000","last_undegraded":"2024-05-09T06:36:58.589522+0000","last_fullsized":"2024-05-09T06:36:58.589522+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:56.544873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:56.544873+0000","last_clean_scrub_stamp":"2024-05-09T06:36:56.544873+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:17:58.482558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T06:36:58.582605+0000","last_change":"2024-05-09T06:36:58.578899+0000","last_active":"2024-05-09T06:36:58.582605+0000","last_peered":"2024-05-09T06:36:58.582605+0000","last_clean":"2024-05-09T06:36:58.582605+0000","last_became_active":"2024-05-09T06:36:58.578611+0000","last_became_peered":"2024-05-09T06:36:58.578611+0000","last_unstale":"2024-05-09T06:36:58.582605+0000","last_undegraded":"2024-05-09T06:36:58.582605+0000","last_fullsized":"2024-05-09T06:36:58.582605+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_clean_scrub_stamp":"2024-05-09T06:36:57.547101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:40:37.529479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"32'2","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T06:36:58.578688+0000","last_change":"2024-05-09T06:36:57.573885+0000","last_active":"2024-05-09T06:36:58.578688+0000","last_peered":"2024-05-09T06:36:58.578688+0000","last_clean":"2024-05-09T06:36:58.578688+0000","last_became_active":"2024-05-09T06:36:57.573554+0000","last_became_peered":"2024-05-09T06:36:57.573554+0000","last_unstale":"2024-05-09T06:36:58.578688+0000","last_undegraded":"2024-05-09T06:36:58.578688+0000","last_fullsized":"2024-05-09T06:36:58.578688+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:56.544873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:56.544873+0000","last_clean_scrub_stamp":"2024-05-09T06:36:56.544873+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:08:21.330262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T06:36:58.588464+0000","last_change":"2024-05-09T06:36:58.586396+0000","last_active":"2024-05-09T06:36:58.588464+0000","last_peered":"2024-05-09T06:36:58.588464+0000","last_clean":"2024-05-09T06:36:58.588464+0000","last_became_active":"2024-05-09T06:36:58.586103+0000","last_became_peered":"2024-05-09T06:36:58.586103+0000","last_unstale":"2024-05-09T06:36:58.588464+0000","last_undegraded":"2024-05-09T06:36:58.588464+0000","last_fullsized":"2024-05-09T06:36:58.588464+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_clean_scrub_stamp":"2024-05-09T06:36:57.547101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:35:21.215213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"32'1","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T06:36:58.584306+0000","last_change":"2024-05-09T06:36:57.572543+0000","last_active":"2024-05-09T06:36:58.584306+0000","last_peered":"2024-05-09T06:36:58.584306+0000","last_clean":"2024-05-09T06:36:58.584306+0000","last_became_active":"2024-05-09T06:36:57.572287+0000","last_became_peered":"2024-05-09T06:36:57.572287+0000","last_unstale":"2024-05-09T06:36:58.584306+0000","last_undegraded":"2024-05-09T06:36:58.584306+0000","last_fullsized":"2024-05-09T06:36:58.584306+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:56.544873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:56.544873+0000","last_clean_scrub_stamp":"2024-05-09T06:36:56.544873+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:22:40.811061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T06:36:58.581990+0000","last_change":"2024-05-09T06:36:58.576676+0000","last_active":"2024-05-09T06:36:58.581990+0000","last_peered":"2024-05-09T06:36:58.581990+0000","last_clean":"2024-05-09T06:36:58.581990+0000","last_became_active":"2024-05-09T06:36:58.576282+0000","last_became_peered":"2024-05-09T06:36:58.576282+0000","last_unstale":"2024-05-09T06:36:58.581990+0000","last_undegraded":"2024-05-09T06:36:58.581990+0000","last_fullsized":"2024-05-09T06:36:58.581990+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_clean_scrub_stamp":"2024-05-09T06:36:57.547101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:41:10.976320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T06:36:58.559460+0000","last_change":"2024-05-09T06:36:57.571629+0000","last_active":"2024-05-09T06:36:58.559460+0000","last_peered":"2024-05-09T06:36:58.559460+0000","last_clean":"2024-05-09T06:36:58.559460+0000","last_became_active":"2024-05-09T06:36:57.571349+0000","last_became_peered":"2024-05-09T06:36:57.571349+0000","last_unstale":"2024-05-09T06:36:58.559460+0000","last_undegraded":"2024-05-09T06:36:58.559460+0000","last_fullsized":"2024-05-09T06:36:58.559460+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:56.544873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:56.544873+0000","last_clean_scrub_stamp":"2024-05-09T06:36:56.544873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:25:29.082214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T06:36:58.569500+0000","last_change":"2024-05-09T06:36:58.568515+0000","last_active":"2024-05-09T06:36:58.569500+0000","last_peered":"2024-05-09T06:36:58.569500+0000","last_clean":"2024-05-09T06:36:58.569500+0000","last_became_active":"2024-05-09T06:36:58.568217+0000","last_became_peered":"2024-05-09T06:36:58.568217+0000","last_unstale":"2024-05-09T06:36:58.569500+0000","last_undegraded":"2024-05-09T06:36:58.569500+0000","last_fullsized":"2024-05-09T06:36:58.569500+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_clean_scrub_stamp":"2024-05-09T06:36:57.547101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:11:18.718993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"32'2","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T06:36:58.585701+0000","last_change":"2024-05-09T06:36:57.571198+0000","last_active":"2024-05-09T06:36:58.585701+0000","last_peered":"2024-05-09T06:36:58.585701+0000","last_clean":"2024-05-09T06:36:58.585701+0000","last_became_active":"2024-05-09T06:36:57.570902+0000","last_became_peered":"2024-05-09T06:36:57.570902+0000","last_unstale":"2024-05-09T06:36:58.585701+0000","last_undegraded":"2024-05-09T06:36:58.585701+0000","last_fullsized":"2024-05-09T06:36:58.585701+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:56.544873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:56.544873+0000","last_clean_scrub_stamp":"2024-05-09T06:36:56.544873+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:13:48.003027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T06:36:58.577070+0000","last_change":"2024-05-09T06:36:58.573013+0000","last_active":"2024-05-09T06:36:58.577070+0000","last_peered":"2024-05-09T06:36:58.577070+0000","last_clean":"2024-05-09T06:36:58.577070+0000","last_became_active":"2024-05-09T06:36:58.572718+0000","last_became_peered":"2024-05-09T06:36:58.572718+0000","last_unstale":"2024-05-09T06:36:58.577070+0000","last_undegraded":"2024-05-09T06:36:58.577070+0000","last_fullsized":"2024-05-09T06:36:58.577070+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_clean_scrub_stamp":"2024-05-09T06:36:57.547101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:18:09.799782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T06:36:58.563676+0000","last_change":"2024-05-09T06:36:57.571413+0000","last_active":"2024-05-09T06:36:58.563676+0000","last_peered":"2024-05-09T06:36:58.563676+0000","last_clean":"2024-05-09T06:36:58.563676+0000","last_became_active":"2024-05-09T06:36:57.570893+0000","last_became_peered":"2024-05-09T06:36:57.570893+0000","last_unstale":"2024-05-09T06:36:58.563676+0000","last_undegraded":"2024-05-09T06:36:58.563676+0000","last_fullsized":"2024-05-09T06:36:58.563676+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:56.544873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:56.544873+0000","last_clean_scrub_stamp":"2024-05-09T06:36:56.544873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:54:46.765892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T06:36:58.581921+0000","last_change":"2024-05-09T06:36:58.574149+0000","last_active":"2024-05-09T06:36:58.581921+0000","last_peered":"2024-05-09T06:36:58.581921+0000","last_clean":"2024-05-09T06:36:58.581921+0000","last_became_active":"2024-05-09T06:36:58.573789+0000","last_became_peered":"2024-05-09T06:36:58.573789+0000","last_unstale":"2024-05-09T06:36:58.581921+0000","last_undegraded":"2024-05-09T06:36:58.581921+0000","last_fullsized":"2024-05-09T06:36:58.581921+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_clean_scrub_stamp":"2024-05-09T06:36:57.547101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:45:52.389606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T06:36:58.558676+0000","last_change":"2024-05-09T06:36:57.576882+0000","last_active":"2024-05-09T06:36:58.558676+0000","last_peered":"2024-05-09T06:36:58.558676+0000","last_clean":"2024-05-09T06:36:58.558676+0000","last_became_active":"2024-05-09T06:36:57.576586+0000","last_became_peered":"2024-05-09T06:36:57.576586+0000","last_unstale":"2024-05-09T06:36:58.558676+0000","last_undegraded":"2024-05-09T06:36:58.558676+0000","last_fullsized":"2024-05-09T06:36:58.558676+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:56.544873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:56.544873+0000","last_clean_scrub_stamp":"2024-05-09T06:36:56.544873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:05:16.941219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T06:36:58.580873+0000","last_change":"2024-05-09T06:36:58.574529+0000","last_active":"2024-05-09T06:36:58.580873+0000","last_peered":"2024-05-09T06:36:58.580873+0000","last_clean":"2024-05-09T06:36:58.580873+0000","last_became_active":"2024-05-09T06:36:58.574272+0000","last_became_peered":"2024-05-09T06:36:58.574272+0000","last_unstale":"2024-05-09T06:36:58.580873+0000","last_undegraded":"2024-05-09T06:36:58.580873+0000","last_fullsized":"2024-05-09T06:36:58.580873+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_clean_scrub_stamp":"2024-05-09T06:36:57.547101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:13:49.005255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T06:36:58.566319+0000","last_change":"2024-05-09T06:36:57.575260+0000","last_active":"2024-05-09T06:36:58.566319+0000","last_peered":"2024-05-09T06:36:58.566319+0000","last_clean":"2024-05-09T06:36:58.566319+0000","last_became_active":"2024-05-09T06:36:57.574970+0000","last_became_peered":"2024-05-09T06:36:57.574970+0000","last_unstale":"2024-05-09T06:36:58.566319+0000","last_undegraded":"2024-05-09T06:36:58.566319+0000","last_fullsized":"2024-05-09T06:36:58.566319+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:56.544873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:56.544873+0000","last_clean_scrub_stamp":"2024-05-09T06:36:56.544873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:47:36.032918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T06:36:58.584127+0000","last_change":"2024-05-09T06:36:58.578342+0000","last_active":"2024-05-09T06:36:58.584127+0000","last_peered":"2024-05-09T06:36:58.584127+0000","last_clean":"2024-05-09T06:36:58.584127+0000","last_became_active":"2024-05-09T06:36:58.578063+0000","last_became_peered":"2024-05-09T06:36:58.578063+0000","last_unstale":"2024-05-09T06:36:58.584127+0000","last_undegraded":"2024-05-09T06:36:58.584127+0000","last_fullsized":"2024-05-09T06:36:58.584127+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_clean_scrub_stamp":"2024-05-09T06:36:57.547101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:05:17.943447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"32'2","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T06:36:58.587478+0000","last_change":"2024-05-09T06:36:57.575847+0000","last_active":"2024-05-09T06:36:58.587478+0000","last_peered":"2024-05-09T06:36:58.587478+0000","last_clean":"2024-05-09T06:36:58.587478+0000","last_became_active":"2024-05-09T06:36:57.575472+0000","last_became_peered":"2024-05-09T06:36:57.575472+0000","last_unstale":"2024-05-09T06:36:58.587478+0000","last_undegraded":"2024-05-09T06:36:58.587478+0000","last_fullsized":"2024-05-09T06:36:58.587478+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:56.544873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:56.544873+0000","last_clean_scrub_stamp":"2024-05-09T06:36:56.544873+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:13:48.003027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T06:36:58.588590+0000","last_change":"2024-05-09T06:36:58.585339+0000","last_active":"2024-05-09T06:36:58.588590+0000","last_peered":"2024-05-09T06:36:58.588590+0000","last_clean":"2024-05-09T06:36:58.588590+0000","last_became_active":"2024-05-09T06:36:58.585018+0000","last_became_peered":"2024-05-09T06:36:58.585018+0000","last_unstale":"2024-05-09T06:36:58.588590+0000","last_undegraded":"2024-05-09T06:36:58.588590+0000","last_fullsized":"2024-05-09T06:36:58.588590+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_clean_scrub_stamp":"2024-05-09T06:36:57.547101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:25:07.739454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T06:36:58.569942+0000","last_change":"2024-05-09T06:36:57.571388+0000","last_active":"2024-05-09T06:36:58.569942+0000","last_peered":"2024-05-09T06:36:58.569942+0000","last_clean":"2024-05-09T06:36:58.569942+0000","last_became_active":"2024-05-09T06:36:57.570865+0000","last_became_peered":"2024-05-09T06:36:57.570865+0000","last_unstale":"2024-05-09T06:36:58.569942+0000","last_undegraded":"2024-05-09T06:36:58.569942+0000","last_fullsized":"2024-05-09T06:36:58.569942+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:56.544873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:56.544873+0000","last_clean_scrub_stamp":"2024-05-09T06:36:56.544873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:02:40.382815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T06:36:58.583993+0000","last_change":"2024-05-09T06:36:58.583093+0000","last_active":"2024-05-09T06:36:58.583993+0000","last_peered":"2024-05-09T06:36:58.583993+0000","last_clean":"2024-05-09T06:36:58.583993+0000","last_became_active":"2024-05-09T06:36:58.582823+0000","last_became_peered":"2024-05-09T06:36:58.582823+0000","last_unstale":"2024-05-09T06:36:58.583993+0000","last_undegraded":"2024-05-09T06:36:58.583993+0000","last_fullsized":"2024-05-09T06:36:58.583993+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_clean_scrub_stamp":"2024-05-09T06:36:57.547101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:14:46.331201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T06:36:58.573381+0000","last_change":"2024-05-09T06:36:57.573519+0000","last_active":"2024-05-09T06:36:58.573381+0000","last_peered":"2024-05-09T06:36:58.573381+0000","last_clean":"2024-05-09T06:36:58.573381+0000","last_became_active":"2024-05-09T06:36:57.573157+0000","last_became_peered":"2024-05-09T06:36:57.573157+0000","last_unstale":"2024-05-09T06:36:58.573381+0000","last_undegraded":"2024-05-09T06:36:58.573381+0000","last_fullsized":"2024-05-09T06:36:58.573381+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:56.544873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:56.544873+0000","last_clean_scrub_stamp":"2024-05-09T06:36:56.544873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:03:44.500904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T06:36:58.579954+0000","last_change":"2024-05-09T06:36:58.572755+0000","last_active":"2024-05-09T06:36:58.579954+0000","last_peered":"2024-05-09T06:36:58.579954+0000","last_clean":"2024-05-09T06:36:58.579954+0000","last_became_active":"2024-05-09T06:36:58.572481+0000","last_became_peered":"2024-05-09T06:36:58.572481+0000","last_unstale":"2024-05-09T06:36:58.579954+0000","last_undegraded":"2024-05-09T06:36:58.579954+0000","last_fullsized":"2024-05-09T06:36:58.579954+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_clean_scrub_stamp":"2024-05-09T06:36:57.547101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:13:49.005255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T06:36:58.570538+0000","last_change":"2024-05-09T06:36:57.572806+0000","last_active":"2024-05-09T06:36:58.570538+0000","last_peered":"2024-05-09T06:36:58.570538+0000","last_clean":"2024-05-09T06:36:58.570538+0000","last_became_active":"2024-05-09T06:36:57.572446+0000","last_became_peered":"2024-05-09T06:36:57.572446+0000","last_unstale":"2024-05-09T06:36:58.570538+0000","last_undegraded":"2024-05-09T06:36:58.570538+0000","last_fullsized":"2024-05-09T06:36:58.570538+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:56.544873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:56.544873+0000","last_clean_scrub_stamp":"2024-05-09T06:36:56.544873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:17:58.482558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T06:36:58.584030+0000","last_change":"2024-05-09T06:36:58.581217+0000","last_active":"2024-05-09T06:36:58.584030+0000","last_peered":"2024-05-09T06:36:58.584030+0000","last_clean":"2024-05-09T06:36:58.584030+0000","last_became_active":"2024-05-09T06:36:58.580927+0000","last_became_peered":"2024-05-09T06:36:58.580927+0000","last_unstale":"2024-05-09T06:36:58.584030+0000","last_undegraded":"2024-05-09T06:36:58.584030+0000","last_fullsized":"2024-05-09T06:36:58.584030+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_clean_scrub_stamp":"2024-05-09T06:36:57.547101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:39:22.810622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T06:36:58.572558+0000","last_change":"2024-05-09T06:36:57.572648+0000","last_active":"2024-05-09T06:36:58.572558+0000","last_peered":"2024-05-09T06:36:58.572558+0000","last_clean":"2024-05-09T06:36:58.572558+0000","last_became_active":"2024-05-09T06:36:57.572365+0000","last_became_peered":"2024-05-09T06:36:57.572365+0000","last_unstale":"2024-05-09T06:36:58.572558+0000","last_undegraded":"2024-05-09T06:36:58.572558+0000","last_fullsized":"2024-05-09T06:36:58.572558+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:56.544873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:56.544873+0000","last_clean_scrub_stamp":"2024-05-09T06:36:56.544873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:02:40.382815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T06:36:58.588004+0000","last_change":"2024-05-09T06:36:58.582957+0000","last_active":"2024-05-09T06:36:58.588004+0000","last_peered":"2024-05-09T06:36:58.588004+0000","last_clean":"2024-05-09T06:36:58.588004+0000","last_became_active":"2024-05-09T06:36:58.582674+0000","last_became_peered":"2024-05-09T06:36:58.582674+0000","last_unstale":"2024-05-09T06:36:58.588004+0000","last_undegraded":"2024-05-09T06:36:58.588004+0000","last_fullsized":"2024-05-09T06:36:58.588004+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_clean_scrub_stamp":"2024-05-09T06:36:57.547101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:44:53.864242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T06:36:58.558908+0000","last_change":"2024-05-09T06:36:57.572999+0000","last_active":"2024-05-09T06:36:58.558908+0000","last_peered":"2024-05-09T06:36:58.558908+0000","last_clean":"2024-05-09T06:36:58.558908+0000","last_became_active":"2024-05-09T06:36:57.572327+0000","last_became_peered":"2024-05-09T06:36:57.572327+0000","last_unstale":"2024-05-09T06:36:58.558908+0000","last_undegraded":"2024-05-09T06:36:58.558908+0000","last_fullsized":"2024-05-09T06:36:58.558908+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:56.544873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:56.544873+0000","last_clean_scrub_stamp":"2024-05-09T06:36:56.544873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:45:00.356381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T06:36:58.573408+0000","last_change":"2024-05-09T06:36:58.565289+0000","last_active":"2024-05-09T06:36:58.573408+0000","last_peered":"2024-05-09T06:36:58.573408+0000","last_clean":"2024-05-09T06:36:58.573408+0000","last_became_active":"2024-05-09T06:36:58.564823+0000","last_became_peered":"2024-05-09T06:36:58.564823+0000","last_unstale":"2024-05-09T06:36:58.573408+0000","last_undegraded":"2024-05-09T06:36:58.573408+0000","last_fullsized":"2024-05-09T06:36:58.573408+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_clean_scrub_stamp":"2024-05-09T06:36:57.547101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:47:54.992336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T06:36:58.558147+0000","last_change":"2024-05-09T06:36:57.576383+0000","last_active":"2024-05-09T06:36:58.558147+0000","last_peered":"2024-05-09T06:36:58.558147+0000","last_clean":"2024-05-09T06:36:58.558147+0000","last_became_active":"2024-05-09T06:36:57.576046+0000","last_became_peered":"2024-05-09T06:36:57.576046+0000","last_unstale":"2024-05-09T06:36:58.558147+0000","last_undegraded":"2024-05-09T06:36:58.558147+0000","last_fullsized":"2024-05-09T06:36:58.558147+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:56.544873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:56.544873+0000","last_clean_scrub_stamp":"2024-05-09T06:36:56.544873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:47:36.032918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T06:36:58.579469+0000","last_change":"2024-05-09T06:36:58.574222+0000","last_active":"2024-05-09T06:36:58.579469+0000","last_peered":"2024-05-09T06:36:58.579469+0000","last_clean":"2024-05-09T06:36:58.579469+0000","last_became_active":"2024-05-09T06:36:58.573947+0000","last_became_peered":"2024-05-09T06:36:58.573947+0000","last_unstale":"2024-05-09T06:36:58.579469+0000","last_undegraded":"2024-05-09T06:36:58.579469+0000","last_fullsized":"2024-05-09T06:36:58.579469+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:57.547101+0000","last_clean_scrub_stamp":"2024-05-09T06:36:57.547101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:03:45.503132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T06:36:58.563918+0000","last_change":"2024-05-09T06:36:57.569766+0000","last_active":"2024-05-09T06:36:58.563918+0000","last_peered":"2024-05-09T06:36:58.563918+0000","last_clean":"2024-05-09T06:36:58.563918+0000","last_became_active":"2024-05-09T06:36:57.569458+0000","last_became_peered":"2024-05-09T06:36:57.569458+0000","last_unstale":"2024-05-09T06:36:58.563918+0000","last_undegraded":"2024-05-09T06:36:58.563918+0000","last_fullsized":"2024-05-09T06:36:58.563918+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:36:56.544873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:36:56.544873+0000","last_clean_scrub_stamp":"2024-05-09T06:36:56.544873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:05:37.367369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":191,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T06:36:58.570131+0000","last_change":"2024-05-09T06:35:55.895214+0000","last_active":"2024-05-09T06:36:58.570131+0000","last_peered":"2024-05-09T06:36:58.570131+0000","last_clean":"2024-05-09T06:36:58.570131+0000","last_became_active":"2024-05-09T06:35:55.894087+0000","last_became_peered":"2024-05-09T06:35:55.894087+0000","last_unstale":"2024-05-09T06:36:58.570131+0000","last_undegraded":"2024-05-09T06:36:58.570131+0000","last_fullsized":"2024-05-09T06:36:58.570131+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:35:54.820102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:35:54.820102+0000","last_clean_scrub_stamp":"2024-05-09T06:35:54.820102+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:29:20.408099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574865,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27944,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343896,"statfs":{"total":96636764160,"available":96608149504,"internally_reserved":0,"allocated":942080,"data_stored":723909,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574865,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27380,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344460,"statfs":{"total":96636764160,"available":96608727040,"internally_reserved":0,"allocated":368640,"data_stored":140932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574864,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27364,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344476,"statfs":{"total":96636764160,"available":96608743424,"internally_reserved":0,"allocated":348160,"data_stored":133541,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574864,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27380,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344460,"statfs":{"total":96636764160,"available":96608727040,"internally_reserved":0,"allocated":368640,"data_stored":140420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574865,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27364,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344476,"statfs":{"total":96636764160,"available":96608743424,"internally_reserved":0,"allocated":348160,"data_stored":133541,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574864,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27364,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344476,"statfs":{"total":96636764160,"available":96608743424,"internally_reserved":0,"allocated":348160,"data_stored":133541,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574864,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27364,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344476,"statfs":{"total":96636764160,"available":96608743424,"internally_reserved":0,"allocated":348160,"data_stored":133541,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574865,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27956,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343884,"statfs":{"total":96636764160,"available":96608137216,"internally_reserved":0,"allocated":954368,"data_stored":730684,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-09T06:37:02.927 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-05-09T06:37:02.928 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-05-09T06:37:03.076 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:03.095+0000 7f9dd36da640 1 Processor -- start 2024-05-09T06:37:03.076 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:03.095+0000 7f9dd36da640 1 -- start start 2024-05-09T06:37:03.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:03.096+0000 7f9dd36da640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f9dcc144a10 0x7f9dcc0cf8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:03.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:03.096+0000 7f9dd36da640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f9dcc0b4130 0x7f9dcc0b45e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:03.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:03.096+0000 7f9dd36da640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9dcc071860 con 0x7f9dcc072900 2024-05-09T06:37:03.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:03.096+0000 7f9dd36da640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f9dcc072470 con 0x7f9dcc144a10 2024-05-09T06:37:03.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:03.096+0000 7f9dd36da640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f9dcc071b40 con 0x7f9dcc0b4130 2024-05-09T06:37:03.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:03.096+0000 7f9dd1476640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f9dcc144a10 0x7f9dcc0cf8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:03.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:03.096+0000 7f9dd1476640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f9dcc144a10 0x7f9dcc0cf8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:39420/0 (socket says 172.21.15.3:39420) 2024-05-09T06:37:03.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:03.096+0000 7f9dd1476640 1 -- 172.21.15.3:0/1328524532 learned_addr learned my addr 172.21.15.3:0/1328524532 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:37:03.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:03.096+0000 7f9dd1c77640 1 --2- 172.21.15.3:0/1328524532 >> v2:172.21.15.190:3300/0 conn(0x7f9dcc0b4130 0x7f9dcc0b45e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:03.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:03.096+0000 7f9dd1476640 1 -- 172.21.15.3:0/1328524532 >> v2:172.21.15.190:3300/0 conn(0x7f9dcc0b4130 msgr2=0x7f9dcc0b45e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:37:03.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:03.096+0000 7f9dd1476640 1 --2- 172.21.15.3:0/1328524532 >> v2:172.21.15.190:3300/0 conn(0x7f9dcc0b4130 0x7f9dcc0b45e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:03.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:03.096+0000 7f9dd1476640 1 -- 172.21.15.3:0/1328524532 >> v1:172.21.15.190:6789/0 conn(0x7f9dcc072900 legacy=0x7f9dcc071080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:03.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:03.096+0000 7f9dd1476640 1 -- 172.21.15.3:0/1328524532 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9dcc071e20 con 0x7f9dcc144a10 2024-05-09T06:37:03.079 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:03.097+0000 7f9dd1476640 1 --2- 172.21.15.3:0/1328524532 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9dcc144a10 0x7f9dcc0cf8a0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f9dcc0b2fb0 tx=0x7f9dc802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=979cff9e2a326661 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:03.079 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:03.097+0000 7f9dc3fff640 1 -- 172.21.15.3:0/1328524532 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dc8037070 con 0x7f9dcc144a10 2024-05-09T06:37:03.079 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:03.097+0000 7f9dc3fff640 1 -- 172.21.15.3:0/1328524532 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9dc8032440 con 0x7f9dcc144a10 2024-05-09T06:37:03.079 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:03.097+0000 7f9dc3fff640 1 -- 172.21.15.3:0/1328524532 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dc8032810 con 0x7f9dcc144a10 2024-05-09T06:37:03.079 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:03.098+0000 7f9dd36da640 1 -- 172.21.15.3:0/1328524532 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9dcc144a10 msgr2=0x7f9dcc0cf8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:03.079 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:03.098+0000 7f9dd36da640 1 --2- 172.21.15.3:0/1328524532 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9dcc144a10 0x7f9dcc0cf8a0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f9dcc0b2fb0 tx=0x7f9dc802a1e0 comp rx=0 tx=0).stop 2024-05-09T06:37:03.079 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:03.098+0000 7f9dd36da640 1 -- 172.21.15.3:0/1328524532 shutdown_connections 2024-05-09T06:37:03.079 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:03.098+0000 7f9dd36da640 1 --2- 172.21.15.3:0/1328524532 >> v2:172.21.15.190:3300/0 conn(0x7f9dcc0b4130 0x7f9dcc0b45e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:03.079 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:03.098+0000 7f9dd36da640 1 --2- 172.21.15.3:0/1328524532 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9dcc144a10 0x7f9dcc0cf8a0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:03.080 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:03.098+0000 7f9dd36da640 1 -- 172.21.15.3:0/1328524532 >> 172.21.15.3:0/1328524532 conn(0x7f9dcc0b1c20 msgr2=0x7f9dcc0b2030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:03.080 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:03.098+0000 7f9dd36da640 1 -- 172.21.15.3:0/1328524532 shutdown_connections 2024-05-09T06:37:03.080 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:03.098+0000 7f9dd36da640 1 -- 172.21.15.3:0/1328524532 wait complete. 2024-05-09T06:37:03.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:03.099+0000 7f9dd36da640 1 Processor -- start 2024-05-09T06:37:03.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:03.099+0000 7f9dd36da640 1 -- start start 2024-05-09T06:37:03.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:03.100+0000 7f9dd36da640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9dcc072900 0x7f9dcc1f5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:03.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:03.100+0000 7f9dd36da640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9dcc0b4130 0x7f9dcc159f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:03.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:03.100+0000 7f9dd36da640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9dcc144a10 0x7f9dcc06e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:03.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:03.100+0000 7f9dd36da640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f9dcc15b750 con 0x7f9dcc144a10 2024-05-09T06:37:03.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:03.100+0000 7f9dd36da640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f9dcc15b5d0 con 0x7f9dcc0b4130 2024-05-09T06:37:03.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:03.100+0000 7f9dd36da640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f9dcc15b8d0 con 0x7f9dcc072900 2024-05-09T06:37:03.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:03.100+0000 7f9dd1c77640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9dcc144a10 0x7f9dcc06e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:03.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:03.100+0000 7f9dd1c77640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9dcc144a10 0x7f9dcc06e0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:39430/0 (socket says 172.21.15.3:39430) 2024-05-09T06:37:03.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:03.100+0000 7f9dd1c77640 1 -- 172.21.15.3:0/3199348007 learned_addr learned my addr 172.21.15.3:0/3199348007 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:37:03.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:03.100+0000 7f9dd1476640 1 --2- 172.21.15.3:0/3199348007 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9dcc072900 0x7f9dcc1f5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:03.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:03.100+0000 7f9dd1c77640 1 -- 172.21.15.3:0/3199348007 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9dcc072900 msgr2=0x7f9dcc1f5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:03.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:03.100+0000 7f9dd1c77640 1 --2- 172.21.15.3:0/3199348007 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9dcc072900 0x7f9dcc1f5050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:03.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:03.100+0000 7f9dd1c77640 1 -- 172.21.15.3:0/3199348007 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9dcc0b4130 msgr2=0x7f9dcc159f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:03.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:03.100+0000 7f9dd0c75640 1 --2- 172.21.15.3:0/3199348007 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9dcc0b4130 0x7f9dcc159f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:03.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:03.100+0000 7f9dd1c77640 1 --2- 172.21.15.3:0/3199348007 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9dcc0b4130 0x7f9dcc159f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:03.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:03.100+0000 7f9dd1c77640 1 -- 172.21.15.3:0/3199348007 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9dcc139450 con 0x7f9dcc144a10 2024-05-09T06:37:03.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:03.100+0000 7f9dd0c75640 1 --2- 172.21.15.3:0/3199348007 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9dcc0b4130 0x7f9dcc159f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:37:03.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:03.100+0000 7f9dd1c77640 1 --2- 172.21.15.3:0/3199348007 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9dcc144a10 0x7f9dcc06e0e0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f9dc4007e70 tx=0x7f9dc400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:03.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:03.101+0000 7f9dc1ffb640 1 -- 172.21.15.3:0/3199348007 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dc4014620 con 0x7f9dcc144a10 2024-05-09T06:37:03.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:03.101+0000 7f9dc1ffb640 1 -- 172.21.15.3:0/3199348007 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9dc4019070 con 0x7f9dcc144a10 2024-05-09T06:37:03.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:03.101+0000 7f9dd36da640 1 -- 172.21.15.3:0/3199348007 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9dcc139730 con 0x7f9dcc144a10 2024-05-09T06:37:03.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:03.101+0000 7f9dc1ffb640 1 -- 172.21.15.3:0/3199348007 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dc4014a20 con 0x7f9dcc144a10 2024-05-09T06:37:03.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:03.101+0000 7f9dd36da640 1 -- 172.21.15.3:0/3199348007 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9dcc139cf0 con 0x7f9dcc144a10 2024-05-09T06:37:03.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:03.102+0000 7f9dd36da640 1 -- 172.21.15.3:0/3199348007 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9dcc069550 con 0x7f9dcc144a10 2024-05-09T06:37:03.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:03.102+0000 7f9dc1ffb640 1 -- 172.21.15.3:0/3199348007 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9dc4014bc0 con 0x7f9dcc144a10 2024-05-09T06:37:03.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:03.103+0000 7f9dc1ffb640 1 --2- 172.21.15.3:0/3199348007 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f9db006be60 0x7f9db006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:03.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:03.103+0000 7f9dc1ffb640 1 -- 172.21.15.3:0/3199348007 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f9dc408b680 con 0x7f9dcc144a10 2024-05-09T06:37:03.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:03.103+0000 7f9dd1476640 1 --2- 172.21.15.3:0/3199348007 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f9db006be60 0x7f9db006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:03.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:03.104+0000 7f9dd1476640 1 --2- 172.21.15.3:0/3199348007 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f9db006be60 0x7f9db006e320 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f9dc8034000 tx=0x7f9dc802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:03.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:03.107+0000 7f9dc1ffb640 1 -- 172.21.15.3:0/3199348007 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f9dc405a0c0 con 0x7f9dcc144a10 2024-05-09T06:37:03.345 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:03.365+0000 7f9dd36da640 1 -- 172.21.15.3:0/3199348007 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f9dcc074770 con 0x7f9dcc144a10 2024-05-09T06:37:03.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:03.367+0000 7f9dc1ffb640 1 -- 172.21.15.3:0/3199348007 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v11) ==== 190+0+147 (secure 0 0 0) 0x7f9dc40599c0 con 0x7f9dcc144a10 2024-05-09T06:37:03.348 INFO:teuthology.orchestra.run.smithi003.stdout:[client.0] 2024-05-09T06:37:03.348 INFO:teuthology.orchestra.run.smithi003.stdout: key = AQCdbjxmhD/JCBAAcZ8c4sZXw8rR2/s1Lr++wQ== 2024-05-09T06:37:03.348 INFO:teuthology.orchestra.run.smithi003.stdout: caps mds = "allow" 2024-05-09T06:37:03.348 INFO:teuthology.orchestra.run.smithi003.stdout: caps mon = "allow r" 2024-05-09T06:37:03.348 INFO:teuthology.orchestra.run.smithi003.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-05-09T06:37:03.348 INFO:teuthology.orchestra.run.smithi003.stderr:updated caps for client.0 2024-05-09T06:37:03.350 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:03.369+0000 7f9dd36da640 1 -- 172.21.15.3:0/3199348007 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f9db006be60 msgr2=0x7f9db006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:03.350 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:03.369+0000 7f9dd36da640 1 --2- 172.21.15.3:0/3199348007 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f9db006be60 0x7f9db006e320 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f9dc8034000 tx=0x7f9dc802e000 comp rx=0 tx=0).stop 2024-05-09T06:37:03.350 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:03.369+0000 7f9dd36da640 1 -- 172.21.15.3:0/3199348007 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9dcc144a10 msgr2=0x7f9dcc06e0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:03.350 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:03.369+0000 7f9dd36da640 1 --2- 172.21.15.3:0/3199348007 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9dcc144a10 0x7f9dcc06e0e0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f9dc4007e70 tx=0x7f9dc400e4b0 comp rx=0 tx=0).stop 2024-05-09T06:37:03.350 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:03.369+0000 7f9dd36da640 1 -- 172.21.15.3:0/3199348007 shutdown_connections 2024-05-09T06:37:03.350 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:03.369+0000 7f9dd36da640 1 --2- 172.21.15.3:0/3199348007 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f9db006be60 0x7f9db006e320 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:03.350 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:03.369+0000 7f9dd36da640 1 --2- 172.21.15.3:0/3199348007 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9dcc144a10 0x7f9dcc06e0e0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:03.350 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:03.369+0000 7f9dd36da640 1 --2- 172.21.15.3:0/3199348007 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9dcc0b4130 0x7f9dcc159f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:03.350 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:03.369+0000 7f9dd36da640 1 --2- 172.21.15.3:0/3199348007 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9dcc072900 0x7f9dcc1f5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:03.351 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:03.369+0000 7f9dd36da640 1 -- 172.21.15.3:0/3199348007 >> 172.21.15.3:0/3199348007 conn(0x7f9dcc0b1c20 msgr2=0x7f9dcc0b4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:03.351 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:03.369+0000 7f9dd36da640 1 -- 172.21.15.3:0/3199348007 shutdown_connections 2024-05-09T06:37:03.351 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:03.369+0000 7f9dd36da640 1 -- 172.21.15.3:0/3199348007 wait complete. 2024-05-09T06:37:03.361 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T06:37:03.461 INFO:tasks.ceph.mds.c.smithi003.stderr:2024-05-09T06:37:03.480+0000 7f116c807640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-09T06:37:03.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:03.483+0000 7fab8c9a5640 1 Processor -- start 2024-05-09T06:37:03.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:03.484+0000 7fab8c9a5640 1 -- start start 2024-05-09T06:37:03.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:03.484+0000 7fab8c9a5640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fab8815ee30 0x7fab8806dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:03.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:03.485+0000 7fab8c9a5640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fab880cfc80 con 0x7fab8806e230 2024-05-09T06:37:03.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:03.485+0000 7fab8c9a5640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fab8815e390 con 0x7fab88154cc0 2024-05-09T06:37:03.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:03.485+0000 7fab8c9a5640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fab8806e6a0 con 0x7fab8815ee30 2024-05-09T06:37:03.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:03.485+0000 7fab85d9b640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fab8815ee30 0x7fab8806dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:03.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:03.485+0000 7fab85d9b640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fab8815ee30 0x7fab8806dcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:39444/0 (socket says 172.21.15.3:39444) 2024-05-09T06:37:03.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:03.485+0000 7fab85d9b640 1 -- 172.21.15.3:0/3521516739 learned_addr learned my addr 172.21.15.3:0/3521516739 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:37:03.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:03.485+0000 7fab86d9d640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fab8806e230 0x7fab880cdeb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:58062/0 (socket says 172.21.15.3:58062) 2024-05-09T06:37:03.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:03.485+0000 7fab85d9b640 1 -- 172.21.15.3:0/3521516739 >> v1:172.21.15.190:6789/0 conn(0x7fab88154cc0 legacy=0x7fab8815b6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:37:03.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:03.485+0000 7fab85d9b640 1 -- 172.21.15.3:0/3521516739 >> v1:172.21.15.3:6789/0 conn(0x7fab8806e230 legacy=0x7fab880cdeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:03.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:03.485+0000 7fab85d9b640 1 -- 172.21.15.3:0/3521516739 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab880cf9a0 con 0x7fab8815ee30 2024-05-09T06:37:03.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:03.485+0000 7fab8559a640 1 -- 172.21.15.3:0/3521516739 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3858633198 0 0) 0x7fab880cfc80 con 0x7fab8806e230 2024-05-09T06:37:03.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:03.486+0000 7fab85d9b640 1 --2- 172.21.15.3:0/3521516739 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fab8815ee30 0x7fab8806dcf0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fab7c009870 tx=0x7fab7c02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ca880c4c29bd00f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:03.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:03.486+0000 7fab8559a640 1 -- 172.21.15.3:0/3521516739 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab7c037070 con 0x7fab8815ee30 2024-05-09T06:37:03.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:03.486+0000 7fab8559a640 1 -- 172.21.15.3:0/3521516739 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab7c032440 con 0x7fab8815ee30 2024-05-09T06:37:03.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:03.486+0000 7fab8559a640 1 -- 172.21.15.3:0/3521516739 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab7c032810 con 0x7fab8815ee30 2024-05-09T06:37:03.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:03.486+0000 7fab8c9a5640 1 -- 172.21.15.3:0/3521516739 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fab8815ee30 msgr2=0x7fab8806dcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:03.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:03.486+0000 7fab8c9a5640 1 --2- 172.21.15.3:0/3521516739 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fab8815ee30 0x7fab8806dcf0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fab7c009870 tx=0x7fab7c02a210 comp rx=0 tx=0).stop 2024-05-09T06:37:03.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:03.487+0000 7fab8c9a5640 1 -- 172.21.15.3:0/3521516739 shutdown_connections 2024-05-09T06:37:03.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:03.487+0000 7fab8c9a5640 1 --2- 172.21.15.3:0/3521516739 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fab8815ee30 0x7fab8806dcf0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:03.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:03.487+0000 7fab8c9a5640 1 -- 172.21.15.3:0/3521516739 >> 172.21.15.3:0/3521516739 conn(0x7fab8806b430 msgr2=0x7fab8806b810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:03.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:03.487+0000 7fab8c9a5640 1 -- 172.21.15.3:0/3521516739 shutdown_connections 2024-05-09T06:37:03.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:03.487+0000 7fab8c9a5640 1 -- 172.21.15.3:0/3521516739 wait complete. 2024-05-09T06:37:03.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:03.488+0000 7fab8c9a5640 1 Processor -- start 2024-05-09T06:37:03.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:03.488+0000 7fab8c9a5640 1 -- start start 2024-05-09T06:37:03.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:03.489+0000 7fab8c9a5640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fab8806e230 0x7fab880b3090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:03.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:03.489+0000 7fab8c9a5640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fab88154cc0 0x7fab880b36e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:03.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:03.489+0000 7fab8c9a5640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fab8815ee30 0x7fab881400e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:03.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:03.489+0000 7fab8c9a5640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fab880cf690 con 0x7fab8815ee30 2024-05-09T06:37:03.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:03.489+0000 7fab8c9a5640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fab8815e910 con 0x7fab8806e230 2024-05-09T06:37:03.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:03.489+0000 7fab8c9a5640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fab881550a0 con 0x7fab88154cc0 2024-05-09T06:37:03.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:03.489+0000 7fab86d9d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fab8815ee30 0x7fab881400e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:03.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:03.489+0000 7fab86d9d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fab8815ee30 0x7fab881400e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:39452/0 (socket says 172.21.15.3:39452) 2024-05-09T06:37:03.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:03.489+0000 7fab86d9d640 1 -- 172.21.15.3:0/2357105066 learned_addr learned my addr 172.21.15.3:0/2357105066 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:37:03.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:03.489+0000 7fab8659c640 1 --2- 172.21.15.3:0/2357105066 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fab8806e230 0x7fab880b3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:03.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:03.489+0000 7fab85d9b640 1 --2- 172.21.15.3:0/2357105066 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fab88154cc0 0x7fab880b36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:03.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:03.489+0000 7fab86d9d640 1 -- 172.21.15.3:0/2357105066 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fab88154cc0 msgr2=0x7fab880b36e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:03.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:03.489+0000 7fab86d9d640 1 --2- 172.21.15.3:0/2357105066 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fab88154cc0 0x7fab880b36e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:03.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:03.489+0000 7fab86d9d640 1 -- 172.21.15.3:0/2357105066 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fab8806e230 msgr2=0x7fab880b3090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:03.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:03.489+0000 7fab86d9d640 1 --2- 172.21.15.3:0/2357105066 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fab8806e230 0x7fab880b3090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:03.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:03.489+0000 7fab86d9d640 1 -- 172.21.15.3:0/2357105066 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab880b3de0 con 0x7fab8815ee30 2024-05-09T06:37:03.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:03.489+0000 7fab8659c640 1 --2- 172.21.15.3:0/2357105066 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fab8806e230 0x7fab880b3090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:37:03.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:03.489+0000 7fab86d9d640 1 --2- 172.21.15.3:0/2357105066 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fab8815ee30 0x7fab881400e0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fab8000c7d0 tx=0x7fab8000cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:03.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:03.489+0000 7fab76ffd640 1 -- 172.21.15.3:0/2357105066 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab80018070 con 0x7fab8815ee30 2024-05-09T06:37:03.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:03.489+0000 7fab76ffd640 1 -- 172.21.15.3:0/2357105066 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab8000a640 con 0x7fab8815ee30 2024-05-09T06:37:03.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:03.489+0000 7fab8c9a5640 1 -- 172.21.15.3:0/2357105066 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab88152d60 con 0x7fab8815ee30 2024-05-09T06:37:03.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:03.490+0000 7fab76ffd640 1 -- 172.21.15.3:0/2357105066 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab8000aa70 con 0x7fab8815ee30 2024-05-09T06:37:03.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:03.490+0000 7fab8c9a5640 1 -- 172.21.15.3:0/2357105066 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab88153320 con 0x7fab8815ee30 2024-05-09T06:37:03.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:03.490+0000 7fab8c9a5640 1 -- 172.21.15.3:0/2357105066 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab8804f660 con 0x7fab8815ee30 2024-05-09T06:37:03.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:03.491+0000 7fab76ffd640 1 -- 172.21.15.3:0/2357105066 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fab8001b070 con 0x7fab8815ee30 2024-05-09T06:37:03.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:03.491+0000 7fab76ffd640 1 --2- 172.21.15.3:0/2357105066 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fab6806be30 0x7fab6806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:03.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:03.491+0000 7fab76ffd640 1 -- 172.21.15.3:0/2357105066 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fab8008a400 con 0x7fab8815ee30 2024-05-09T06:37:03.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:03.492+0000 7fab8659c640 1 --2- 172.21.15.3:0/2357105066 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fab6806be30 0x7fab6806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:03.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:03.492+0000 7fab8659c640 1 --2- 172.21.15.3:0/2357105066 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fab6806be30 0x7fab6806e2f0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fab881f7970 tx=0x7fab78009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:03.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:03.496+0000 7fab76ffd640 1 -- 172.21.15.3:0/2357105066 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fab80058e40 con 0x7fab8815ee30 2024-05-09T06:37:03.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:03.718+0000 7fab8c9a5640 1 -- 172.21.15.3:0/2357105066 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fab88068b10 con 0x7fab8815ee30 2024-05-09T06:37:03.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:03.719+0000 7fab76ffd640 1 -- 172.21.15.3:0/2357105066 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5181 (secure 0 0 0) 0x7fab80058740 con 0x7fab8815ee30 2024-05-09T06:37:03.699 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-09T06:37:03.699 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4912,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6836/2400645940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":2400645940},{"type":"v1","addr":"172.21.15.190:6836","nonce":2400645940}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4915,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6837/3849729576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6835","nonce":3849729576},{"type":"v1","addr":"172.21.15.190:6837","nonce":3849729576}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4918,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6834/2647782276","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":2647782276},{"type":"v1","addr":"172.21.15.3:6834","nonce":2647782276}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-09T06:36:57.929421+0000","modified":"2024-05-09T06:37:00.941894+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5103},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5103":{"gid":5103,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.3:6835/2138710335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6833","nonce":2138710335},{"type":"v1","addr":"172.21.15.3:6835","nonce":2138710335}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5103,"qdb_cluster":[5103]},"id":2}]} 2024-05-09T06:37:03.700 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 17 2024-05-09T06:37:03.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:03.720+0000 7fab8c9a5640 1 -- 172.21.15.3:0/2357105066 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fab6806be30 msgr2=0x7fab6806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:03.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:03.720+0000 7fab8c9a5640 1 --2- 172.21.15.3:0/2357105066 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fab6806be30 0x7fab6806e2f0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fab881f7970 tx=0x7fab78009040 comp rx=0 tx=0).stop 2024-05-09T06:37:03.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:03.720+0000 7fab8c9a5640 1 -- 172.21.15.3:0/2357105066 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fab8815ee30 msgr2=0x7fab881400e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:03.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:03.720+0000 7fab8c9a5640 1 --2- 172.21.15.3:0/2357105066 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fab8815ee30 0x7fab881400e0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fab8000c7d0 tx=0x7fab8000cc90 comp rx=0 tx=0).stop 2024-05-09T06:37:03.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:03.720+0000 7fab8c9a5640 1 -- 172.21.15.3:0/2357105066 shutdown_connections 2024-05-09T06:37:03.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:03.721+0000 7fab8c9a5640 1 --2- 172.21.15.3:0/2357105066 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fab6806be30 0x7fab6806e2f0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:03.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:03.721+0000 7fab8c9a5640 1 --2- 172.21.15.3:0/2357105066 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fab8815ee30 0x7fab881400e0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:03.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:03.721+0000 7fab8c9a5640 1 --2- 172.21.15.3:0/2357105066 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fab88154cc0 0x7fab880b36e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:03.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:03.721+0000 7fab8c9a5640 1 --2- 172.21.15.3:0/2357105066 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fab8806e230 0x7fab880b3090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:03.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:03.721+0000 7fab8c9a5640 1 -- 172.21.15.3:0/2357105066 >> 172.21.15.3:0/2357105066 conn(0x7fab8806b430 msgr2=0x7fab8806b810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:03.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:03.721+0000 7fab8c9a5640 1 -- 172.21.15.3:0/2357105066 shutdown_connections 2024-05-09T06:37:03.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:03.721+0000 7fab8c9a5640 1 -- 172.21.15.3:0/2357105066 wait complete. 2024-05-09T06:37:04.115 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-09T06:36:57.929421+0000', 'modified': '2024-05-09T06:37:00.941894+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5103}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5103': {'gid': 5103, 'name': 'c', 'rank': 0, 'incarnation': 13, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.3:6835/2138710335', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.3:6833', 'nonce': 2138710335}, {'type': 'v1', 'addr': '172.21.15.3:6835', 'nonce': 2138710335}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [5], 'metadata_pool': 4, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5103, 'qdb_cluster': [5103]} 2024-05-09T06:37:04.115 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-09T06:37:04.116 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-05-09T06:37:04.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.244+0000 7f876285d640 1 Processor -- start 2024-05-09T06:37:04.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.244+0000 7f876285d640 1 -- start start 2024-05-09T06:37:04.226 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.245+0000 7f876285d640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f875c15e490 0x7f875c162590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:04.226 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.245+0000 7f876285d640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f875c06b250 0x7f875c06b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:04.226 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.245+0000 7f876285d640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f875c06cad0 con 0x7f875c15e0b0 2024-05-09T06:37:04.226 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.245+0000 7f876285d640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f875c0b3eb0 con 0x7f875c06b250 2024-05-09T06:37:04.226 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.245+0000 7f876285d640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f875c0b4190 con 0x7f875c15e490 2024-05-09T06:37:04.226 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.245+0000 7f8760dfa640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f875c06b250 0x7f875c06b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:04.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.245+0000 7f8760dfa640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f875c06b250 0x7f875c06b6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:39466/0 (socket says 172.21.15.3:39466) 2024-05-09T06:37:04.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.245+0000 7f8760dfa640 1 -- 172.21.15.3:0/806004566 learned_addr learned my addr 172.21.15.3:0/806004566 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:37:04.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.245+0000 7f875b7fe640 1 --2- 172.21.15.3:0/806004566 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f875c15e490 0x7f875c162590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:04.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.245+0000 7f8760dfa640 1 -- 172.21.15.3:0/806004566 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f875c15e490 msgr2=0x7f875c162590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:37:04.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.245+0000 7f8760dfa640 1 --2- 172.21.15.3:0/806004566 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f875c15e490 0x7f875c162590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:04.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.245+0000 7f8760dfa640 1 -- 172.21.15.3:0/806004566 >> v1:172.21.15.190:6789/0 conn(0x7f875c15e0b0 legacy=0x7f875c161e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:37:04.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.245+0000 7f8760dfa640 1 -- 172.21.15.3:0/806004566 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f875c0b4470 con 0x7f875c06b250 2024-05-09T06:37:04.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.246+0000 7f8760dfa640 1 --2- 172.21.15.3:0/806004566 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f875c06b250 0x7f875c06b6c0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f8754009e50 tx=0x7f875402a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4412ed42dce2e925 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:04.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.246+0000 7f875affd640 1 -- 172.21.15.3:0/806004566 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8754036070 con 0x7f875c06b250 2024-05-09T06:37:04.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.246+0000 7f875affd640 1 -- 172.21.15.3:0/806004566 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8754031600 con 0x7f875c06b250 2024-05-09T06:37:04.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.246+0000 7f875affd640 1 -- 172.21.15.3:0/806004566 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87540319d0 con 0x7f875c06b250 2024-05-09T06:37:04.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.247+0000 7f876285d640 1 -- 172.21.15.3:0/806004566 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f875c06b250 msgr2=0x7f875c06b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:04.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.247+0000 7f876285d640 1 --2- 172.21.15.3:0/806004566 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f875c06b250 0x7f875c06b6c0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f8754009e50 tx=0x7f875402a6c0 comp rx=0 tx=0).stop 2024-05-09T06:37:04.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.247+0000 7f876285d640 1 -- 172.21.15.3:0/806004566 shutdown_connections 2024-05-09T06:37:04.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.247+0000 7f876285d640 1 --2- 172.21.15.3:0/806004566 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f875c06b250 0x7f875c06b6c0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:04.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.247+0000 7f876285d640 1 --2- 172.21.15.3:0/806004566 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f875c15e490 0x7f875c162590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:04.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.247+0000 7f876285d640 1 -- 172.21.15.3:0/806004566 >> 172.21.15.3:0/806004566 conn(0x7f875c0d07b0 msgr2=0x7f875c0d0bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:04.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.247+0000 7f876285d640 1 -- 172.21.15.3:0/806004566 shutdown_connections 2024-05-09T06:37:04.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.247+0000 7f876285d640 1 -- 172.21.15.3:0/806004566 wait complete. 2024-05-09T06:37:04.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.248+0000 7f876285d640 1 Processor -- start 2024-05-09T06:37:04.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.248+0000 7f876285d640 1 -- start start 2024-05-09T06:37:04.231 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.249+0000 7f876285d640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f875c06b250 0x7f875c15c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:04.232 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.249+0000 7f876285d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f875c15e0b0 0x7f875c1f3be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:04.232 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.249+0000 7f876285d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f875c15e490 0x7f875c15f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:04.232 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.249+0000 7f876285d640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f875c06da70 con 0x7f875c15e0b0 2024-05-09T06:37:04.232 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.249+0000 7f876285d640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f875c06d8f0 con 0x7f875c15e490 2024-05-09T06:37:04.232 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.249+0000 7f876285d640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f875c162ad0 con 0x7f875c06b250 2024-05-09T06:37:04.232 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.249+0000 7f875b7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f875c15e0b0 0x7f875c1f3be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:04.232 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.249+0000 7f875b7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f875c15e0b0 0x7f875c1f3be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:39478/0 (socket says 172.21.15.3:39478) 2024-05-09T06:37:04.232 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.249+0000 7f875b7fe640 1 -- 172.21.15.3:0/2586909082 learned_addr learned my addr 172.21.15.3:0/2586909082 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:37:04.232 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.249+0000 7f875bfff640 1 --2- 172.21.15.3:0/2586909082 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f875c06b250 0x7f875c15c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:04.232 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.249+0000 7f8760dfa640 1 --2- 172.21.15.3:0/2586909082 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f875c15e490 0x7f875c15f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:04.232 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.249+0000 7f875b7fe640 1 -- 172.21.15.3:0/2586909082 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f875c06b250 msgr2=0x7f875c15c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:04.232 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.249+0000 7f875b7fe640 1 --2- 172.21.15.3:0/2586909082 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f875c06b250 0x7f875c15c8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:04.232 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.249+0000 7f875b7fe640 1 -- 172.21.15.3:0/2586909082 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f875c15e490 msgr2=0x7f875c15f4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:04.232 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.249+0000 7f875b7fe640 1 --2- 172.21.15.3:0/2586909082 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f875c15e490 0x7f875c15f4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:04.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.249+0000 7f875b7fe640 1 -- 172.21.15.3:0/2586909082 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f875c1f46a0 con 0x7f875c15e0b0 2024-05-09T06:37:04.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.249+0000 7f8760dfa640 1 --2- 172.21.15.3:0/2586909082 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f875c15e490 0x7f875c15f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:37:04.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.249+0000 7f875b7fe640 1 --2- 172.21.15.3:0/2586909082 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f875c15e0b0 0x7f875c1f3be0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f875000d7d0 tx=0x7f875000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:04.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.250+0000 7f8758ff9640 1 -- 172.21.15.3:0/2586909082 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f875000b6c0 con 0x7f875c15e0b0 2024-05-09T06:37:04.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.250+0000 7f8758ff9640 1 -- 172.21.15.3:0/2586909082 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8750019070 con 0x7f875c15e0b0 2024-05-09T06:37:04.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.250+0000 7f876285d640 1 -- 172.21.15.3:0/2586909082 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f875c1f4980 con 0x7f875c15e0b0 2024-05-09T06:37:04.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.250+0000 7f8758ff9640 1 -- 172.21.15.3:0/2586909082 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f875000bac0 con 0x7f875c15e0b0 2024-05-09T06:37:04.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.250+0000 7f876285d640 1 -- 172.21.15.3:0/2586909082 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f875c1f5200 con 0x7f875c15e0b0 2024-05-09T06:37:04.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.251+0000 7f8758ff9640 1 -- 172.21.15.3:0/2586909082 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f875000bc60 con 0x7f875c15e0b0 2024-05-09T06:37:04.241 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.252+0000 7f8758ff9640 1 --2- 172.21.15.3:0/2586909082 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f874806be30 0x7f874806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:04.242 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.252+0000 7f8758ff9640 1 -- 172.21.15.3:0/2586909082 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f875008b320 con 0x7f875c15e0b0 2024-05-09T06:37:04.242 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.252+0000 7f875bfff640 1 --2- 172.21.15.3:0/2586909082 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f874806be30 0x7f874806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:04.242 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.252+0000 7f875bfff640 1 --2- 172.21.15.3:0/2586909082 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f874806be30 0x7f874806e2f0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f875c0af8c0 tx=0x7f874c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:04.242 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.253+0000 7f876556f740 1 Processor -- start 2024-05-09T06:37:04.242 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.253+0000 7f876556f740 1 -- start start 2024-05-09T06:37:04.242 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.253+0000 7f876556f740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56434f9db670 0x56434f9dba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:04.242 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.253+0000 7f876556f740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56434f9f8f10 0x56434f9f9460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:04.242 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.253+0000 7f876556f740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56434f9b1350 0x56434f9f75d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:04.242 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.253+0000 7f876556f740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x56434f9144d0 con 0x56434f9db670 2024-05-09T06:37:04.242 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.253+0000 7f876556f740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x56434f8bc3e0 con 0x56434f9f8f10 2024-05-09T06:37:04.242 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.253+0000 7f876556f740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x56434f910490 con 0x56434f9b1350 2024-05-09T06:37:04.242 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.253+0000 7f875bfff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56434f9db670 0x56434f9dba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:04.242 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.253+0000 7f875bfff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56434f9db670 0x56434f9dba50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:39490/0 (socket says 172.21.15.3:39490) 2024-05-09T06:37:04.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.253+0000 7f875bfff640 1 -- 172.21.15.3:0/380329114 learned_addr learned my addr 172.21.15.3:0/380329114 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:37:04.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.254+0000 7f8760dfa640 1 --2- 172.21.15.3:0/380329114 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56434f9b1350 0x56434f9f75d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:04.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.254+0000 7f875b7fe640 1 --2- 172.21.15.3:0/380329114 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56434f9f8f10 0x56434f9f9460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:04.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.254+0000 7f875bfff640 1 -- 172.21.15.3:0/380329114 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56434f9b1350 msgr2=0x56434f9f75d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:37:04.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.254+0000 7f875bfff640 1 --2- 172.21.15.3:0/380329114 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56434f9b1350 0x56434f9f75d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:04.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.254+0000 7f875bfff640 1 -- 172.21.15.3:0/380329114 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56434f9f8f10 msgr2=0x56434f9f9460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:37:04.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.254+0000 7f875bfff640 1 --2- 172.21.15.3:0/380329114 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56434f9f8f10 0x56434f9f9460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:04.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.254+0000 7f875bfff640 1 -- 172.21.15.3:0/380329114 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56434f6282d0 con 0x56434f9db670 2024-05-09T06:37:04.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.254+0000 7f875b7fe640 1 --2- 172.21.15.3:0/380329114 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56434f9f8f10 0x56434f9f9460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:37:04.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.254+0000 7f875bfff640 1 --2- 172.21.15.3:0/380329114 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56434f9db670 0x56434f9dba50 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f874c00e7e0 tx=0x7f874c011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=beee2c9d11b04f11 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:04.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.254+0000 7f8741ffb640 1 -- 172.21.15.3:0/380329114 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f874c025070 con 0x56434f9db670 2024-05-09T06:37:04.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.254+0000 7f8741ffb640 1 -- 172.21.15.3:0/380329114 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f874c0159b0 con 0x56434f9db670 2024-05-09T06:37:04.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.254+0000 7f8741ffb640 1 -- 172.21.15.3:0/380329114 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f874c015d60 con 0x56434f9db670 2024-05-09T06:37:04.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.255+0000 7f876556f740 1 -- 172.21.15.3:0/380329114 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56434f9db670 msgr2=0x56434f9dba50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:04.244 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.255+0000 7f876556f740 1 --2- 172.21.15.3:0/380329114 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56434f9db670 0x56434f9dba50 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f874c00e7e0 tx=0x7f874c011000 comp rx=0 tx=0).stop 2024-05-09T06:37:04.244 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.255+0000 7f876556f740 1 -- 172.21.15.3:0/380329114 shutdown_connections 2024-05-09T06:37:04.244 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.255+0000 7f876556f740 1 --2- 172.21.15.3:0/380329114 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56434f9b1350 0x56434f9f75d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:04.244 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.255+0000 7f876556f740 1 --2- 172.21.15.3:0/380329114 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56434f9f8f10 0x56434f9f9460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:04.244 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.255+0000 7f876556f740 1 --2- 172.21.15.3:0/380329114 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56434f9db670 0x56434f9dba50 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:04.244 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.255+0000 7f876556f740 1 -- 172.21.15.3:0/380329114 >> 172.21.15.3:0/380329114 conn(0x56434f9adc80 msgr2=0x56434f9f6480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:04.244 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.255+0000 7f876556f740 1 -- 172.21.15.3:0/380329114 shutdown_connections 2024-05-09T06:37:04.244 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.255+0000 7f876556f740 1 -- 172.21.15.3:0/380329114 wait complete. 2024-05-09T06:37:04.244 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.255+0000 7f876556f740 1 -- start start 2024-05-09T06:37:04.244 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.256+0000 7f876556f740 1 Processor -- start 2024-05-09T06:37:04.244 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.256+0000 7f876556f740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56434f9db670 0x56434f9f3930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:04.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.256+0000 7f876556f740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56434f9f8f10 0x56434f9edde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:04.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.256+0000 7f876556f740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56434fa09c90 0x56434fa0a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:04.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.256+0000 7f876556f740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x56434f9b20f0 con 0x56434f9f8f10 2024-05-09T06:37:04.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.256+0000 7f876556f740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x56434f8cf8e0 con 0x56434fa09c90 2024-05-09T06:37:04.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.256+0000 7f876556f740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x56434f9abbb0 con 0x56434f9db670 2024-05-09T06:37:04.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.256+0000 7f876556f740 10 client.0 _ll_register_callbacks cb 0x56434f89c9c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T06:37:04.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.256+0000 7f875b7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56434f9f8f10 0x56434f9edde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:04.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.256+0000 7f875b7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56434f9f8f10 0x56434f9edde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:39506/0 (socket says 172.21.15.3:39506) 2024-05-09T06:37:04.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.256+0000 7f875b7fe640 1 -- 172.21.15.3:0/312774578 learned_addr learned my addr 172.21.15.3:0/312774578 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:37:04.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.257+0000 7f8760dfa640 1 --2- 172.21.15.3:0/312774578 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56434fa09c90 0x56434fa0a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:04.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.257+0000 7f875bfff640 1 --2- 172.21.15.3:0/312774578 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56434f9db670 0x56434f9f3930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:04.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.257+0000 7f875b7fe640 1 -- 172.21.15.3:0/312774578 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56434f9db670 msgr2=0x56434f9f3930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:37:04.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.257+0000 7f875b7fe640 1 --2- 172.21.15.3:0/312774578 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56434f9db670 0x56434f9f3930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:04.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.257+0000 7f875b7fe640 1 -- 172.21.15.3:0/312774578 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56434fa09c90 msgr2=0x56434fa0a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:37:04.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.257+0000 7f875b7fe640 1 --2- 172.21.15.3:0/312774578 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56434fa09c90 0x56434fa0a0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:04.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.257+0000 7f875b7fe640 1 -- 172.21.15.3:0/312774578 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56434f6f26b0 con 0x56434f9f8f10 2024-05-09T06:37:04.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.257+0000 7f875bfff640 1 --2- 172.21.15.3:0/312774578 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56434f9db670 0x56434f9f3930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:37:04.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.257+0000 7f875b7fe640 1 --2- 172.21.15.3:0/312774578 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56434f9f8f10 0x56434f9edde0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f8750060050 tx=0x7f8750059d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=daa5da7a92c147ad server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:04.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.257+0000 7f873affd640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-05-09T06:37:04.247 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.257+0000 7f873affd640 1 -- 172.21.15.3:0/312774578 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87500621b0 con 0x56434f9f8f10 2024-05-09T06:37:04.247 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.257+0000 7f873affd640 1 -- 172.21.15.3:0/312774578 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f8750055c40 con 0x56434f9f8f10 2024-05-09T06:37:04.247 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.257+0000 7f876556f740 1 -- 172.21.15.3:0/312774578 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f874c0159b0 con 0x56434f9f8f10 2024-05-09T06:37:04.247 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.257+0000 7f873affd640 1 -- 172.21.15.3:0/312774578 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f8750058a20 con 0x56434f9f8f10 2024-05-09T06:37:04.247 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.257+0000 7f876556f740 10 client.5268 fetch_fsmap learned FSMap version 17 2024-05-09T06:37:04.247 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.257+0000 7f876556f740 1 -- 172.21.15.3:0/312774578 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) -- 0x56434f9ef4d0 con 0x56434f9f8f10 2024-05-09T06:37:04.247 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.257+0000 7f873affd640 1 -- 172.21.15.3:0/312774578 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f874c0159b0 con 0x56434f9f8f10 2024-05-09T06:37:04.247 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.257+0000 7f873affd640 1 -- 172.21.15.3:0/312774578 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 17) ==== 3494+0+0 (secure 0 0 0) 0x7f8750013070 con 0x56434f9f8f10 2024-05-09T06:37:04.247 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.257+0000 7f876556f740 10 client.5268 fetch_fsmap finished waiting for FSMap version 17 2024-05-09T06:37:04.247 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.257+0000 7f876556f740 10 client.5268 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-09T06:37:04.247 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.257+0000 7f876556f740 20 client.5268 populate_metadata read hostname 'smithi003' 2024-05-09T06:37:04.247 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.257+0000 7f876556f740 1 --2- 172.21.15.3:0/312774578 >> [v2:172.21.15.3:6833/2138710335,v1:172.21.15.3:6835/2138710335] conn(0x56434f9cf5a0 0x56434f9f57e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:04.247 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.257+0000 7f876556f740 4 client.5268 mds_command: new command op to 5103 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T06:37:04.247 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.257+0000 7f876556f740 1 -- 172.21.15.3:0/312774578 --> [v2:172.21.15.3:6833/2138710335,v1:172.21.15.3:6835/2138710335] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56434f921c00 con 0x56434f9cf5a0 2024-05-09T06:37:04.247 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.258+0000 7f875bfff640 1 --2- 172.21.15.3:0/312774578 >> [v2:172.21.15.3:6833/2138710335,v1:172.21.15.3:6835/2138710335] conn(0x56434f9cf5a0 0x56434f9f57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:04.248 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.258+0000 7f875bfff640 1 --2- 172.21.15.3:0/312774578 >> [v2:172.21.15.3:6833/2138710335,v1:172.21.15.3:6835/2138710335] conn(0x56434f9cf5a0 0x56434f9f57e0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a6da31b6e2857a97 server_cookie=5eca070e4803dd85 in_seq=0 out_seq=0 2024-05-09T06:37:04.248 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.258+0000 7f873affd640 10 client.5268 ms_handle_connect on v2:172.21.15.3:6833/2138710335 2024-05-09T06:37:04.248 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.258+0000 7f873affd640 1 -- 172.21.15.3:0/312774578 <== mds.0 v2:172.21.15.3:6833/2138710335 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56434f921c00 con 0x56434f9cf5a0 2024-05-09T06:37:04.248 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.259+0000 7f873affd640 10 client.5268 handle_command_reply: tid=0 2024-05-09T06:37:04.248 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.259+0000 7f876556f740 1 client.5268 shutdown 2024-05-09T06:37:04.248 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.259+0000 7f876556f740 1 -- 172.21.15.3:0/312774578 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56434f9f8f10 msgr2=0x56434f9edde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:04.248 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.259+0000 7f876556f740 1 --2- 172.21.15.3:0/312774578 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56434f9f8f10 0x56434f9edde0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f8750060050 tx=0x7f8750059d90 comp rx=0 tx=0).stop 2024-05-09T06:37:04.248 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.259+0000 7f876556f740 1 -- 172.21.15.3:0/312774578 shutdown_connections 2024-05-09T06:37:04.248 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.259+0000 7f876556f740 1 --2- 172.21.15.3:0/312774578 >> [v2:172.21.15.3:6833/2138710335,v1:172.21.15.3:6835/2138710335] conn(0x56434f9cf5a0 0x56434f9f57e0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:04.248 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.259+0000 7f876556f740 1 --2- 172.21.15.3:0/312774578 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56434fa09c90 0x56434fa0a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:04.248 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.259+0000 7f876556f740 1 --2- 172.21.15.3:0/312774578 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56434f9f8f10 0x56434f9edde0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:04.248 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.259+0000 7f873affd640 0 client.5268 ms_handle_reset on v2:172.21.15.3:6833/2138710335 2024-05-09T06:37:04.248 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.259+0000 7f876556f740 1 --2- 172.21.15.3:0/312774578 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56434f9db670 0x56434f9f3930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:04.248 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.259+0000 7f876556f740 1 -- 172.21.15.3:0/312774578 >> 172.21.15.3:0/312774578 conn(0x56434f9b1350 msgr2=0x56434f9fa290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:04.248 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.259+0000 7f876556f740 1 -- 172.21.15.3:0/312774578 shutdown_connections 2024-05-09T06:37:04.249 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.259+0000 7f876556f740 1 -- 172.21.15.3:0/312774578 wait complete. 2024-05-09T06:37:04.249 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.259+0000 7f876556f740 20 client.5268 trim_cache size 0 max 16384 2024-05-09T06:37:04.258 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.277+0000 7f876556f740 1 Processor -- start 2024-05-09T06:37:04.258 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.277+0000 7f876556f740 1 -- start start 2024-05-09T06:37:04.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.278+0000 7f876556f740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56434f9db670 0x56434f9f57b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:04.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.278+0000 7f876556f740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56434f9f8f10 0x56434f9b1920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:04.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.278+0000 7f876556f740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56434fa09c90 0x56434fa193a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:04.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.278+0000 7f876556f740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f875c1ff890 con 0x56434f9f8f10 2024-05-09T06:37:04.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.278+0000 7f876556f740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f875c1f62c0 con 0x56434fa09c90 2024-05-09T06:37:04.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.278+0000 7f876556f740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f875c1f6910 con 0x56434f9db670 2024-05-09T06:37:04.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.278+0000 7f875b7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56434f9f8f10 0x56434f9b1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:04.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.278+0000 7f875b7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56434f9f8f10 0x56434f9b1920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:39518/0 (socket says 172.21.15.3:39518) 2024-05-09T06:37:04.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.278+0000 7f875b7fe640 1 -- 172.21.15.3:0/586037363 learned_addr learned my addr 172.21.15.3:0/586037363 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:37:04.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.278+0000 7f875bfff640 1 --2- 172.21.15.3:0/586037363 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56434f9db670 0x56434f9f57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:04.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.278+0000 7f8760dfa640 1 --2- 172.21.15.3:0/586037363 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56434fa09c90 0x56434fa193a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:04.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.278+0000 7f875b7fe640 1 -- 172.21.15.3:0/586037363 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56434f9db670 msgr2=0x56434f9f57b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:37:04.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.278+0000 7f875b7fe640 1 --2- 172.21.15.3:0/586037363 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56434f9db670 0x56434f9f57b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:04.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.278+0000 7f875b7fe640 1 -- 172.21.15.3:0/586037363 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56434fa09c90 msgr2=0x56434fa193a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:37:04.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.278+0000 7f875b7fe640 1 --2- 172.21.15.3:0/586037363 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56434fa09c90 0x56434fa193a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:04.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.278+0000 7f875b7fe640 1 -- 172.21.15.3:0/586037363 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56434fa05c10 con 0x56434f9f8f10 2024-05-09T06:37:04.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.278+0000 7f875bfff640 1 --2- 172.21.15.3:0/586037363 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56434f9db670 0x56434f9f57b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:37:04.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.278+0000 7f8760dfa640 1 --2- 172.21.15.3:0/586037363 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56434fa09c90 0x56434fa193a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:37:04.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.279+0000 7f875b7fe640 1 --2- 172.21.15.3:0/586037363 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56434f9f8f10 0x56434f9b1920 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f8750061c80 tx=0x7f875008b050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22b972996f401224 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:04.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.279+0000 7f87417fa640 1 -- 172.21.15.3:0/586037363 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87500593a0 con 0x56434f9f8f10 2024-05-09T06:37:04.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.279+0000 7f87417fa640 1 -- 172.21.15.3:0/586037363 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8750055c40 con 0x56434f9f8f10 2024-05-09T06:37:04.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.279+0000 7f87417fa640 1 -- 172.21.15.3:0/586037363 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f875005eb10 con 0x56434f9f8f10 2024-05-09T06:37:04.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.279+0000 7f876556f740 1 -- 172.21.15.3:0/586037363 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56434f9f8f10 msgr2=0x56434f9b1920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:04.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.279+0000 7f876556f740 1 --2- 172.21.15.3:0/586037363 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56434f9f8f10 0x56434f9b1920 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f8750061c80 tx=0x7f875008b050 comp rx=0 tx=0).stop 2024-05-09T06:37:04.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.279+0000 7f876556f740 1 -- 172.21.15.3:0/586037363 shutdown_connections 2024-05-09T06:37:04.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.279+0000 7f876556f740 1 --2- 172.21.15.3:0/586037363 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56434fa09c90 0x56434fa193a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:04.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.279+0000 7f876556f740 1 --2- 172.21.15.3:0/586037363 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56434f9f8f10 0x56434f9b1920 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:04.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.279+0000 7f876556f740 1 --2- 172.21.15.3:0/586037363 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56434f9db670 0x56434f9f57b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:04.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.279+0000 7f876556f740 1 -- 172.21.15.3:0/586037363 >> 172.21.15.3:0/586037363 conn(0x56434f9b1350 msgr2=0x56434f9f3740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:04.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.279+0000 7f876556f740 1 -- 172.21.15.3:0/586037363 shutdown_connections 2024-05-09T06:37:04.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.279+0000 7f876556f740 1 -- 172.21.15.3:0/586037363 wait complete. 2024-05-09T06:37:04.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.280+0000 7f876556f740 1 -- start start 2024-05-09T06:37:04.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.280+0000 7f876556f740 1 Processor -- start 2024-05-09T06:37:04.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.281+0000 7f876556f740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56434f9db670 0x56434fa6fa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:04.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.281+0000 7f876556f740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56434f9f8f10 0x56434f9ecff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:04.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.281+0000 7f876556f740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56434fa09c90 0x56434f9ed530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:04.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.281+0000 7f876556f740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x56434f9ee460 con 0x56434f9db670 2024-05-09T06:37:04.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.281+0000 7f876556f740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x56434f9f6770 con 0x56434f9f8f10 2024-05-09T06:37:04.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.281+0000 7f876556f740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x56434f9cf9f0 con 0x56434fa09c90 2024-05-09T06:37:04.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.281+0000 7f876556f740 10 client.0 _ll_register_callbacks cb 0x7f87500558f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T06:37:04.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.281+0000 7f875bfff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56434f9db670 0x56434fa6fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:04.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.281+0000 7f875bfff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56434f9db670 0x56434fa6fa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:39534/0 (socket says 172.21.15.3:39534) 2024-05-09T06:37:04.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.281+0000 7f875bfff640 1 -- 172.21.15.3:0/3575718024 learned_addr learned my addr 172.21.15.3:0/3575718024 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:37:04.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.281+0000 7f875bfff640 1 -- 172.21.15.3:0/3575718024 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56434fa09c90 msgr2=0x56434f9ed530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:37:04.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.281+0000 7f8760dfa640 1 --2- 172.21.15.3:0/3575718024 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56434fa09c90 0x56434f9ed530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:04.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.281+0000 7f875b7fe640 1 --2- 172.21.15.3:0/3575718024 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56434f9f8f10 0x56434f9ecff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:04.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.281+0000 7f875bfff640 1 --2- 172.21.15.3:0/3575718024 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56434fa09c90 0x56434f9ed530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:04.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.281+0000 7f875bfff640 1 -- 172.21.15.3:0/3575718024 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56434f9f8f10 msgr2=0x56434f9ecff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:37:04.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.281+0000 7f875bfff640 1 --2- 172.21.15.3:0/3575718024 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56434f9f8f10 0x56434f9ecff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:04.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.281+0000 7f875bfff640 1 -- 172.21.15.3:0/3575718024 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56434f9b1ea0 con 0x56434f9db670 2024-05-09T06:37:04.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.281+0000 7f8760dfa640 1 --2- 172.21.15.3:0/3575718024 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56434fa09c90 0x56434f9ed530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:37:04.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.281+0000 7f875b7fe640 1 --2- 172.21.15.3:0/3575718024 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56434f9f8f10 0x56434f9ecff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:37:04.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.282+0000 7f875bfff640 1 --2- 172.21.15.3:0/3575718024 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56434f9db670 0x56434fa6fa30 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f874c015800 tx=0x7f874c0036f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9db754fae77d6ee0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:04.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.282+0000 7f873affd640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-05-09T06:37:04.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.282+0000 7f873affd640 1 -- 172.21.15.3:0/3575718024 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f874c015e60 con 0x56434f9db670 2024-05-09T06:37:04.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.282+0000 7f873affd640 1 -- 172.21.15.3:0/3575718024 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f874c02d200 con 0x56434f9db670 2024-05-09T06:37:04.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.282+0000 7f876556f740 1 -- 172.21.15.3:0/3575718024 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8750055c40 con 0x56434f9db670 2024-05-09T06:37:04.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.282+0000 7f873affd640 1 -- 172.21.15.3:0/3575718024 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f874c038050 con 0x56434f9db670 2024-05-09T06:37:04.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.282+0000 7f876556f740 10 client.5274 fetch_fsmap learned FSMap version 17 2024-05-09T06:37:04.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.282+0000 7f876556f740 1 -- 172.21.15.3:0/3575718024 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) -- 0x56434f9b0120 con 0x56434f9db670 2024-05-09T06:37:04.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.282+0000 7f873affd640 1 -- 172.21.15.3:0/3575718024 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8750055c40 con 0x56434f9db670 2024-05-09T06:37:04.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.282+0000 7f873affd640 1 -- 172.21.15.3:0/3575718024 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 17) ==== 3494+0+0 (secure 0 0 0) 0x7f874c03b020 con 0x56434f9db670 2024-05-09T06:37:04.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.282+0000 7f876556f740 10 client.5274 fetch_fsmap finished waiting for FSMap version 17 2024-05-09T06:37:04.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.282+0000 7f876556f740 10 client.5274 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-09T06:37:04.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.282+0000 7f876556f740 20 client.5274 populate_metadata read hostname 'smithi003' 2024-05-09T06:37:04.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.282+0000 7f876556f740 1 --2- 172.21.15.3:0/3575718024 >> [v2:172.21.15.3:6833/2138710335,v1:172.21.15.3:6835/2138710335] conn(0x56434f9afce0 0x56434f9aa5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:04.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.282+0000 7f876556f740 4 client.5274 mds_command: new command op to 5103 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-09T06:37:04.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.282+0000 7f876556f740 1 -- 172.21.15.3:0/3575718024 --> [v2:172.21.15.3:6833/2138710335,v1:172.21.15.3:6835/2138710335] -- command(tid 0: {"prefix": "status"}) -- 0x56434f91bf50 con 0x56434f9afce0 2024-05-09T06:37:04.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.282+0000 7f875b7fe640 1 --2- 172.21.15.3:0/3575718024 >> [v2:172.21.15.3:6833/2138710335,v1:172.21.15.3:6835/2138710335] conn(0x56434f9afce0 0x56434f9aa5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:04.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.283+0000 7f873affd640 10 client.5274 ms_handle_connect on v2:172.21.15.3:6833/2138710335 2024-05-09T06:37:04.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.283+0000 7f875b7fe640 1 --2- 172.21.15.3:0/3575718024 >> [v2:172.21.15.3:6833/2138710335,v1:172.21.15.3:6835/2138710335] conn(0x56434f9afce0 0x56434f9aa5f0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8e84867f03243b71 server_cookie=fc4d06d658c79270 in_seq=0 out_seq=0 2024-05-09T06:37:04.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.283+0000 7f873affd640 1 -- 172.21.15.3:0/3575718024 <== mds.0 v2:172.21.15.3:6833/2138710335 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x56434f91bf50 con 0x56434f9afce0 2024-05-09T06:37:04.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.283+0000 7f873affd640 10 client.5274 handle_command_reply: tid=0 2024-05-09T06:37:04.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.283+0000 7f876556f740 1 client.5274 shutdown 2024-05-09T06:37:04.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.283+0000 7f876556f740 1 -- 172.21.15.3:0/3575718024 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56434f9db670 msgr2=0x56434fa6fa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:04.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.283+0000 7f876556f740 1 --2- 172.21.15.3:0/3575718024 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56434f9db670 0x56434fa6fa30 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f874c015800 tx=0x7f874c0036f0 comp rx=0 tx=0).stop 2024-05-09T06:37:04.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.283+0000 7f876556f740 1 -- 172.21.15.3:0/3575718024 shutdown_connections 2024-05-09T06:37:04.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.283+0000 7f876556f740 1 --2- 172.21.15.3:0/3575718024 >> [v2:172.21.15.3:6833/2138710335,v1:172.21.15.3:6835/2138710335] conn(0x56434f9afce0 0x56434f9aa5f0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:04.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.283+0000 7f876556f740 1 --2- 172.21.15.3:0/3575718024 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56434fa09c90 0x56434f9ed530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:04.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.283+0000 7f876556f740 1 --2- 172.21.15.3:0/3575718024 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56434f9f8f10 0x56434f9ecff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:04.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.283+0000 7f873affd640 0 client.5274 ms_handle_reset on v2:172.21.15.3:6833/2138710335 2024-05-09T06:37:04.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.283+0000 7f876556f740 1 --2- 172.21.15.3:0/3575718024 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56434f9db670 0x56434fa6fa30 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:04.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.283+0000 7f876556f740 1 -- 172.21.15.3:0/3575718024 >> 172.21.15.3:0/3575718024 conn(0x56434f9b1350 msgr2=0x56434f9f3740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:04.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.283+0000 7f876556f740 1 -- 172.21.15.3:0/3575718024 shutdown_connections 2024-05-09T06:37:04.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.283+0000 7f876556f740 1 -- 172.21.15.3:0/3575718024 wait complete. 2024-05-09T06:37:04.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.283+0000 7f876556f740 20 client.5274 trim_cache size 0 max 16384 2024-05-09T06:37:04.272 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-05-09T06:37:04.272 INFO:teuthology.orchestra.run.smithi003.stdout: "cluster_fsid": "16ed737d-2665-431a-8ac6-ef242be3975a", 2024-05-09T06:37:04.272 INFO:teuthology.orchestra.run.smithi003.stdout: "whoami": 0, 2024-05-09T06:37:04.272 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 5103, 2024-05-09T06:37:04.272 INFO:teuthology.orchestra.run.smithi003.stdout: "want_state": "up:active", 2024-05-09T06:37:04.272 INFO:teuthology.orchestra.run.smithi003.stdout: "state": "up:active", 2024-05-09T06:37:04.272 INFO:teuthology.orchestra.run.smithi003.stdout: "fs_name": "cephfs", 2024-05-09T06:37:04.272 INFO:teuthology.orchestra.run.smithi003.stdout: "rank_uptime": 5.7286195290000004, 2024-05-09T06:37:04.273 INFO:teuthology.orchestra.run.smithi003.stdout: "mdsmap_epoch": 17, 2024-05-09T06:37:04.273 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch": 33, 2024-05-09T06:37:04.273 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch_barrier": 33, 2024-05-09T06:37:04.273 INFO:teuthology.orchestra.run.smithi003.stdout: "uptime": 12.659095002000001 2024-05-09T06:37:04.273 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-05-09T06:37:04.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.284+0000 7f876285d640 1 -- 172.21.15.3:0/2586909082 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f874806be30 msgr2=0x7f874806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:04.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.284+0000 7f876285d640 1 --2- 172.21.15.3:0/2586909082 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f874806be30 0x7f874806e2f0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f875c0af8c0 tx=0x7f874c009040 comp rx=0 tx=0).stop 2024-05-09T06:37:04.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.284+0000 7f876285d640 1 -- 172.21.15.3:0/2586909082 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f875c15e0b0 msgr2=0x7f875c1f3be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:04.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.284+0000 7f876285d640 1 --2- 172.21.15.3:0/2586909082 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f875c15e0b0 0x7f875c1f3be0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f875000d7d0 tx=0x7f875000dc90 comp rx=0 tx=0).stop 2024-05-09T06:37:04.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.284+0000 7f876285d640 1 -- 172.21.15.3:0/2586909082 shutdown_connections 2024-05-09T06:37:04.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.284+0000 7f876285d640 1 --2- 172.21.15.3:0/2586909082 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f874806be30 0x7f874806e2f0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:04.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.284+0000 7f876285d640 1 --2- 172.21.15.3:0/2586909082 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f875c15e490 0x7f875c15f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:04.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.284+0000 7f876285d640 1 --2- 172.21.15.3:0/2586909082 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f875c15e0b0 0x7f875c1f3be0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:04.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.284+0000 7f876285d640 1 --2- 172.21.15.3:0/2586909082 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f875c06b250 0x7f875c15c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:04.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.284+0000 7f876285d640 1 -- 172.21.15.3:0/2586909082 >> 172.21.15.3:0/2586909082 conn(0x7f875c0d07b0 msgr2=0x7f875c06e710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:04.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.284+0000 7f876285d640 1 -- 172.21.15.3:0/2586909082 shutdown_connections 2024-05-09T06:37:04.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.284+0000 7f876285d640 1 -- 172.21.15.3:0/2586909082 wait complete. 2024-05-09T06:37:04.276 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-09T06:37:04.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.399+0000 7f3fac3dc640 1 Processor -- start 2024-05-09T06:37:04.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.399+0000 7f3fac3dc640 1 -- start start 2024-05-09T06:37:04.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.400+0000 7f3fac3dc640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3fa40cc6f0 0x7f3fa40b4250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:04.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.400+0000 7f3fac3dc640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f3fa40ccad0 0x7f3fa415d160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:04.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.400+0000 7f3fac3dc640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3fa40b48e0 con 0x7f3fa40cd740 2024-05-09T06:37:04.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.400+0000 7f3fac3dc640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f3fa40b4bc0 con 0x7f3fa40ccad0 2024-05-09T06:37:04.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.400+0000 7f3fac3dc640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f3fa40b4ea0 con 0x7f3fa40cc6f0 2024-05-09T06:37:04.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.401+0000 7f3fa9977640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f3fa40ccad0 0x7f3fa415d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:04.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.401+0000 7f3faa178640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3fa40cc6f0 0x7f3fa40b4250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:04.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.401+0000 7f3faa979640 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7f3fa40cd740 0x7f3fa415fc60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.3:37700/0 (socket says 172.21.15.3:37700) 2024-05-09T06:37:04.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.401+0000 7f3faa979640 1 -- 172.21.15.3:0/3294667004 learned_addr learned my addr 172.21.15.3:0/3294667004 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-05-09T06:37:04.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.401+0000 7f3faa178640 1 -- 172.21.15.3:0/3294667004 >> v2:172.21.15.190:3300/0 conn(0x7f3fa40ccad0 msgr2=0x7f3fa415d160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:37:04.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.401+0000 7f3faa178640 1 --2- 172.21.15.3:0/3294667004 >> v2:172.21.15.190:3300/0 conn(0x7f3fa40ccad0 0x7f3fa415d160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:04.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.401+0000 7f3faa178640 1 -- 172.21.15.3:0/3294667004 >> v1:172.21.15.190:6789/0 conn(0x7f3fa40cd740 legacy=0x7f3fa415fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:04.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.401+0000 7f3faa178640 1 -- 172.21.15.3:0/3294667004 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fa40b5180 con 0x7f3fa40cc6f0 2024-05-09T06:37:04.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.402+0000 7f3faa178640 1 --2- 172.21.15.3:0/3294667004 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3fa40cc6f0 0x7f3fa40b4250 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f3fa00097a0 tx=0x7f3fa002a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cdb17a15c6c9ae31 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:04.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.402+0000 7f3fa9176640 1 -- 172.21.15.3:0/3294667004 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fa0037070 con 0x7f3fa40cc6f0 2024-05-09T06:37:04.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.402+0000 7f3fa9176640 1 -- 172.21.15.3:0/3294667004 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3fa0032440 con 0x7f3fa40cc6f0 2024-05-09T06:37:04.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.402+0000 7f3fa9176640 1 -- 172.21.15.3:0/3294667004 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fa0032810 con 0x7f3fa40cc6f0 2024-05-09T06:37:04.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.402+0000 7f3fac3dc640 1 -- 172.21.15.3:0/3294667004 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3fa40cc6f0 msgr2=0x7f3fa40b4250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:04.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.402+0000 7f3fac3dc640 1 --2- 172.21.15.3:0/3294667004 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3fa40cc6f0 0x7f3fa40b4250 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f3fa00097a0 tx=0x7f3fa002a180 comp rx=0 tx=0).stop 2024-05-09T06:37:04.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.403+0000 7f3fac3dc640 1 -- 172.21.15.3:0/3294667004 shutdown_connections 2024-05-09T06:37:04.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.403+0000 7f3fac3dc640 1 --2- 172.21.15.3:0/3294667004 >> v2:172.21.15.190:3300/0 conn(0x7f3fa40ccad0 0x7f3fa415d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:04.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.403+0000 7f3fac3dc640 1 --2- 172.21.15.3:0/3294667004 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3fa40cc6f0 0x7f3fa40b4250 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:04.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.403+0000 7f3fac3dc640 1 -- 172.21.15.3:0/3294667004 >> 172.21.15.3:0/3294667004 conn(0x7f3fa406b430 msgr2=0x7f3fa406b810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:04.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.403+0000 7f3fac3dc640 1 -- 172.21.15.3:0/3294667004 shutdown_connections 2024-05-09T06:37:04.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.403+0000 7f3fac3dc640 1 -- 172.21.15.3:0/3294667004 wait complete. 2024-05-09T06:37:04.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.404+0000 7f3fac3dc640 1 Processor -- start 2024-05-09T06:37:04.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.404+0000 7f3fac3dc640 1 -- start start 2024-05-09T06:37:04.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.405+0000 7f3fac3dc640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3fa40cc6f0 0x7f3fa4148490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:04.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.405+0000 7f3fac3dc640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3fa40ccad0 0x7f3fa4072540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:04.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.405+0000 7f3fac3dc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3fa40cd740 0x7f3fa415b140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:04.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.405+0000 7f3fac3dc640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f3fa406ebb0 con 0x7f3fa40cd740 2024-05-09T06:37:04.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.405+0000 7f3fac3dc640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f3fa406ea30 con 0x7f3fa40ccad0 2024-05-09T06:37:04.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.405+0000 7f3fac3dc640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f3fa406ed30 con 0x7f3fa40cc6f0 2024-05-09T06:37:04.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.405+0000 7f3faa979640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3fa40cd740 0x7f3fa415b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:04.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.405+0000 7f3faa979640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3fa40cd740 0x7f3fa415b140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:39548/0 (socket says 172.21.15.3:39548) 2024-05-09T06:37:04.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.405+0000 7f3faa979640 1 -- 172.21.15.3:0/396721017 learned_addr learned my addr 172.21.15.3:0/396721017 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:37:04.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.405+0000 7f3faa178640 1 --2- 172.21.15.3:0/396721017 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3fa40cc6f0 0x7f3fa4148490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:04.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.405+0000 7f3fa9977640 1 --2- 172.21.15.3:0/396721017 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3fa40ccad0 0x7f3fa4072540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:04.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.405+0000 7f3faa979640 1 -- 172.21.15.3:0/396721017 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3fa40cc6f0 msgr2=0x7f3fa4148490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:04.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.405+0000 7f3faa979640 1 --2- 172.21.15.3:0/396721017 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3fa40cc6f0 0x7f3fa4148490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:04.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.405+0000 7f3faa979640 1 -- 172.21.15.3:0/396721017 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3fa40ccad0 msgr2=0x7f3fa4072540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:04.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.405+0000 7f3faa979640 1 --2- 172.21.15.3:0/396721017 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3fa40ccad0 0x7f3fa4072540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:04.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.405+0000 7f3faa979640 1 -- 172.21.15.3:0/396721017 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fa41489d0 con 0x7f3fa40cd740 2024-05-09T06:37:04.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.405+0000 7f3faa979640 1 --2- 172.21.15.3:0/396721017 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3fa40cd740 0x7f3fa415b140 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f3f9c00cbf0 tx=0x7f3f9c00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:04.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.406+0000 7f3f9affd640 1 -- 172.21.15.3:0/396721017 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f9c017070 con 0x7f3fa40cd740 2024-05-09T06:37:04.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.406+0000 7f3f9affd640 1 -- 172.21.15.3:0/396721017 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f9c00ad40 con 0x7f3fa40cd740 2024-05-09T06:37:04.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.406+0000 7f3f9affd640 1 -- 172.21.15.3:0/396721017 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f9c012600 con 0x7f3fa40cd740 2024-05-09T06:37:04.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.406+0000 7f3fac3dc640 1 -- 172.21.15.3:0/396721017 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3fa4151ff0 con 0x7f3fa40cd740 2024-05-09T06:37:04.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.406+0000 7f3fac3dc640 1 -- 172.21.15.3:0/396721017 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3fa41525b0 con 0x7f3fa40cd740 2024-05-09T06:37:04.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.407+0000 7f3fac3dc640 1 -- 172.21.15.3:0/396721017 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3fa404f660 con 0x7f3fa40cd740 2024-05-09T06:37:04.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.407+0000 7f3f9affd640 1 -- 172.21.15.3:0/396721017 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3f9c01a070 con 0x7f3fa40cd740 2024-05-09T06:37:04.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.408+0000 7f3f9affd640 1 --2- 172.21.15.3:0/396721017 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f3f8806be30 0x7f3f8806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:04.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.408+0000 7f3f9affd640 1 -- 172.21.15.3:0/396721017 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f3f9c08aae0 con 0x7f3fa40cd740 2024-05-09T06:37:04.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.408+0000 7f3faa178640 1 --2- 172.21.15.3:0/396721017 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f3f8806be30 0x7f3f8806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:04.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.409+0000 7f3faa178640 1 --2- 172.21.15.3:0/396721017 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f3f8806be30 0x7f3f8806e2f0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f3fa0034000 tx=0x7f3fa002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:04.393 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.412+0000 7f3f9affd640 1 -- 172.21.15.3:0/396721017 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f3f9c058e40 con 0x7f3fa40cd740 2024-05-09T06:37:04.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.635+0000 7f3fac3dc640 1 -- 172.21.15.3:0/396721017 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f3fa40b1840 con 0x7f3fa40cd740 2024-05-09T06:37:04.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.636+0000 7f3f9affd640 1 -- 172.21.15.3:0/396721017 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) ==== 53+0+73 (secure 0 0 0) 0x7f3f9c05d140 con 0x7f3fa40cd740 2024-05-09T06:37:04.617 INFO:teuthology.orchestra.run.smithi003.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-09T06:37:04.619 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.638+0000 7f3fac3dc640 1 -- 172.21.15.3:0/396721017 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f3f8806be30 msgr2=0x7f3f8806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:04.619 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.638+0000 7f3fac3dc640 1 --2- 172.21.15.3:0/396721017 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f3f8806be30 0x7f3f8806e2f0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f3fa0034000 tx=0x7f3fa002e000 comp rx=0 tx=0).stop 2024-05-09T06:37:04.619 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.638+0000 7f3fac3dc640 1 -- 172.21.15.3:0/396721017 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3fa40cd740 msgr2=0x7f3fa415b140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:04.619 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.638+0000 7f3fac3dc640 1 --2- 172.21.15.3:0/396721017 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3fa40cd740 0x7f3fa415b140 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f3f9c00cbf0 tx=0x7f3f9c00a430 comp rx=0 tx=0).stop 2024-05-09T06:37:04.619 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.638+0000 7f3fac3dc640 1 -- 172.21.15.3:0/396721017 shutdown_connections 2024-05-09T06:37:04.622 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.638+0000 7f3fac3dc640 1 --2- 172.21.15.3:0/396721017 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f3f8806be30 0x7f3f8806e2f0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:04.622 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.638+0000 7f3fac3dc640 1 --2- 172.21.15.3:0/396721017 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3fa40cd740 0x7f3fa415b140 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:04.622 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.638+0000 7f3fac3dc640 1 --2- 172.21.15.3:0/396721017 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3fa40ccad0 0x7f3fa4072540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:04.622 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.638+0000 7f3fac3dc640 1 --2- 172.21.15.3:0/396721017 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3fa40cc6f0 0x7f3fa4148490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:04.622 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.638+0000 7f3fac3dc640 1 -- 172.21.15.3:0/396721017 >> 172.21.15.3:0/396721017 conn(0x7f3fa406b430 msgr2=0x7f3fa40cdbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:04.622 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.638+0000 7f3fac3dc640 1 -- 172.21.15.3:0/396721017 shutdown_connections 2024-05-09T06:37:04.622 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04.638+0000 7f3fac3dc640 1 -- 172.21.15.3:0/396721017 wait complete. 2024-05-09T06:37:04.629 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-05-09T06:37:04.629 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-09T06:37:04.630 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi003.front.sepia.ceph.com 2024-05-09T06:37:04.630 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-09T06:37:04.630 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-09T06:37:04.630 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-05-09T06:37:04.630 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-05-09T06:37:04.630 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-09T06:37:04.630 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-09T06:37:04.630 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:37:04.630 DEBUG:teuthology.orchestra.run.smithi003:> ip addr 2024-05-09T06:37:04.647 INFO:teuthology.orchestra.run.smithi003.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-09T06:37:04.647 INFO:teuthology.orchestra.run.smithi003.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-09T06:37:04.647 INFO:teuthology.orchestra.run.smithi003.stdout: inet 127.0.0.1/8 scope host lo 2024-05-09T06:37:04.647 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-05-09T06:37:04.647 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 ::1/128 scope host 2024-05-09T06:37:04.647 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-05-09T06:37:04.647 INFO:teuthology.orchestra.run.smithi003.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-09T06:37:04.647 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:6b:fd:ac brd ff:ff:ff:ff:ff:ff 2024-05-09T06:37:04.647 INFO:teuthology.orchestra.run.smithi003.stdout: altname enp7s0f0 2024-05-09T06:37:04.648 INFO:teuthology.orchestra.run.smithi003.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-09T06:37:04.648 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:6b:fd:ad brd ff:ff:ff:ff:ff:ff 2024-05-09T06:37:04.648 INFO:teuthology.orchestra.run.smithi003.stdout: altname enp7s0f1 2024-05-09T06:37:04.648 INFO:teuthology.orchestra.run.smithi003.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-09T06:37:04.648 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:bd:15:dc brd ff:ff:ff:ff:ff:ff 2024-05-09T06:37:04.648 INFO:teuthology.orchestra.run.smithi003.stdout: altname enp1s0f0 2024-05-09T06:37:04.648 INFO:teuthology.orchestra.run.smithi003.stdout: inet 172.21.15.3/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-05-09T06:37:04.648 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft 42651sec preferred_lft 42651sec 2024-05-09T06:37:04.648 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 fe80::ec4:7aff:febd:15dc/64 scope link noprefixroute 2024-05-09T06:37:04.648 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-05-09T06:37:04.648 INFO:teuthology.orchestra.run.smithi003.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-09T06:37:04.648 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:bd:15:dd brd ff:ff:ff:ff:ff:ff 2024-05-09T06:37:04.648 INFO:teuthology.orchestra.run.smithi003.stdout: altname enp1s0f1 2024-05-09T06:37:04.648 INFO:teuthology.orchestra.run.smithi003.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-09T06:37:04.649 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether ba:a3:85:d3:eb:6c brd ff:ff:ff:ff:ff:ff 2024-05-09T06:37:04.649 INFO:teuthology.orchestra.run.smithi003.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-09T06:37:04.649 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-05-09T06:37:04.649 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 fe80::7079:baff:fe57:1840/64 scope link 2024-05-09T06:37:04.649 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-05-09T06:37:04.649 INFO:teuthology.orchestra.run.smithi003.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-09T06:37:04.649 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether e2:25:5f:8c:a9:de brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-09T06:37:04.649 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 fe80::e025:5fff:fe8c:a9de/64 scope link 2024-05-09T06:37:04.649 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-05-09T06:37:04.649 INFO:teuthology.orchestra.run.smithi003.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-09T06:37:04.649 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether ba:a3:85:d3:eb:6c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-09T06:37:04.649 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 fe80::b8a3:85ff:fed3:eb6c/64 scope link 2024-05-09T06:37:04.649 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-05-09T06:37:04.650 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:37:04.650 DEBUG:teuthology.orchestra.run.smithi003:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-09T06:37:04.712 INFO:teuthology.orchestra.run.smithi003.stdout:1 2024-05-09T06:37:04.713 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:37:04.713 DEBUG:teuthology.orchestra.run.smithi003:> route 2024-05-09T06:37:04.771 INFO:teuthology.orchestra.run.smithi003.stdout:Kernel IP routing table 2024-05-09T06:37:04.771 INFO:teuthology.orchestra.run.smithi003.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-09T06:37:04.771 INFO:teuthology.orchestra.run.smithi003.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-05-09T06:37:04.771 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-05-09T06:37:04.771 INFO:teuthology.orchestra.run.smithi003.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-09T06:37:04.771 DEBUG:teuthology.orchestra.run.smithi003:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-09T06:37:04.772 DEBUG:teuthology.orchestra.run.smithi003:> set -e 2024-05-09T06:37:04.772 DEBUG:teuthology.orchestra.run.smithi003:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-05-09T06:37:04.772 DEBUG:teuthology.orchestra.run.smithi003:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-05-09T06:37:04.772 DEBUG:teuthology.orchestra.run.smithi003:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-05-09T06:37:04.772 DEBUG:teuthology.orchestra.run.smithi003:> ') 2024-05-09T06:37:04.862 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-09T06:37:04.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T06:37:04.902 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:37:04.903 DEBUG:teuthology.orchestra.run.smithi003:> ip netns list 2024-05-09T06:37:04.919 INFO:teuthology.orchestra.run.smithi003.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-05-09T06:37:04.919 INFO:teuthology.orchestra.run.smithi003.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-09T06:37:04.919 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-09T06:37:04.919 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:37:04.919 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip link set brx.0 up 2024-05-09T06:37:04.985 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-09T06:37:04.986 DEBUG:teuthology.orchestra.run.smithi003:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-09T06:37:05.042 INFO:teuthology.orchestra.run.smithi003.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-09T06:37:05.042 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-09T06:37:05.042 DEBUG:teuthology.orchestra.run.smithi003:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-09T06:37:05.098 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-05-09T06:37:05.098 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:37:05.098 DEBUG:teuthology.orchestra.run.smithi003:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-05-09T06:37:05.183 INFO:teuthology.orchestra.run.smithi003.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-05-09T06:37:05.183 INFO:teuthology.orchestra.run.smithi003.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-05-09T06:37:05.190 INFO:teuthology.orchestra.run.smithi003.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-05-09T06:37:05.191 INFO:teuthology.orchestra.run.smithi003.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-05-09T06:37:05.191 INFO:teuthology.orchestra.run.smithi003.stdout:mount.ceph: resolved to: "172.21.15.3:3300,172.21.15.190:3300,172.21.15.190:3301" 2024-05-09T06:37:05.191 INFO:teuthology.orchestra.run.smithi003.stdout:mount.ceph: trying mount with new device syntax: 0@16ed737d-2665-431a-8ac6-ef242be3975a.cephfs=/ 2024-05-09T06:37:05.191 INFO:teuthology.orchestra.run.smithi003.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.3:3300/172.21.15.190:3300/172.21.15.190:3301" will pass to kernel 2024-05-09T06:37:05.191 INFO:teuthology.orchestra.run.smithi003.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-05-09T06:37:05.191 INFO:teuthology.orchestra.run.smithi003.stdout: You just mounted a file system that supports labels which does not 2024-05-09T06:37:05.191 INFO:teuthology.orchestra.run.smithi003.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-05-09T06:37:05.191 INFO:teuthology.orchestra.run.smithi003.stdout: applications will generate AVC messages and not be allowed access to 2024-05-09T06:37:05.192 INFO:teuthology.orchestra.run.smithi003.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-05-09T06:37:05.193 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-09T06:37:05.193 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:37:05.193 DEBUG:teuthology.orchestra.run.smithi003:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-09T06:37:05.225 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-09T06:37:05.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:05 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-09T06:37:05.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T06:37:05.328 INFO:teuthology.orchestra.run.smithi003.stdout:client5280 2024-05-09T06:37:05.328 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-05-09T06:37:05.329 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T06:37:05.346 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-09T06:37:05.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:05.508+0000 7fd35d929640 1 Processor -- start 2024-05-09T06:37:05.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:05.508+0000 7fd35d929640 1 -- start start 2024-05-09T06:37:05.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:05.509+0000 7fd35d929640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd35815e480 0x7fd3580d1dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:05.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:05.509+0000 7fd35d929640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd3580d4e50 con 0x7fd3580d2ed0 2024-05-09T06:37:05.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:05.509+0000 7fd35d929640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd3580d0270 con 0x7fd3580d2310 2024-05-09T06:37:05.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:05.509+0000 7fd35d929640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fd3580cf1d0 con 0x7fd35815e480 2024-05-09T06:37:05.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:05.509+0000 7fd3577fe640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fd3580d2ed0 0x7fd3581683a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:58064/0 (socket says 172.21.15.3:58064) 2024-05-09T06:37:05.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:05.509+0000 7fd3577fe640 1 -- 172.21.15.3:0/3668759745 learned_addr learned my addr 172.21.15.3:0/3668759745 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-05-09T06:37:05.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:05.510+0000 7fd356ffd640 1 --2- 172.21.15.3:0/3668759745 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd35815e480 0x7fd3580d1dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:05.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:05.510+0000 7fd355ffb640 1 -- 172.21.15.3:0/3668759745 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3379790841 0 0) 0x7fd350001110 con 0x7fd3580d2ed0 2024-05-09T06:37:05.494 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:05.510+0000 7fd355ffb640 1 -- 172.21.15.3:0/3668759745 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd344003520 con 0x7fd3580d2ed0 2024-05-09T06:37:05.494 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:05.510+0000 7fd355ffb640 1 -- 172.21.15.3:0/3668759745 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2932254016 0 0) 0x7fd3580d0270 con 0x7fd3580d2310 2024-05-09T06:37:05.494 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:05.510+0000 7fd355ffb640 1 -- 172.21.15.3:0/3668759745 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd350001110 con 0x7fd3580d2310 2024-05-09T06:37:05.494 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:05.510+0000 7fd355ffb640 1 -- 172.21.15.3:0/3668759745 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2703219199 0 0) 0x7fd344003520 con 0x7fd3580d2ed0 2024-05-09T06:37:05.494 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:05.510+0000 7fd355ffb640 1 -- 172.21.15.3:0/3668759745 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd35815e480 msgr2=0x7fd3580d1dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:37:05.494 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:05.510+0000 7fd355ffb640 1 --2- 172.21.15.3:0/3668759745 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd35815e480 0x7fd3580d1dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:05.494 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:05.510+0000 7fd355ffb640 1 -- 172.21.15.3:0/3668759745 >> v1:172.21.15.190:6789/0 conn(0x7fd3580d2310 legacy=0x7fd3580d27c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:05.494 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:05.510+0000 7fd355ffb640 1 -- 172.21.15.3:0/3668759745 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3580cf4b0 con 0x7fd3580d2ed0 2024-05-09T06:37:05.494 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:05.510+0000 7fd355ffb640 1 -- 172.21.15.3:0/3668759745 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 563312031 0 0) 0x7fd350001840 con 0x7fd3580d2ed0 2024-05-09T06:37:05.494 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:05.510+0000 7fd355ffb640 1 -- 172.21.15.3:0/3668759745 >> v1:172.21.15.3:6789/0 conn(0x7fd3580d2ed0 legacy=0x7fd3581683a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:05.494 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:05.510+0000 7fd355ffb640 1 --2- 172.21.15.3:0/3668759745 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd344003b70 0x7fd344003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:05.495 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:05.510+0000 7fd355ffb640 1 --2- 172.21.15.3:0/3668759745 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd344004800 0x7fd344004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:05.495 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:05.510+0000 7fd355ffb640 1 --2- 172.21.15.3:0/3668759745 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd3440051c0 0x7fd34400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:05.495 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:05.510+0000 7fd355ffb640 1 -- 172.21.15.3:0/3668759745 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fd3580cf1d0 con 0x7fd344003b70 2024-05-09T06:37:05.495 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:05.510+0000 7fd355ffb640 1 -- 172.21.15.3:0/3668759745 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fd34400bfb0 con 0x7fd344004800 2024-05-09T06:37:05.495 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:05.510+0000 7fd355ffb640 1 -- 172.21.15.3:0/3668759745 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fd34400c1b0 con 0x7fd3440051c0 2024-05-09T06:37:05.495 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:05.510+0000 7fd356ffd640 1 --2- 172.21.15.3:0/3668759745 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd35815e480 0x7fd3580d1dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T06:37:05.495 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:05.510+0000 7fd356ffd640 1 --2- 172.21.15.3:0/3668759745 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd344003b70 0x7fd344003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:05.495 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:05.510+0000 7fd3567fc640 1 --2- 172.21.15.3:0/3668759745 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd344004800 0x7fd344004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:05.495 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:05.510+0000 7fd3577fe640 1 --2- 172.21.15.3:0/3668759745 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd3440051c0 0x7fd34400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:05.495 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:05.511+0000 7fd356ffd640 1 -- 172.21.15.3:0/3668759745 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd3440051c0 msgr2=0x7fd34400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:37:05.495 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:05.511+0000 7fd356ffd640 1 --2- 172.21.15.3:0/3668759745 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd3440051c0 0x7fd34400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:05.495 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:05.511+0000 7fd356ffd640 1 -- 172.21.15.3:0/3668759745 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd344004800 msgr2=0x7fd344004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:37:05.495 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:05.511+0000 7fd356ffd640 1 --2- 172.21.15.3:0/3668759745 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd344004800 0x7fd344004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:05.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:05.511+0000 7fd356ffd640 1 -- 172.21.15.3:0/3668759745 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd344003520 con 0x7fd344003b70 2024-05-09T06:37:05.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:05.511+0000 7fd356ffd640 1 --2- 172.21.15.3:0/3668759745 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd344003b70 0x7fd344003f70 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fd35808de10 tx=0x7fd344005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf14948d43fa2a98 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:05.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:05.511+0000 7fd355ffb640 1 -- 172.21.15.3:0/3668759745 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd348036070 con 0x7fd344003b70 2024-05-09T06:37:05.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:05.511+0000 7fd355ffb640 1 -- 172.21.15.3:0/3668759745 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd348003c50 con 0x7fd344003b70 2024-05-09T06:37:05.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:05.511+0000 7fd355ffb640 1 -- 172.21.15.3:0/3668759745 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd348009d40 con 0x7fd344003b70 2024-05-09T06:37:05.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:05.512+0000 7fd35d929640 1 -- 172.21.15.3:0/3668759745 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd344003b70 msgr2=0x7fd344003f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:05.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:05.512+0000 7fd35d929640 1 --2- 172.21.15.3:0/3668759745 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd344003b70 0x7fd344003f70 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fd35808de10 tx=0x7fd344005830 comp rx=0 tx=0).stop 2024-05-09T06:37:05.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:05.512+0000 7fd3577fe640 1 -- 172.21.15.3:0/3668759745 reap_dead start 2024-05-09T06:37:05.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:05.512+0000 7fd35d929640 1 -- 172.21.15.3:0/3668759745 shutdown_connections 2024-05-09T06:37:05.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:05.512+0000 7fd35d929640 1 -- 172.21.15.3:0/3668759745 >> 172.21.15.3:0/3668759745 conn(0x7fd35809d4d0 msgr2=0x7fd3580d4200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:05.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:05.512+0000 7fd35d929640 1 -- 172.21.15.3:0/3668759745 shutdown_connections 2024-05-09T06:37:05.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:05.512+0000 7fd35d929640 1 -- 172.21.15.3:0/3668759745 wait complete. 2024-05-09T06:37:05.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:05.513+0000 7fd35d929640 1 Processor -- start 2024-05-09T06:37:05.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:05.513+0000 7fd35d929640 1 -- start start 2024-05-09T06:37:05.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:05.514+0000 7fd35d929640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd344003b70 0x7fd3581f5420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:05.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:05.514+0000 7fd35d929640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd35815b810 0x7fd3580b3430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:05.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:05.514+0000 7fd35d929640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd3581f7e80 0x7fd35806cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:05.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:05.514+0000 7fd35d929640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fd3580cfd50 con 0x7fd344003b70 2024-05-09T06:37:05.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:05.514+0000 7fd35d929640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fd3580cfa70 con 0x7fd3581f7e80 2024-05-09T06:37:05.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:05.514+0000 7fd35d929640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fd3580ceec0 con 0x7fd35815b810 2024-05-09T06:37:05.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:05.514+0000 7fd356ffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd344003b70 0x7fd3581f5420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:05.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:05.514+0000 7fd356ffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd344003b70 0x7fd3581f5420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:39572/0 (socket says 172.21.15.3:39572) 2024-05-09T06:37:05.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:05.514+0000 7fd356ffd640 1 -- 172.21.15.3:0/1954375437 learned_addr learned my addr 172.21.15.3:0/1954375437 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:37:05.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:05.514+0000 7fd3577fe640 1 --2- 172.21.15.3:0/1954375437 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd3581f7e80 0x7fd35806cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:05.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:05.514+0000 7fd3567fc640 1 --2- 172.21.15.3:0/1954375437 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd35815b810 0x7fd3580b3430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:05.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:05.515+0000 7fd356ffd640 1 -- 172.21.15.3:0/1954375437 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd35815b810 msgr2=0x7fd3580b3430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:05.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:05.515+0000 7fd356ffd640 1 --2- 172.21.15.3:0/1954375437 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd35815b810 0x7fd3580b3430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:05.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:05.515+0000 7fd356ffd640 1 -- 172.21.15.3:0/1954375437 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd3581f7e80 msgr2=0x7fd35806cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:05.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:05.515+0000 7fd356ffd640 1 --2- 172.21.15.3:0/1954375437 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd3581f7e80 0x7fd35806cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:05.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:05.515+0000 7fd356ffd640 1 -- 172.21.15.3:0/1954375437 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd35815bc90 con 0x7fd344003b70 2024-05-09T06:37:05.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:05.515+0000 7fd3577fe640 1 --2- 172.21.15.3:0/1954375437 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd3581f7e80 0x7fd35806cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:37:05.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:05.515+0000 7fd356ffd640 1 --2- 172.21.15.3:0/1954375437 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd344003b70 0x7fd3581f5420 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fd34802af80 tx=0x7fd348009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:05.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:05.515+0000 7fd34ffff640 1 -- 172.21.15.3:0/1954375437 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd348041020 con 0x7fd344003b70 2024-05-09T06:37:05.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:05.515+0000 7fd34ffff640 1 -- 172.21.15.3:0/1954375437 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd348037040 con 0x7fd344003b70 2024-05-09T06:37:05.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:05.515+0000 7fd34ffff640 1 -- 172.21.15.3:0/1954375437 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd348036070 con 0x7fd344003b70 2024-05-09T06:37:05.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:05.515+0000 7fd35d929640 1 -- 172.21.15.3:0/1954375437 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3580b3970 con 0x7fd344003b70 2024-05-09T06:37:05.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:05.515+0000 7fd35d929640 1 -- 172.21.15.3:0/1954375437 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3581f8990 con 0x7fd344003b70 2024-05-09T06:37:05.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:05.516+0000 7fd35d929640 1 -- 172.21.15.3:0/1954375437 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd35804f660 con 0x7fd344003b70 2024-05-09T06:37:05.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:05.517+0000 7fd34ffff640 1 -- 172.21.15.3:0/1954375437 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd348044020 con 0x7fd344003b70 2024-05-09T06:37:05.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:05.517+0000 7fd34ffff640 1 --2- 172.21.15.3:0/1954375437 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fd33806be60 0x7fd33806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:05.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:05.517+0000 7fd34ffff640 1 -- 172.21.15.3:0/1954375437 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fd3480abef0 con 0x7fd344003b70 2024-05-09T06:37:05.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:05.517+0000 7fd3567fc640 1 --2- 172.21.15.3:0/1954375437 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fd33806be60 0x7fd33806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:05.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:05.518+0000 7fd3567fc640 1 --2- 172.21.15.3:0/1954375437 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fd33806be60 0x7fd33806e320 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fd3580c9c30 tx=0x7fd340009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:05.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:05.521+0000 7fd34ffff640 1 -- 172.21.15.3:0/1954375437 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fd34804b020 con 0x7fd344003b70 2024-05-09T06:37:05.727 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:05.746+0000 7fd35d929640 1 -- 172.21.15.3:0/1954375437 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fd3580d18f0 con 0x7fd344003b70 2024-05-09T06:37:05.727 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:05.747+0000 7fd34ffff640 1 -- 172.21.15.3:0/1954375437 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) ==== 53+0+73 (secure 0 0 0) 0x7fd34807a8b0 con 0x7fd344003b70 2024-05-09T06:37:05.728 INFO:teuthology.orchestra.run.smithi003.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-09T06:37:05.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:05.748+0000 7fd35d929640 1 -- 172.21.15.3:0/1954375437 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fd33806be60 msgr2=0x7fd33806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:05.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:05.748+0000 7fd35d929640 1 --2- 172.21.15.3:0/1954375437 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fd33806be60 0x7fd33806e320 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fd3580c9c30 tx=0x7fd340009040 comp rx=0 tx=0).stop 2024-05-09T06:37:05.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:05.748+0000 7fd35d929640 1 -- 172.21.15.3:0/1954375437 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd344003b70 msgr2=0x7fd3581f5420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:05.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:05.748+0000 7fd35d929640 1 --2- 172.21.15.3:0/1954375437 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd344003b70 0x7fd3581f5420 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fd34802af80 tx=0x7fd348009750 comp rx=0 tx=0).stop 2024-05-09T06:37:05.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:05.748+0000 7fd35d929640 1 -- 172.21.15.3:0/1954375437 shutdown_connections 2024-05-09T06:37:05.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:05.748+0000 7fd35d929640 1 --2- 172.21.15.3:0/1954375437 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fd33806be60 0x7fd33806e320 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:05.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:05.748+0000 7fd35d929640 1 --2- 172.21.15.3:0/1954375437 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd3581f7e80 0x7fd35806cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:05.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:05.748+0000 7fd35d929640 1 --2- 172.21.15.3:0/1954375437 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd35815b810 0x7fd3580b3430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:05.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:05.748+0000 7fd35d929640 1 --2- 172.21.15.3:0/1954375437 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd344003b70 0x7fd3581f5420 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:05.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:05.748+0000 7fd35d929640 1 -- 172.21.15.3:0/1954375437 >> 172.21.15.3:0/1954375437 conn(0x7fd35809d4d0 msgr2=0x7fd3580b7610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:05.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:05.749+0000 7fd35d929640 1 -- 172.21.15.3:0/1954375437 shutdown_connections 2024-05-09T06:37:05.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:05.749+0000 7fd35d929640 1 -- 172.21.15.3:0/1954375437 wait complete. 2024-05-09T06:37:05.740 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-05-09T06:37:05.740 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-09T06:37:05.740 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi190.front.sepia.ceph.com 2024-05-09T06:37:05.740 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-05-09T06:37:05.740 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-05-09T06:37:05.740 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-05-09T06:37:05.740 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-05-09T06:37:05.740 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-09T06:37:05.740 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-05-09T06:37:05.740 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:37:05.741 DEBUG:teuthology.orchestra.run.smithi190:> ip addr 2024-05-09T06:37:05.760 INFO:teuthology.orchestra.run.smithi190.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-09T06:37:05.761 INFO:teuthology.orchestra.run.smithi190.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-09T06:37:05.761 INFO:teuthology.orchestra.run.smithi190.stdout: inet 127.0.0.1/8 scope host lo 2024-05-09T06:37:05.761 INFO:teuthology.orchestra.run.smithi190.stdout: valid_lft forever preferred_lft forever 2024-05-09T06:37:05.761 INFO:teuthology.orchestra.run.smithi190.stdout: inet6 ::1/128 scope host 2024-05-09T06:37:05.761 INFO:teuthology.orchestra.run.smithi190.stdout: valid_lft forever preferred_lft forever 2024-05-09T06:37:05.761 INFO:teuthology.orchestra.run.smithi190.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-09T06:37:05.761 INFO:teuthology.orchestra.run.smithi190.stdout: link/ether 0c:c4:7a:d9:36:d0 brd ff:ff:ff:ff:ff:ff 2024-05-09T06:37:05.761 INFO:teuthology.orchestra.run.smithi190.stdout: altname enp7s0f0 2024-05-09T06:37:05.761 INFO:teuthology.orchestra.run.smithi190.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-09T06:37:05.761 INFO:teuthology.orchestra.run.smithi190.stdout: link/ether 0c:c4:7a:d9:36:d1 brd ff:ff:ff:ff:ff:ff 2024-05-09T06:37:05.761 INFO:teuthology.orchestra.run.smithi190.stdout: altname enp7s0f1 2024-05-09T06:37:05.761 INFO:teuthology.orchestra.run.smithi190.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-09T06:37:05.761 INFO:teuthology.orchestra.run.smithi190.stdout: link/ether 0c:c4:7a:88:77:62 brd ff:ff:ff:ff:ff:ff 2024-05-09T06:37:05.762 INFO:teuthology.orchestra.run.smithi190.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-09T06:37:05.762 INFO:teuthology.orchestra.run.smithi190.stdout: link/ether 0c:c4:7a:88:77:63 brd ff:ff:ff:ff:ff:ff 2024-05-09T06:37:05.762 INFO:teuthology.orchestra.run.smithi190.stdout: inet 172.21.15.190/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-05-09T06:37:05.762 INFO:teuthology.orchestra.run.smithi190.stdout: valid_lft 42688sec preferred_lft 42688sec 2024-05-09T06:37:05.762 INFO:teuthology.orchestra.run.smithi190.stdout: inet6 fe80::ec4:7aff:fe88:7763/64 scope link 2024-05-09T06:37:05.762 INFO:teuthology.orchestra.run.smithi190.stdout: valid_lft forever preferred_lft forever 2024-05-09T06:37:05.762 INFO:teuthology.orchestra.run.smithi190.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-09T06:37:05.762 INFO:teuthology.orchestra.run.smithi190.stdout: link/ether c6:5d:ad:92:ed:35 brd ff:ff:ff:ff:ff:ff 2024-05-09T06:37:05.762 INFO:teuthology.orchestra.run.smithi190.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-09T06:37:05.762 INFO:teuthology.orchestra.run.smithi190.stdout: valid_lft forever preferred_lft forever 2024-05-09T06:37:05.762 INFO:teuthology.orchestra.run.smithi190.stdout: inet6 fe80::2c35:40ff:fe34:5f3a/64 scope link 2024-05-09T06:37:05.762 INFO:teuthology.orchestra.run.smithi190.stdout: valid_lft forever preferred_lft forever 2024-05-09T06:37:05.762 INFO:teuthology.orchestra.run.smithi190.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-09T06:37:05.762 INFO:teuthology.orchestra.run.smithi190.stdout: link/ether c6:5d:ad:92:ed:35 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-09T06:37:05.763 INFO:teuthology.orchestra.run.smithi190.stdout: inet6 fe80::c45d:adff:fe92:ed35/64 scope link 2024-05-09T06:37:05.763 INFO:teuthology.orchestra.run.smithi190.stdout: valid_lft forever preferred_lft forever 2024-05-09T06:37:05.763 INFO:teuthology.orchestra.run.smithi190.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-09T06:37:05.763 INFO:teuthology.orchestra.run.smithi190.stdout: link/ether e6:2d:e4:b9:3e:3e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-05-09T06:37:05.763 INFO:teuthology.orchestra.run.smithi190.stdout: inet6 fe80::e42d:e4ff:feb9:3e3e/64 scope link 2024-05-09T06:37:05.763 INFO:teuthology.orchestra.run.smithi190.stdout: valid_lft forever preferred_lft forever 2024-05-09T06:37:05.763 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:37:05.763 DEBUG:teuthology.orchestra.run.smithi190:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-09T06:37:05.830 INFO:teuthology.orchestra.run.smithi190.stdout:1 2024-05-09T06:37:05.832 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:37:05.832 DEBUG:teuthology.orchestra.run.smithi190:> route 2024-05-09T06:37:05.891 INFO:teuthology.orchestra.run.smithi190.stdout:Kernel IP routing table 2024-05-09T06:37:05.891 INFO:teuthology.orchestra.run.smithi190.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-09T06:37:05.892 INFO:teuthology.orchestra.run.smithi190.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-09T06:37:05.892 INFO:teuthology.orchestra.run.smithi190.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-09T06:37:05.892 INFO:teuthology.orchestra.run.smithi190.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-09T06:37:05.892 DEBUG:teuthology.orchestra.run.smithi190:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-09T06:37:05.892 DEBUG:teuthology.orchestra.run.smithi190:> set -e 2024-05-09T06:37:05.892 DEBUG:teuthology.orchestra.run.smithi190:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-09T06:37:05.892 DEBUG:teuthology.orchestra.run.smithi190:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-09T06:37:05.892 DEBUG:teuthology.orchestra.run.smithi190:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-09T06:37:05.892 DEBUG:teuthology.orchestra.run.smithi190:> ') 2024-05-09T06:37:05.985 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:37:06 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-09T06:37:06.019 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:37:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T06:37:06.026 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:37:06.026 DEBUG:teuthology.orchestra.run.smithi190:> ip netns list 2024-05-09T06:37:06.042 INFO:teuthology.orchestra.run.smithi190.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-05-09T06:37:06.042 INFO:teuthology.orchestra.run.smithi190.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-05-09T06:37:06.043 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-05-09T06:37:06.043 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:37:06.043 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip link set brx.0 up 2024-05-09T06:37:06.110 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-09T06:37:06.110 DEBUG:teuthology.orchestra.run.smithi190:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-05-09T06:37:06.167 INFO:teuthology.orchestra.run.smithi190.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-05-09T06:37:06.167 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-09T06:37:06.167 DEBUG:teuthology.orchestra.run.smithi190:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-05-09T06:37:06.223 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2024-05-09T06:37:06.223 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:37:06.223 DEBUG:teuthology.orchestra.run.smithi190:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-05-09T06:37:06.308 INFO:teuthology.orchestra.run.smithi190.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-05-09T06:37:06.308 INFO:teuthology.orchestra.run.smithi190.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-05-09T06:37:06.323 INFO:teuthology.orchestra.run.smithi190.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-05-09T06:37:06.323 INFO:teuthology.orchestra.run.smithi190.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-05-09T06:37:06.323 INFO:teuthology.orchestra.run.smithi190.stdout:mount.ceph: resolved to: "172.21.15.3:3300,172.21.15.190:3300,172.21.15.190:3301" 2024-05-09T06:37:06.323 INFO:teuthology.orchestra.run.smithi190.stdout:mount.ceph: trying mount with new device syntax: 1@16ed737d-2665-431a-8ac6-ef242be3975a.cephfs=/ 2024-05-09T06:37:06.323 INFO:teuthology.orchestra.run.smithi190.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.3:3300/172.21.15.190:3300/172.21.15.190:3301" will pass to kernel 2024-05-09T06:37:06.323 INFO:teuthology.orchestra.run.smithi190.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-05-09T06:37:06.324 INFO:teuthology.orchestra.run.smithi190.stdout: You just mounted a file system that supports labels which does not 2024-05-09T06:37:06.324 INFO:teuthology.orchestra.run.smithi190.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-05-09T06:37:06.324 INFO:teuthology.orchestra.run.smithi190.stdout: applications will generate AVC messages and not be allowed access to 2024-05-09T06:37:06.324 INFO:teuthology.orchestra.run.smithi190.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-05-09T06:37:06.326 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-09T06:37:06.326 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:37:06.326 DEBUG:teuthology.orchestra.run.smithi190:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-05-09T06:37:06.351 DEBUG:teuthology.orchestra.run.smithi190:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-09T06:37:06.362 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-09T06:37:06.364 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-09T06:37:06.443 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:37:06 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-09T06:37:06.445 INFO:teuthology.orchestra.run.smithi190.stderr:2024-05-09T06:37:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T06:37:06.450 INFO:teuthology.orchestra.run.smithi190.stdout:client5289 2024-05-09T06:37:06.451 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-05-09T06:37:06.451 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T06:37:06.508 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-05-09T06:37:06.631 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:06.650+0000 7fb2336a5640 1 Processor -- start 2024-05-09T06:37:06.631 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:06.651+0000 7fb2336a5640 1 -- start start 2024-05-09T06:37:06.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:06.651+0000 7fb2336a5640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fb22c15e400 0x7fb22c0b99c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:06.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:06.651+0000 7fb2336a5640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fb22c0b32b0 0x7fb22c0b71d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:06.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:06.651+0000 7fb2336a5640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb22c0cee30 0x7fb22c0cf2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:06.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:06.651+0000 7fb2336a5640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fb22c0d2f50 con 0x7fb22c0b32b0 2024-05-09T06:37:06.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:06.651+0000 7fb2336a5640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7fb22c0b9f00 con 0x7fb22c15e400 2024-05-09T06:37:06.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:06.651+0000 7fb2336a5640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fb22c0cf800 con 0x7fb22c0cee30 2024-05-09T06:37:06.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:06.652+0000 7fb230c40640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fb22c0b32b0 0x7fb22c0b71d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:06.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:06.652+0000 7fb230c40640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fb22c0b32b0 0x7fb22c0b71d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:39584/0 (socket says 172.21.15.3:39584) 2024-05-09T06:37:06.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:06.652+0000 7fb230c40640 1 -- 172.21.15.3:0/784645658 learned_addr learned my addr 172.21.15.3:0/784645658 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:37:06.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:06.652+0000 7fb231441640 1 --2- 172.21.15.3:0/784645658 >> v2:172.21.15.190:3300/0 conn(0x7fb22c15e400 0x7fb22c0b99c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:06.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:06.652+0000 7fb231c42640 1 --2- 172.21.15.3:0/784645658 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb22c0cee30 0x7fb22c0cf2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:06.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:06.652+0000 7fb230c40640 1 -- 172.21.15.3:0/784645658 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb22c0cee30 msgr2=0x7fb22c0cf2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:37:06.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:06.652+0000 7fb230c40640 1 --2- 172.21.15.3:0/784645658 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb22c0cee30 0x7fb22c0cf2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:06.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:06.652+0000 7fb230c40640 1 -- 172.21.15.3:0/784645658 >> v2:172.21.15.190:3300/0 conn(0x7fb22c15e400 msgr2=0x7fb22c0b99c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:37:06.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:06.652+0000 7fb230c40640 1 --2- 172.21.15.3:0/784645658 >> v2:172.21.15.190:3300/0 conn(0x7fb22c15e400 0x7fb22c0b99c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:06.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:06.652+0000 7fb230c40640 1 -- 172.21.15.3:0/784645658 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb22c0d2620 con 0x7fb22c0b32b0 2024-05-09T06:37:06.636 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:06.653+0000 7fb230c40640 1 --2- 172.21.15.3:0/784645658 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb22c0b32b0 0x7fb22c0b71d0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fb22c0d2450 tx=0x7fb21c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d60db90ce5ac90d0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:06.636 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:06.653+0000 7fb223fff640 1 -- 172.21.15.3:0/784645658 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb21c037070 con 0x7fb22c0b32b0 2024-05-09T06:37:06.636 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:06.653+0000 7fb223fff640 1 -- 172.21.15.3:0/784645658 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb21c032440 con 0x7fb22c0b32b0 2024-05-09T06:37:06.636 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:06.653+0000 7fb223fff640 1 -- 172.21.15.3:0/784645658 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb21c032810 con 0x7fb22c0b32b0 2024-05-09T06:37:06.636 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:06.653+0000 7fb2336a5640 1 -- 172.21.15.3:0/784645658 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb22c0b32b0 msgr2=0x7fb22c0b71d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:06.636 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:06.653+0000 7fb2336a5640 1 --2- 172.21.15.3:0/784645658 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb22c0b32b0 0x7fb22c0b71d0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fb22c0d2450 tx=0x7fb21c02a270 comp rx=0 tx=0).stop 2024-05-09T06:37:06.636 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:06.654+0000 7fb2336a5640 1 -- 172.21.15.3:0/784645658 shutdown_connections 2024-05-09T06:37:06.636 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:06.654+0000 7fb2336a5640 1 --2- 172.21.15.3:0/784645658 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb22c0cee30 0x7fb22c0cf2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:06.636 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:06.654+0000 7fb2336a5640 1 --2- 172.21.15.3:0/784645658 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb22c0b32b0 0x7fb22c0b71d0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:06.636 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:06.654+0000 7fb2336a5640 1 --2- 172.21.15.3:0/784645658 >> v2:172.21.15.190:3300/0 conn(0x7fb22c15e400 0x7fb22c0b99c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:06.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:06.654+0000 7fb2336a5640 1 -- 172.21.15.3:0/784645658 >> 172.21.15.3:0/784645658 conn(0x7fb22c0675c0 msgr2=0x7fb22c0d0be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:06.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:06.654+0000 7fb2336a5640 1 -- 172.21.15.3:0/784645658 shutdown_connections 2024-05-09T06:37:06.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:06.654+0000 7fb2336a5640 1 -- 172.21.15.3:0/784645658 wait complete. 2024-05-09T06:37:06.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:06.655+0000 7fb2336a5640 1 Processor -- start 2024-05-09T06:37:06.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:06.655+0000 7fb2336a5640 1 -- start start 2024-05-09T06:37:06.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:06.656+0000 7fb2336a5640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb22c0b32b0 0x7fb22c1549b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:06.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:06.656+0000 7fb2336a5640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb22c0cee30 0x7fb22c15fb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:06.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:06.656+0000 7fb2336a5640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb22c15e400 0x7fb22c0cc820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:06.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:06.656+0000 7fb2336a5640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fb22c0b7710 con 0x7fb22c0b32b0 2024-05-09T06:37:06.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:06.656+0000 7fb2336a5640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fb22c0d2be0 con 0x7fb22c15e400 2024-05-09T06:37:06.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:06.656+0000 7fb2336a5640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fb22c0b7890 con 0x7fb22c0cee30 2024-05-09T06:37:06.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:06.656+0000 7fb231441640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb22c0b32b0 0x7fb22c1549b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:06.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:06.656+0000 7fb231441640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb22c0b32b0 0x7fb22c1549b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:39586/0 (socket says 172.21.15.3:39586) 2024-05-09T06:37:06.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:06.656+0000 7fb231441640 1 -- 172.21.15.3:0/953476817 learned_addr learned my addr 172.21.15.3:0/953476817 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:37:06.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:06.656+0000 7fb230c40640 1 --2- 172.21.15.3:0/953476817 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb22c0cee30 0x7fb22c15fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:06.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:06.656+0000 7fb231c42640 1 --2- 172.21.15.3:0/953476817 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb22c15e400 0x7fb22c0cc820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:06.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:06.656+0000 7fb231441640 1 -- 172.21.15.3:0/953476817 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb22c0cee30 msgr2=0x7fb22c15fb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:06.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:06.656+0000 7fb231441640 1 --2- 172.21.15.3:0/953476817 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb22c0cee30 0x7fb22c15fb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:06.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:06.656+0000 7fb231441640 1 -- 172.21.15.3:0/953476817 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb22c15e400 msgr2=0x7fb22c0cc820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:06.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:06.656+0000 7fb231441640 1 --2- 172.21.15.3:0/953476817 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb22c15e400 0x7fb22c0cc820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:06.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:06.656+0000 7fb231441640 1 -- 172.21.15.3:0/953476817 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb22c161680 con 0x7fb22c0b32b0 2024-05-09T06:37:06.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:06.656+0000 7fb230c40640 1 --2- 172.21.15.3:0/953476817 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb22c0cee30 0x7fb22c15fb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T06:37:06.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:06.656+0000 7fb231441640 1 --2- 172.21.15.3:0/953476817 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb22c0b32b0 0x7fb22c1549b0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fb228009d10 tx=0x7fb22800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:06.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:06.657+0000 7fb221ffb640 1 -- 172.21.15.3:0/953476817 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb228018020 con 0x7fb22c0b32b0 2024-05-09T06:37:06.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:06.657+0000 7fb221ffb640 1 -- 172.21.15.3:0/953476817 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb228014480 con 0x7fb22c0b32b0 2024-05-09T06:37:06.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:06.657+0000 7fb2336a5640 1 -- 172.21.15.3:0/953476817 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb22c1605e0 con 0x7fb22c0b32b0 2024-05-09T06:37:06.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:06.657+0000 7fb221ffb640 1 -- 172.21.15.3:0/953476817 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb228014830 con 0x7fb22c0b32b0 2024-05-09T06:37:06.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:06.657+0000 7fb2336a5640 1 -- 172.21.15.3:0/953476817 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb22c160ba0 con 0x7fb22c0b32b0 2024-05-09T06:37:06.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:06.658+0000 7fb2336a5640 1 -- 172.21.15.3:0/953476817 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb22c069550 con 0x7fb22c0b32b0 2024-05-09T06:37:06.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:06.658+0000 7fb221ffb640 1 -- 172.21.15.3:0/953476817 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb228005d10 con 0x7fb22c0b32b0 2024-05-09T06:37:06.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:06.659+0000 7fb221ffb640 1 --2- 172.21.15.3:0/953476817 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fb21006be30 0x7fb21006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:06.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:06.659+0000 7fb221ffb640 1 -- 172.21.15.3:0/953476817 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fb22808b730 con 0x7fb22c0b32b0 2024-05-09T06:37:06.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:06.659+0000 7fb230c40640 1 --2- 172.21.15.3:0/953476817 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fb21006be30 0x7fb21006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:06.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:06.660+0000 7fb230c40640 1 --2- 172.21.15.3:0/953476817 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fb21006be30 0x7fb21006e2f0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fb21c034000 tx=0x7fb21c02e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:06.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:06.664+0000 7fb221ffb640 1 -- 172.21.15.3:0/953476817 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fb228022070 con 0x7fb22c0b32b0 2024-05-09T06:37:06.867 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:06.887+0000 7fb2336a5640 1 -- 172.21.15.3:0/953476817 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7fb22c0b89d0 con 0x7fb22c0b32b0 2024-05-09T06:37:07.578 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:07.597+0000 7fb221ffb640 1 -- 172.21.15.3:0/953476817 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v18) ==== 136+0+0 (secure 0 0 0) 0x7fb228010040 con 0x7fb22c0b32b0 2024-05-09T06:37:07.581 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:07.600+0000 7fb2336a5640 1 -- 172.21.15.3:0/953476817 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fb21006be30 msgr2=0x7fb21006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:07.581 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:07.600+0000 7fb2336a5640 1 --2- 172.21.15.3:0/953476817 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fb21006be30 0x7fb21006e2f0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fb21c034000 tx=0x7fb21c02e000 comp rx=0 tx=0).stop 2024-05-09T06:37:07.581 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:07.600+0000 7fb2336a5640 1 -- 172.21.15.3:0/953476817 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb22c0b32b0 msgr2=0x7fb22c1549b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:07.581 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:07.600+0000 7fb2336a5640 1 --2- 172.21.15.3:0/953476817 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb22c0b32b0 0x7fb22c1549b0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fb228009d10 tx=0x7fb22800c490 comp rx=0 tx=0).stop 2024-05-09T06:37:07.581 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:07.600+0000 7fb2336a5640 1 -- 172.21.15.3:0/953476817 shutdown_connections 2024-05-09T06:37:07.581 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:07.600+0000 7fb2336a5640 1 --2- 172.21.15.3:0/953476817 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fb21006be30 0x7fb21006e2f0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:07.581 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:07.600+0000 7fb2336a5640 1 --2- 172.21.15.3:0/953476817 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb22c15e400 0x7fb22c0cc820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:07.581 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:07.600+0000 7fb2336a5640 1 --2- 172.21.15.3:0/953476817 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb22c0cee30 0x7fb22c15fb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:07.581 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:07.600+0000 7fb2336a5640 1 --2- 172.21.15.3:0/953476817 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb22c0b32b0 0x7fb22c1549b0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:07.581 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:07.600+0000 7fb2336a5640 1 -- 172.21.15.3:0/953476817 >> 172.21.15.3:0/953476817 conn(0x7fb22c0675c0 msgr2=0x7fb22c0b81c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:07.581 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:07.600+0000 7fb2336a5640 1 -- 172.21.15.3:0/953476817 shutdown_connections 2024-05-09T06:37:07.581 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:07.600+0000 7fb2336a5640 1 -- 172.21.15.3:0/953476817 wait complete. 2024-05-09T06:37:07.594 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-05-09T06:37:07.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:07.726+0000 7fbfaca5d640 1 Processor -- start 2024-05-09T06:37:07.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:07.726+0000 7fbfaca5d640 1 -- start start 2024-05-09T06:37:07.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:07.727+0000 7fbfaca5d640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbfa8072900 0x7fbfa80cf8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:07.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:07.727+0000 7fbfaca5d640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbfa8071860 con 0x7fbfa8144a10 2024-05-09T06:37:07.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:07.727+0000 7fbfaca5d640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbfa8071b40 con 0x7fbfa80b4130 2024-05-09T06:37:07.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:07.727+0000 7fbfaca5d640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fbfa8072470 con 0x7fbfa8072900 2024-05-09T06:37:07.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:07.727+0000 7fbfa659c640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fbfa8144a10 0x7fbfa8071080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:58070/0 (socket says 172.21.15.3:58070) 2024-05-09T06:37:07.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:07.727+0000 7fbfa659c640 1 -- 172.21.15.3:0/2973612929 learned_addr learned my addr 172.21.15.3:0/2973612929 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-05-09T06:37:07.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:07.727+0000 7fbfa5d9b640 1 --2- 172.21.15.3:0/2973612929 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbfa8072900 0x7fbfa80cf8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:07.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:07.727+0000 7fbfa559a640 1 -- 172.21.15.3:0/2973612929 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4005766832 0 0) 0x7fbfa8071860 con 0x7fbfa8144a10 2024-05-09T06:37:07.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:07.727+0000 7fbfa559a640 1 -- 172.21.15.3:0/2973612929 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbf94003520 con 0x7fbfa8144a10 2024-05-09T06:37:07.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:07.728+0000 7fbfa559a640 1 -- 172.21.15.3:0/2973612929 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2588589427 0 0) 0x7fbfa8071b40 con 0x7fbfa80b4130 2024-05-09T06:37:07.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:07.728+0000 7fbfa559a640 1 -- 172.21.15.3:0/2973612929 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbfa8071860 con 0x7fbfa80b4130 2024-05-09T06:37:07.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:07.728+0000 7fbfa5d9b640 1 -- 172.21.15.3:0/2973612929 >> v1:172.21.15.190:6789/0 conn(0x7fbfa80b4130 legacy=0x7fbfa815b5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:07.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:07.728+0000 7fbfa5d9b640 1 -- 172.21.15.3:0/2973612929 >> v1:172.21.15.3:6789/0 conn(0x7fbfa8144a10 legacy=0x7fbfa8071080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:07.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:07.728+0000 7fbfa5d9b640 1 -- 172.21.15.3:0/2973612929 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfa8071e20 con 0x7fbfa8072900 2024-05-09T06:37:07.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:07.728+0000 7fbfa5d9b640 1 --2- 172.21.15.3:0/2973612929 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbfa8072900 0x7fbfa80cf8a0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fbfa80725f0 tx=0x7fbf9002a1a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3446cf9c58b5d1a9 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:07.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:07.729+0000 7fbfa559a640 1 -- 172.21.15.3:0/2973612929 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf90037070 con 0x7fbfa8072900 2024-05-09T06:37:07.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:07.729+0000 7fbfa559a640 1 -- 172.21.15.3:0/2973612929 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf90032440 con 0x7fbfa8072900 2024-05-09T06:37:07.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:07.729+0000 7fbfa559a640 1 -- 172.21.15.3:0/2973612929 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf90032810 con 0x7fbfa8072900 2024-05-09T06:37:07.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:07.729+0000 7fbfaca5d640 1 -- 172.21.15.3:0/2973612929 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbfa8072900 msgr2=0x7fbfa80cf8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:07.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:07.729+0000 7fbfaca5d640 1 --2- 172.21.15.3:0/2973612929 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbfa8072900 0x7fbfa80cf8a0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fbfa80725f0 tx=0x7fbf9002a1a0 comp rx=0 tx=0).stop 2024-05-09T06:37:07.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:07.729+0000 7fbfaca5d640 1 -- 172.21.15.3:0/2973612929 shutdown_connections 2024-05-09T06:37:07.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:07.729+0000 7fbfaca5d640 1 --2- 172.21.15.3:0/2973612929 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbfa8072900 0x7fbfa80cf8a0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:07.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:07.729+0000 7fbfaca5d640 1 -- 172.21.15.3:0/2973612929 >> 172.21.15.3:0/2973612929 conn(0x7fbfa80b1c20 msgr2=0x7fbfa80b2030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:07.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:07.729+0000 7fbfaca5d640 1 -- 172.21.15.3:0/2973612929 shutdown_connections 2024-05-09T06:37:07.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:07.729+0000 7fbfaca5d640 1 -- 172.21.15.3:0/2973612929 wait complete. 2024-05-09T06:37:07.713 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:07.731+0000 7fbfaca5d640 1 Processor -- start 2024-05-09T06:37:07.713 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:07.731+0000 7fbfaca5d640 1 -- start start 2024-05-09T06:37:07.713 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:07.731+0000 7fbfaca5d640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbfa8072900 0x7fbfa806dc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:07.713 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:07.731+0000 7fbfaca5d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbfa80b4130 0x7fbfa813a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:07.713 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:07.731+0000 7fbfaca5d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbfa8144a10 0x7fbfa813e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:07.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:07.731+0000 7fbfaca5d640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fbfa80b4760 con 0x7fbfa8144a10 2024-05-09T06:37:07.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:07.731+0000 7fbfaca5d640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fbfa80b45e0 con 0x7fbfa80b4130 2024-05-09T06:37:07.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:07.731+0000 7fbfaca5d640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fbfa80b48e0 con 0x7fbfa8072900 2024-05-09T06:37:07.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:07.732+0000 7fbfa6d9d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbfa8144a10 0x7fbfa813e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:07.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:07.732+0000 7fbfa659c640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbfa8072900 0x7fbfa806dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:07.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:07.732+0000 7fbfa6d9d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbfa8144a10 0x7fbfa813e870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:39596/0 (socket says 172.21.15.3:39596) 2024-05-09T06:37:07.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:07.732+0000 7fbfa5d9b640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbfa80b4130 0x7fbfa813a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:07.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:07.732+0000 7fbfa6d9d640 1 -- 172.21.15.3:0/1260719447 learned_addr learned my addr 172.21.15.3:0/1260719447 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:37:07.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:07.732+0000 7fbfa6d9d640 1 -- 172.21.15.3:0/1260719447 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbfa8072900 msgr2=0x7fbfa806dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:07.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:07.732+0000 7fbfa6d9d640 1 --2- 172.21.15.3:0/1260719447 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbfa8072900 0x7fbfa806dc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:07.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:07.732+0000 7fbfa6d9d640 1 -- 172.21.15.3:0/1260719447 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbfa80b4130 msgr2=0x7fbfa813a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:07.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:07.732+0000 7fbfa6d9d640 1 --2- 172.21.15.3:0/1260719447 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbfa80b4130 0x7fbfa813a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:07.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:07.732+0000 7fbfa6d9d640 1 -- 172.21.15.3:0/1260719447 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfa81392a0 con 0x7fbfa8144a10 2024-05-09T06:37:07.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:07.732+0000 7fbfa659c640 1 --2- 172.21.15.3:0/1260719447 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbfa8072900 0x7fbfa806dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T06:37:07.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:07.732+0000 7fbfa5d9b640 1 --2- 172.21.15.3:0/1260719447 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbfa80b4130 0x7fbfa813a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T06:37:07.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:07.732+0000 7fbfa6d9d640 1 --2- 172.21.15.3:0/1260719447 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbfa8144a10 0x7fbfa813e870 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fbf9800c390 tx=0x7fbf9800c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:07.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:07.732+0000 7fbf9effd640 1 -- 172.21.15.3:0/1260719447 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf98017070 con 0x7fbfa8144a10 2024-05-09T06:37:07.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:07.732+0000 7fbfaca5d640 1 -- 172.21.15.3:0/1260719447 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbfa8139580 con 0x7fbfa8144a10 2024-05-09T06:37:07.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:07.732+0000 7fbf9effd640 1 -- 172.21.15.3:0/1260719447 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf980052d0 con 0x7fbfa8144a10 2024-05-09T06:37:07.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:07.732+0000 7fbf9effd640 1 -- 172.21.15.3:0/1260719447 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf98003830 con 0x7fbfa8144a10 2024-05-09T06:37:07.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:07.732+0000 7fbfaca5d640 1 -- 172.21.15.3:0/1260719447 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbfa8139b40 con 0x7fbfa8144a10 2024-05-09T06:37:07.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:07.733+0000 7fbfaca5d640 1 -- 172.21.15.3:0/1260719447 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbfa8069550 con 0x7fbfa8144a10 2024-05-09T06:37:07.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:07.734+0000 7fbf9effd640 1 -- 172.21.15.3:0/1260719447 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fbf9801a070 con 0x7fbfa8144a10 2024-05-09T06:37:07.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:07.734+0000 7fbf9effd640 1 --2- 172.21.15.3:0/1260719447 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fbf8806be60 0x7fbf8806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:07.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:07.734+0000 7fbf9effd640 1 -- 172.21.15.3:0/1260719447 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fbf9808a150 con 0x7fbfa8144a10 2024-05-09T06:37:07.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:07.735+0000 7fbfa659c640 1 --2- 172.21.15.3:0/1260719447 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fbf8806be60 0x7fbf8806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:07.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:07.735+0000 7fbfa659c640 1 --2- 172.21.15.3:0/1260719447 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fbf8806be60 0x7fbf8806e320 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fbfa00012e0 tx=0x7fbfa0001ad0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:07.719 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:07.739+0000 7fbf9effd640 1 -- 172.21.15.3:0/1260719447 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fbf98058420 con 0x7fbfa8144a10 2024-05-09T06:37:07.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:07.945+0000 7fbfaca5d640 1 -- 172.21.15.3:0/1260719447 --> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7fbfa80cd340 con 0x7fbf8806be60 2024-05-09T06:37:07.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:07.947+0000 7fbf9effd640 1 -- 172.21.15.3:0/1260719447 <== mgr.4105 v2:172.21.15.190:6832/975893321 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7fbfa80cd340 con 0x7fbf8806be60 2024-05-09T06:37:07.928 INFO:teuthology.orchestra.run.smithi003.stdout:[ 2024-05-09T06:37:07.928 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-09T06:37:07.928 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "cephfs" 2024-05-09T06:37:07.928 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-09T06:37:07.928 INFO:teuthology.orchestra.run.smithi003.stdout:] 2024-05-09T06:37:07.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:07.949+0000 7fbfaca5d640 1 -- 172.21.15.3:0/1260719447 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fbf8806be60 msgr2=0x7fbf8806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:07.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:07.949+0000 7fbfaca5d640 1 --2- 172.21.15.3:0/1260719447 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fbf8806be60 0x7fbf8806e320 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fbfa00012e0 tx=0x7fbfa0001ad0 comp rx=0 tx=0).stop 2024-05-09T06:37:07.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:07.949+0000 7fbfaca5d640 1 -- 172.21.15.3:0/1260719447 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbfa8144a10 msgr2=0x7fbfa813e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:07.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:07.949+0000 7fbfaca5d640 1 --2- 172.21.15.3:0/1260719447 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbfa8144a10 0x7fbfa813e870 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fbf9800c390 tx=0x7fbf9800c850 comp rx=0 tx=0).stop 2024-05-09T06:37:07.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:07.949+0000 7fbfaca5d640 1 -- 172.21.15.3:0/1260719447 shutdown_connections 2024-05-09T06:37:07.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:07.949+0000 7fbfaca5d640 1 --2- 172.21.15.3:0/1260719447 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fbf8806be60 0x7fbf8806e320 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:07.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:07.949+0000 7fbfaca5d640 1 --2- 172.21.15.3:0/1260719447 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbfa8144a10 0x7fbfa813e870 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:07.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:07.949+0000 7fbfaca5d640 1 --2- 172.21.15.3:0/1260719447 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbfa80b4130 0x7fbfa813a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:07.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:07.949+0000 7fbfaca5d640 1 --2- 172.21.15.3:0/1260719447 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbfa8072900 0x7fbfa806dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:07.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:07.949+0000 7fbfaca5d640 1 -- 172.21.15.3:0/1260719447 >> 172.21.15.3:0/1260719447 conn(0x7fbfa80b1c20 msgr2=0x7fbfa813fb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:07.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:07.949+0000 7fbfaca5d640 1 -- 172.21.15.3:0/1260719447 shutdown_connections 2024-05-09T06:37:07.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:07.949+0000 7fbfaca5d640 1 -- 172.21.15.3:0/1260719447 wait complete. 2024-05-09T06:37:07.941 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-05-09T06:37:08.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.068+0000 7f5146d84640 1 Processor -- start 2024-05-09T06:37:08.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.068+0000 7f5146d84640 1 -- start start 2024-05-09T06:37:08.052 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.068+0000 7f5146d84640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f51400d2800 0x7f51400d4c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:08.052 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.068+0000 7f5146d84640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f514015e9a0 con 0x7f51400d1300 2024-05-09T06:37:08.052 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.068+0000 7f5146d84640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f51400d31c0 con 0x7f514015e5c0 2024-05-09T06:37:08.052 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.068+0000 7f5146d84640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f51400d2c90 con 0x7f51400d2800 2024-05-09T06:37:08.052 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.069+0000 7f513ffff640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f51400d1300 0x7f51400b47e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:58086/0 (socket says 172.21.15.3:58086) 2024-05-09T06:37:08.053 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.069+0000 7f513ffff640 1 -- 172.21.15.3:0/2459247636 learned_addr learned my addr 172.21.15.3:0/2459247636 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-05-09T06:37:08.053 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.069+0000 7f5145321640 1 --2- 172.21.15.3:0/2459247636 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f51400d2800 0x7f51400d4c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:08.053 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.069+0000 7f513f7fe640 1 -- 172.21.15.3:0/2459247636 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1081709412 0 0) 0x7f514015e9a0 con 0x7f51400d1300 2024-05-09T06:37:08.053 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.069+0000 7f513f7fe640 1 -- 172.21.15.3:0/2459247636 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f512c003520 con 0x7f51400d1300 2024-05-09T06:37:08.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.069+0000 7f5145321640 1 -- 172.21.15.3:0/2459247636 >> v1:172.21.15.190:6789/0 conn(0x7f514015e5c0 legacy=0x7f51400d0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:08.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.069+0000 7f513f7fe640 1 -- 172.21.15.3:0/2459247636 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2404967437 0 0) 0x7f51400d31c0 con 0x7f514015e5c0 2024-05-09T06:37:08.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.069+0000 7f5145321640 1 -- 172.21.15.3:0/2459247636 >> v1:172.21.15.3:6789/0 conn(0x7f51400d1300 legacy=0x7f51400b47e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:08.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.069+0000 7f5145321640 1 -- 172.21.15.3:0/2459247636 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51400d2ee0 con 0x7f51400d2800 2024-05-09T06:37:08.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.070+0000 7f5145321640 1 --2- 172.21.15.3:0/2459247636 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f51400d2800 0x7f51400d4c40 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f51280097d0 tx=0x7f512802a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=910ff9dd1b7c6fa server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:08.055 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.070+0000 7f513f7fe640 1 -- 172.21.15.3:0/2459247636 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5128037070 con 0x7f51400d2800 2024-05-09T06:37:08.055 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.070+0000 7f513f7fe640 1 -- 172.21.15.3:0/2459247636 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f512802ac10 con 0x7f51400d2800 2024-05-09T06:37:08.055 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.070+0000 7f513f7fe640 1 -- 172.21.15.3:0/2459247636 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51300019f0 con 0x7f51400d2800 2024-05-09T06:37:08.055 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.071+0000 7f5146d84640 1 -- 172.21.15.3:0/2459247636 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f51400d2800 msgr2=0x7f51400d4c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:08.055 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.071+0000 7f5146d84640 1 --2- 172.21.15.3:0/2459247636 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f51400d2800 0x7f51400d4c40 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f51280097d0 tx=0x7f512802a180 comp rx=0 tx=0).stop 2024-05-09T06:37:08.055 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.071+0000 7f5146d84640 1 -- 172.21.15.3:0/2459247636 shutdown_connections 2024-05-09T06:37:08.055 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.071+0000 7f5146d84640 1 --2- 172.21.15.3:0/2459247636 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f51400d2800 0x7f51400d4c40 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:08.055 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.071+0000 7f5146d84640 1 -- 172.21.15.3:0/2459247636 >> 172.21.15.3:0/2459247636 conn(0x7f514009d500 msgr2=0x7f51400d4220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:08.055 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.071+0000 7f5146d84640 1 -- 172.21.15.3:0/2459247636 shutdown_connections 2024-05-09T06:37:08.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.071+0000 7f5146d84640 1 -- 172.21.15.3:0/2459247636 wait complete. 2024-05-09T06:37:08.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.072+0000 7f5146d84640 1 Processor -- start 2024-05-09T06:37:08.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.072+0000 7f5146d84640 1 -- start start 2024-05-09T06:37:08.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.073+0000 7f5146d84640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f51400d1300 0x7f5140157980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:08.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.073+0000 7f5146d84640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f51400d2800 0x7f514015a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:08.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.073+0000 7f5146d84640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f514015e5c0 0x7f5140158390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:08.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.073+0000 7f5146d84640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f51400b42c0 con 0x7f51400d1300 2024-05-09T06:37:08.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.073+0000 7f5146d84640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f51400b3fe0 con 0x7f514015e5c0 2024-05-09T06:37:08.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.073+0000 7f5146d84640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f51400b4ef0 con 0x7f51400d2800 2024-05-09T06:37:08.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.073+0000 7f5144b20640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f51400d1300 0x7f5140157980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:08.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.073+0000 7f5144b20640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f51400d1300 0x7f5140157980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:39610/0 (socket says 172.21.15.3:39610) 2024-05-09T06:37:08.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.073+0000 7f5144b20640 1 -- 172.21.15.3:0/97700317 learned_addr learned my addr 172.21.15.3:0/97700317 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:37:08.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.073+0000 7f5145321640 1 --2- 172.21.15.3:0/97700317 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f514015e5c0 0x7f5140158390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:08.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.073+0000 7f513ffff640 1 --2- 172.21.15.3:0/97700317 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f51400d2800 0x7f514015a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:08.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.073+0000 7f5144b20640 1 -- 172.21.15.3:0/97700317 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f51400d2800 msgr2=0x7f514015a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:08.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.073+0000 7f5144b20640 1 --2- 172.21.15.3:0/97700317 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f51400d2800 0x7f514015a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:08.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.073+0000 7f5144b20640 1 -- 172.21.15.3:0/97700317 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f514015e5c0 msgr2=0x7f5140158390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:08.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.073+0000 7f5144b20640 1 --2- 172.21.15.3:0/97700317 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f514015e5c0 0x7f5140158390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:08.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.073+0000 7f5144b20640 1 -- 172.21.15.3:0/97700317 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f514015acf0 con 0x7f51400d1300 2024-05-09T06:37:08.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.073+0000 7f513ffff640 1 --2- 172.21.15.3:0/97700317 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f51400d2800 0x7f514015a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:37:08.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.073+0000 7f5144b20640 1 --2- 172.21.15.3:0/97700317 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f51400d1300 0x7f5140157980 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f513800c410 tx=0x7f513800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:08.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.074+0000 7f513d7fa640 1 -- 172.21.15.3:0/97700317 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5138017070 con 0x7f51400d1300 2024-05-09T06:37:08.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.074+0000 7f513d7fa640 1 -- 172.21.15.3:0/97700317 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51380035f0 con 0x7f51400d1300 2024-05-09T06:37:08.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.074+0000 7f5146d84640 1 -- 172.21.15.3:0/97700317 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f514015afd0 con 0x7f51400d1300 2024-05-09T06:37:08.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.074+0000 7f513d7fa640 1 -- 172.21.15.3:0/97700317 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51380039e0 con 0x7f51400d1300 2024-05-09T06:37:08.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.074+0000 7f5146d84640 1 -- 172.21.15.3:0/97700317 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51400cc490 con 0x7f51400d1300 2024-05-09T06:37:08.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.075+0000 7f5146d84640 1 -- 172.21.15.3:0/97700317 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5140069550 con 0x7f51400d1300 2024-05-09T06:37:08.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.075+0000 7f513d7fa640 1 -- 172.21.15.3:0/97700317 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f513801a070 con 0x7f51400d1300 2024-05-09T06:37:08.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.076+0000 7f513d7fa640 1 --2- 172.21.15.3:0/97700317 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f512006be30 0x7f512006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:08.064 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.076+0000 7f513d7fa640 1 -- 172.21.15.3:0/97700317 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f513808a1b0 con 0x7f51400d1300 2024-05-09T06:37:08.064 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.076+0000 7f513ffff640 1 --2- 172.21.15.3:0/97700317 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f512006be30 0x7f512006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:08.064 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.076+0000 7f513ffff640 1 --2- 172.21.15.3:0/97700317 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f512006be30 0x7f512006e2f0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f5140161690 tx=0x7f5130005b70 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:08.064 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.080+0000 7f513d7fa640 1 -- 172.21.15.3:0/97700317 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f5138058510 con 0x7f51400d1300 2024-05-09T06:37:08.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.262+0000 7f5146d84640 1 -- 172.21.15.3:0/97700317 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f51400cc260 con 0x7f51400d1300 2024-05-09T06:37:08.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.262+0000 7f513d7fa640 1 -- 172.21.15.3:0/97700317 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) ==== 122+0+0 (secure 0 0 0) 0x7f513805c810 con 0x7f51400d1300 2024-05-09T06:37:08.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.264+0000 7f5146d84640 1 -- 172.21.15.3:0/97700317 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f512006be30 msgr2=0x7f512006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:08.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.264+0000 7f5146d84640 1 --2- 172.21.15.3:0/97700317 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f512006be30 0x7f512006e2f0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f5140161690 tx=0x7f5130005b70 comp rx=0 tx=0).stop 2024-05-09T06:37:08.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.264+0000 7f5146d84640 1 -- 172.21.15.3:0/97700317 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f51400d1300 msgr2=0x7f5140157980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:08.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.264+0000 7f5146d84640 1 --2- 172.21.15.3:0/97700317 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f51400d1300 0x7f5140157980 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f513800c410 tx=0x7f513800c8d0 comp rx=0 tx=0).stop 2024-05-09T06:37:08.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.264+0000 7f5146d84640 1 -- 172.21.15.3:0/97700317 shutdown_connections 2024-05-09T06:37:08.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.264+0000 7f5146d84640 1 --2- 172.21.15.3:0/97700317 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f512006be30 0x7f512006e2f0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:08.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.264+0000 7f5146d84640 1 --2- 172.21.15.3:0/97700317 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f514015e5c0 0x7f5140158390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:08.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.264+0000 7f5146d84640 1 --2- 172.21.15.3:0/97700317 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f51400d2800 0x7f514015a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:08.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.264+0000 7f5146d84640 1 --2- 172.21.15.3:0/97700317 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f51400d1300 0x7f5140157980 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:08.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.264+0000 7f5146d84640 1 -- 172.21.15.3:0/97700317 >> 172.21.15.3:0/97700317 conn(0x7f514009d500 msgr2=0x7f51400b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:08.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.264+0000 7f5146d84640 1 -- 172.21.15.3:0/97700317 shutdown_connections 2024-05-09T06:37:08.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.264+0000 7f5146d84640 1 -- 172.21.15.3:0/97700317 wait complete. 2024-05-09T06:37:08.256 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create cephfs subvol_grp46030 2024-05-09T06:37:08.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.383+0000 7f325115a640 1 Processor -- start 2024-05-09T06:37:08.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.383+0000 7f325115a640 1 -- start start 2024-05-09T06:37:08.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.383+0000 7f325115a640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f324c144910 0x7f324c0cf8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:08.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.383+0000 7f325115a640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f324c072970 0x7f324c071080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:08.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.383+0000 7f325115a640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f324c144cf0 con 0x7f324c0b4130 2024-05-09T06:37:08.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.383+0000 7f325115a640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f324c0724e0 con 0x7f324c072970 2024-05-09T06:37:08.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.383+0000 7f325115a640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f324c0718d0 con 0x7f324c144910 2024-05-09T06:37:08.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.384+0000 7f324a59c640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f324c072970 0x7f324c071080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:08.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.384+0000 7f324ad9d640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f324c144910 0x7f324c0cf8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:08.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.384+0000 7f324a59c640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f324c072970 0x7f324c071080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.3:52010/0 (socket says 172.21.15.3:52010) 2024-05-09T06:37:08.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.384+0000 7f324a59c640 1 -- 172.21.15.3:0/721412937 learned_addr learned my addr 172.21.15.3:0/721412937 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:37:08.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.384+0000 7f324b59e640 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7f324c0b4130 0x7f324c06e250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.3:37740/0 (socket says 172.21.15.3:37740) 2024-05-09T06:37:08.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.384+0000 7f324a59c640 1 -- 172.21.15.3:0/721412937 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f324c144910 msgr2=0x7f324c0cf8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:37:08.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.384+0000 7f324a59c640 1 --2- 172.21.15.3:0/721412937 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f324c144910 0x7f324c0cf8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:08.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.384+0000 7f324a59c640 1 -- 172.21.15.3:0/721412937 >> v1:172.21.15.190:6789/0 conn(0x7f324c0b4130 legacy=0x7f324c06e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:08.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.384+0000 7f324a59c640 1 -- 172.21.15.3:0/721412937 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f324c071bb0 con 0x7f324c072970 2024-05-09T06:37:08.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.384+0000 7f3249d9b640 1 -- 172.21.15.3:0/721412937 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3792949482 0 0) 0x7f324c144cf0 con 0x7f324c0b4130 2024-05-09T06:37:08.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.385+0000 7f324a59c640 1 --2- 172.21.15.3:0/721412937 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f324c072970 0x7f324c071080 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f324000bb10 tx=0x7f324002c920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=30a63abb201b6041 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:08.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.385+0000 7f3249d9b640 1 -- 172.21.15.3:0/721412937 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3240036020 con 0x7f324c072970 2024-05-09T06:37:08.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.385+0000 7f3249d9b640 1 -- 172.21.15.3:0/721412937 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3240032820 con 0x7f324c072970 2024-05-09T06:37:08.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.385+0000 7f3249d9b640 1 -- 172.21.15.3:0/721412937 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3240032bf0 con 0x7f324c072970 2024-05-09T06:37:08.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.386+0000 7f325115a640 1 -- 172.21.15.3:0/721412937 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f324c072970 msgr2=0x7f324c071080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:08.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.386+0000 7f325115a640 1 --2- 172.21.15.3:0/721412937 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f324c072970 0x7f324c071080 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f324000bb10 tx=0x7f324002c920 comp rx=0 tx=0).stop 2024-05-09T06:37:08.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.386+0000 7f325115a640 1 -- 172.21.15.3:0/721412937 shutdown_connections 2024-05-09T06:37:08.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.386+0000 7f325115a640 1 --2- 172.21.15.3:0/721412937 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f324c072970 0x7f324c071080 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:08.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.386+0000 7f325115a640 1 --2- 172.21.15.3:0/721412937 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f324c144910 0x7f324c0cf8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:08.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.386+0000 7f325115a640 1 -- 172.21.15.3:0/721412937 >> 172.21.15.3:0/721412937 conn(0x7f324c0b1c20 msgr2=0x7f324c0b2030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:08.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.386+0000 7f325115a640 1 -- 172.21.15.3:0/721412937 shutdown_connections 2024-05-09T06:37:08.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.386+0000 7f325115a640 1 -- 172.21.15.3:0/721412937 wait complete. 2024-05-09T06:37:08.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.387+0000 7f325115a640 1 Processor -- start 2024-05-09T06:37:08.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.387+0000 7f325115a640 1 -- start start 2024-05-09T06:37:08.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.388+0000 7f325115a640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f324c072970 0x7f324c15a930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:08.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.388+0000 7f325115a640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f324c0b4130 0x7f324c15d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:08.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.388+0000 7f325115a640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f324c144910 0x7f324c15dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:08.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.388+0000 7f325115a640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f324c0715c0 con 0x7f324c0b4130 2024-05-09T06:37:08.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.388+0000 7f325115a640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f324c072170 con 0x7f324c144910 2024-05-09T06:37:08.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.388+0000 7f325115a640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f324c0b45c0 con 0x7f324c072970 2024-05-09T06:37:08.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.388+0000 7f324a59c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f324c0b4130 0x7f324c15d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:08.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.388+0000 7f324a59c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f324c0b4130 0x7f324c15d6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:39614/0 (socket says 172.21.15.3:39614) 2024-05-09T06:37:08.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.388+0000 7f324a59c640 1 -- 172.21.15.3:0/887672691 learned_addr learned my addr 172.21.15.3:0/887672691 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:37:08.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.388+0000 7f324ad9d640 1 --2- 172.21.15.3:0/887672691 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f324c072970 0x7f324c15a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:08.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.388+0000 7f324b59e640 1 --2- 172.21.15.3:0/887672691 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f324c144910 0x7f324c15dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:08.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.388+0000 7f324a59c640 1 -- 172.21.15.3:0/887672691 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f324c072970 msgr2=0x7f324c15a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:08.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.388+0000 7f324a59c640 1 --2- 172.21.15.3:0/887672691 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f324c072970 0x7f324c15a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:08.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.388+0000 7f324a59c640 1 -- 172.21.15.3:0/887672691 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f324c144910 msgr2=0x7f324c15dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:08.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.388+0000 7f324a59c640 1 --2- 172.21.15.3:0/887672691 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f324c144910 0x7f324c15dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:08.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.388+0000 7f324a59c640 1 -- 172.21.15.3:0/887672691 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f324c0ae800 con 0x7f324c0b4130 2024-05-09T06:37:08.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.389+0000 7f324a59c640 1 --2- 172.21.15.3:0/887672691 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f324c0b4130 0x7f324c15d6a0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f3240000c00 tx=0x7f32400046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:08.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.389+0000 7f32337fe640 1 -- 172.21.15.3:0/887672691 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3240036040 con 0x7f324c0b4130 2024-05-09T06:37:08.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.389+0000 7f32337fe640 1 -- 172.21.15.3:0/887672691 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f324003cb60 con 0x7f324c0b4130 2024-05-09T06:37:08.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.389+0000 7f325115a640 1 -- 172.21.15.3:0/887672691 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f324c0aeae0 con 0x7f324c0b4130 2024-05-09T06:37:08.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.389+0000 7f32337fe640 1 -- 172.21.15.3:0/887672691 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3240004030 con 0x7f324c0b4130 2024-05-09T06:37:08.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.389+0000 7f325115a640 1 -- 172.21.15.3:0/887672691 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f324c0af0a0 con 0x7f324c0b4130 2024-05-09T06:37:08.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.390+0000 7f325115a640 1 -- 172.21.15.3:0/887672691 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f324c0af2e0 con 0x7f324c0b4130 2024-05-09T06:37:08.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.390+0000 7f32337fe640 1 -- 172.21.15.3:0/887672691 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f324004d060 con 0x7f324c0b4130 2024-05-09T06:37:08.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.391+0000 7f32337fe640 1 --2- 172.21.15.3:0/887672691 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f323806be30 0x7f323806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:08.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.391+0000 7f32337fe640 1 -- 172.21.15.3:0/887672691 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f32400adc30 con 0x7f324c0b4130 2024-05-09T06:37:08.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.391+0000 7f324ad9d640 1 --2- 172.21.15.3:0/887672691 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f323806be30 0x7f323806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:08.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.392+0000 7f324ad9d640 1 --2- 172.21.15.3:0/887672691 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f323806be30 0x7f323806e2f0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f323c009aa0 tx=0x7f323c009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:08.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.395+0000 7f32337fe640 1 -- 172.21.15.3:0/887672691 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f324c0af2e0 con 0x7f324c0b4130 2024-05-09T06:37:08.584 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.603+0000 7f325115a640 1 -- 172.21.15.3:0/887672691 --> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "subvol_grp46030", "target": ["mon-mgr", ""]}) -- 0x7f324c069550 con 0x7f323806be30 2024-05-09T06:37:08.589 INFO:tasks.ceph.mgr.x.smithi190.stderr:2024-05-09T06:37:08.609+0000 7faa86a22640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-09T06:37:08.589 INFO:tasks.ceph.mgr.x.smithi190.stderr:2024-05-09T06:37:08.609+0000 7faa86a22640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-09T06:37:08.589 INFO:tasks.ceph.mgr.x.smithi190.stderr:2024-05-09T06:37:08.609+0000 7faa86a22640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-09T06:37:08.589 INFO:tasks.ceph.mgr.x.smithi190.stderr:2024-05-09T06:37:08.609+0000 7faa86a22640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-09T06:37:08.589 INFO:tasks.ceph.mgr.x.smithi190.stderr:2024-05-09T06:37:08.609+0000 7faa86a22640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-09T06:37:08.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.622+0000 7f32337fe640 1 -- 172.21.15.3:0/887672691 <== mgr.4105 v2:172.21.15.190:6832/975893321 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f324c069550 con 0x7f323806be30 2024-05-09T06:37:08.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.624+0000 7f325115a640 1 -- 172.21.15.3:0/887672691 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f323806be30 msgr2=0x7f323806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:08.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.624+0000 7f325115a640 1 --2- 172.21.15.3:0/887672691 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f323806be30 0x7f323806e2f0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f323c009aa0 tx=0x7f323c009190 comp rx=0 tx=0).stop 2024-05-09T06:37:08.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.624+0000 7f325115a640 1 -- 172.21.15.3:0/887672691 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f324c0b4130 msgr2=0x7f324c15d6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:08.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.624+0000 7f325115a640 1 --2- 172.21.15.3:0/887672691 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f324c0b4130 0x7f324c15d6a0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f3240000c00 tx=0x7f32400046c0 comp rx=0 tx=0).stop 2024-05-09T06:37:08.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.624+0000 7f325115a640 1 -- 172.21.15.3:0/887672691 shutdown_connections 2024-05-09T06:37:08.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.624+0000 7f325115a640 1 --2- 172.21.15.3:0/887672691 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f323806be30 0x7f323806e2f0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:08.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.624+0000 7f325115a640 1 --2- 172.21.15.3:0/887672691 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f324c144910 0x7f324c15dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:08.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.624+0000 7f325115a640 1 --2- 172.21.15.3:0/887672691 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f324c0b4130 0x7f324c15d6a0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:08.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.624+0000 7f325115a640 1 --2- 172.21.15.3:0/887672691 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f324c072970 0x7f324c15a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:08.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.624+0000 7f325115a640 1 -- 172.21.15.3:0/887672691 >> 172.21.15.3:0/887672691 conn(0x7f324c0b1c20 msgr2=0x7f324c06d710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:08.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.624+0000 7f325115a640 1 -- 172.21.15.3:0/887672691 shutdown_connections 2024-05-09T06:37:08.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.624+0000 7f325115a640 1 -- 172.21.15.3:0/887672691 wait complete. 2024-05-09T06:37:08.616 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs subvol46440 --group_name subvol_grp46030 2024-05-09T06:37:08.725 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.744+0000 7fdaf60a5640 1 Processor -- start 2024-05-09T06:37:08.725 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.744+0000 7fdaf60a5640 1 -- start start 2024-05-09T06:37:08.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.745+0000 7fdaf60a5640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fdaf00cdec0 0x7fdaf015c690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:08.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.745+0000 7fdaf60a5640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdaf0145c90 con 0x7fdaf01458b0 2024-05-09T06:37:08.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.745+0000 7fdaf60a5640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdaf00b4b60 con 0x7fdaf00b4410 2024-05-09T06:37:08.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.745+0000 7fdaf60a5640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7fdaf00b4e40 con 0x7fdaf00cdec0 2024-05-09T06:37:08.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.745+0000 7fdaef7fe640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fdaf01458b0 0x7fdaf00cd7b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:58100/0 (socket says 172.21.15.3:58100) 2024-05-09T06:37:08.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.745+0000 7fdaef7fe640 1 -- 172.21.15.3:0/2133969888 learned_addr learned my addr 172.21.15.3:0/2133969888 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-05-09T06:37:08.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.746+0000 7fdaeeffd640 1 --2- 172.21.15.3:0/2133969888 >> v2:172.21.15.190:3300/0 conn(0x7fdaf00cdec0 0x7fdaf015c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:08.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.746+0000 7fdaee7fc640 1 -- 172.21.15.3:0/2133969888 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3592501928 0 0) 0x7fdaf0145c90 con 0x7fdaf01458b0 2024-05-09T06:37:08.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.746+0000 7fdaee7fc640 1 -- 172.21.15.3:0/2133969888 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdad8003520 con 0x7fdaf01458b0 2024-05-09T06:37:08.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.746+0000 7fdaee7fc640 1 -- 172.21.15.3:0/2133969888 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3703730210 0 0) 0x7fdad8003520 con 0x7fdaf01458b0 2024-05-09T06:37:08.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.746+0000 7fdaee7fc640 1 -- 172.21.15.3:0/2133969888 >> v2:172.21.15.190:3300/0 conn(0x7fdaf00cdec0 msgr2=0x7fdaf015c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:37:08.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.746+0000 7fdaee7fc640 1 --2- 172.21.15.3:0/2133969888 >> v2:172.21.15.190:3300/0 conn(0x7fdaf00cdec0 0x7fdaf015c690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:08.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.746+0000 7fdaee7fc640 1 -- 172.21.15.3:0/2133969888 >> v1:172.21.15.190:6789/0 conn(0x7fdaf00b4410 legacy=0x7fdaf006dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:08.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.746+0000 7fdaee7fc640 1 -- 172.21.15.3:0/2133969888 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdaf00b5120 con 0x7fdaf01458b0 2024-05-09T06:37:08.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.746+0000 7fdaee7fc640 1 -- 172.21.15.3:0/2133969888 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 563312031 0 0) 0x7fdae0001670 con 0x7fdaf01458b0 2024-05-09T06:37:08.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.746+0000 7fdaee7fc640 1 -- 172.21.15.3:0/2133969888 >> v1:172.21.15.3:6789/0 conn(0x7fdaf01458b0 legacy=0x7fdaf00cd7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:08.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.746+0000 7fdaee7fc640 1 --2- 172.21.15.3:0/2133969888 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdad80039b0 0x7fdad8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:08.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.746+0000 7fdaee7fc640 1 --2- 172.21.15.3:0/2133969888 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdad8004800 0x7fdad8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:08.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.746+0000 7fdaee7fc640 1 --2- 172.21.15.3:0/2133969888 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdad80051c0 0x7fdad800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:08.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.746+0000 7fdaee7fc640 1 -- 172.21.15.3:0/2133969888 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fdaf00b4e40 con 0x7fdad80039b0 2024-05-09T06:37:08.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.746+0000 7fdaee7fc640 1 -- 172.21.15.3:0/2133969888 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fdad800c160 con 0x7fdad8004800 2024-05-09T06:37:08.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.746+0000 7fdaee7fc640 1 -- 172.21.15.3:0/2133969888 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fdad800c360 con 0x7fdad80051c0 2024-05-09T06:37:08.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.746+0000 7fdaeeffd640 1 --2- 172.21.15.3:0/2133969888 >> v2:172.21.15.190:3300/0 conn(0x7fdaf00cdec0 0x7fdaf015c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T06:37:08.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.746+0000 7fdaef7fe640 1 --2- 172.21.15.3:0/2133969888 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdad80039b0 0x7fdad8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:08.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.746+0000 7fdaeffff640 1 --2- 172.21.15.3:0/2133969888 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdad80051c0 0x7fdad800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:08.731 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.746+0000 7fdaeeffd640 1 --2- 172.21.15.3:0/2133969888 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdad8004800 0x7fdad8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:08.731 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.746+0000 7fdaef7fe640 1 -- 172.21.15.3:0/2133969888 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdad80051c0 msgr2=0x7fdad800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:37:08.731 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.746+0000 7fdaef7fe640 1 --2- 172.21.15.3:0/2133969888 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdad80051c0 0x7fdad800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:08.731 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.746+0000 7fdaef7fe640 1 -- 172.21.15.3:0/2133969888 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdad8004800 msgr2=0x7fdad8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:37:08.731 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.746+0000 7fdaef7fe640 1 --2- 172.21.15.3:0/2133969888 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdad8004800 0x7fdad8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:08.731 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.746+0000 7fdaef7fe640 1 -- 172.21.15.3:0/2133969888 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdad8003520 con 0x7fdad80039b0 2024-05-09T06:37:08.731 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.746+0000 7fdaeffff640 1 --2- 172.21.15.3:0/2133969888 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdad80051c0 0x7fdad800b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:37:08.731 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.746+0000 7fdaeeffd640 1 --2- 172.21.15.3:0/2133969888 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdad8004800 0x7fdad8004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:37:08.731 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.746+0000 7fdaeffff640 1 -- 172.21.15.3:0/2133969888 reap_dead start 2024-05-09T06:37:08.731 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.747+0000 7fdaef7fe640 1 --2- 172.21.15.3:0/2133969888 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdad80039b0 0x7fdad8003dd0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fdae000c240 tx=0x7fdad8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f4484f4f9d925ba server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:08.731 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.747+0000 7fdaee7fc640 1 -- 172.21.15.3:0/2133969888 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdae0007cf0 con 0x7fdad80039b0 2024-05-09T06:37:08.731 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.747+0000 7fdaee7fc640 1 -- 172.21.15.3:0/2133969888 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdae00058d0 con 0x7fdad80039b0 2024-05-09T06:37:08.731 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.747+0000 7fdaee7fc640 1 -- 172.21.15.3:0/2133969888 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdae0005d20 con 0x7fdad80039b0 2024-05-09T06:37:08.731 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.748+0000 7fdaf60a5640 1 -- 172.21.15.3:0/2133969888 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdad80039b0 msgr2=0x7fdad8003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:08.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.748+0000 7fdaf60a5640 1 --2- 172.21.15.3:0/2133969888 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdad80039b0 0x7fdad8003dd0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fdae000c240 tx=0x7fdad8005830 comp rx=0 tx=0).stop 2024-05-09T06:37:08.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.748+0000 7fdaf60a5640 1 -- 172.21.15.3:0/2133969888 shutdown_connections 2024-05-09T06:37:08.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.748+0000 7fdaf60a5640 1 --2- 172.21.15.3:0/2133969888 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdad80039b0 0x7fdad8003dd0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:08.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.748+0000 7fdaf60a5640 1 -- 172.21.15.3:0/2133969888 >> 172.21.15.3:0/2133969888 conn(0x7fdaf006b640 msgr2=0x7fdaf006ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:08.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.748+0000 7fdaf60a5640 1 -- 172.21.15.3:0/2133969888 shutdown_connections 2024-05-09T06:37:08.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.748+0000 7fdaf60a5640 1 -- 172.21.15.3:0/2133969888 wait complete. 2024-05-09T06:37:08.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.749+0000 7fdaf60a5640 1 Processor -- start 2024-05-09T06:37:08.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.749+0000 7fdaf60a5640 1 -- start start 2024-05-09T06:37:08.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.750+0000 7fdaf60a5640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdad80039b0 0x7fdaf013ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:08.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.750+0000 7fdaf60a5640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdaf013d4d0 0x7fdaf013d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:08.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.750+0000 7fdaf60a5640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdaf00b0770 0x7fdaf00b0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:08.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.750+0000 7fdaf60a5640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fdaf00b4860 con 0x7fdaf013d4d0 2024-05-09T06:37:08.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.750+0000 7fdaf60a5640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fdaf006d6b0 con 0x7fdaf00b0770 2024-05-09T06:37:08.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.750+0000 7fdaf60a5640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fdaf015c690 con 0x7fdad80039b0 2024-05-09T06:37:08.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.750+0000 7fdaeeffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdaf013d4d0 0x7fdaf013d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:08.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.750+0000 7fdaeeffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdaf013d4d0 0x7fdaf013d950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:39632/0 (socket says 172.21.15.3:39632) 2024-05-09T06:37:08.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.750+0000 7fdaeeffd640 1 -- 172.21.15.3:0/2621402698 learned_addr learned my addr 172.21.15.3:0/2621402698 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:37:08.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.751+0000 7fdaeffff640 1 --2- 172.21.15.3:0/2621402698 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdaf00b0770 0x7fdaf00b0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:08.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.751+0000 7fdaef7fe640 1 --2- 172.21.15.3:0/2621402698 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdad80039b0 0x7fdaf013ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:08.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.751+0000 7fdaeeffd640 1 -- 172.21.15.3:0/2621402698 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdad80039b0 msgr2=0x7fdaf013ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:08.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.751+0000 7fdaeeffd640 1 --2- 172.21.15.3:0/2621402698 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdad80039b0 0x7fdaf013ac50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:08.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.751+0000 7fdaeeffd640 1 -- 172.21.15.3:0/2621402698 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdaf00b0770 msgr2=0x7fdaf00b0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:08.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.751+0000 7fdaeeffd640 1 --2- 172.21.15.3:0/2621402698 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdaf00b0770 0x7fdaf00b0bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:08.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.751+0000 7fdaeeffd640 1 -- 172.21.15.3:0/2621402698 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdaf013e000 con 0x7fdaf013d4d0 2024-05-09T06:37:08.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.751+0000 7fdaeffff640 1 --2- 172.21.15.3:0/2621402698 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdaf00b0770 0x7fdaf00b0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:37:08.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.751+0000 7fdaeeffd640 1 --2- 172.21.15.3:0/2621402698 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdaf013d4d0 0x7fdaf013d950 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fdae4011860 tx=0x7fdae4011d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:08.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.751+0000 7fdad7fff640 1 -- 172.21.15.3:0/2621402698 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdae400fbd0 con 0x7fdaf013d4d0 2024-05-09T06:37:08.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.751+0000 7fdad7fff640 1 -- 172.21.15.3:0/2621402698 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdae400fd70 con 0x7fdaf013d4d0 2024-05-09T06:37:08.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.751+0000 7fdaf60a5640 1 -- 172.21.15.3:0/2621402698 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdaf013e2e0 con 0x7fdaf013d4d0 2024-05-09T06:37:08.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.751+0000 7fdad7fff640 1 -- 172.21.15.3:0/2621402698 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdae4015660 con 0x7fdaf013d4d0 2024-05-09T06:37:08.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.751+0000 7fdaf60a5640 1 -- 172.21.15.3:0/2621402698 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdaf013e8a0 con 0x7fdaf013d4d0 2024-05-09T06:37:08.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.752+0000 7fdaf60a5640 1 -- 172.21.15.3:0/2621402698 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdaf0069550 con 0x7fdaf013d4d0 2024-05-09T06:37:08.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.753+0000 7fdad7fff640 1 -- 172.21.15.3:0/2621402698 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fdae4015800 con 0x7fdaf013d4d0 2024-05-09T06:37:08.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.753+0000 7fdad7fff640 1 --2- 172.21.15.3:0/2621402698 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fdadc06be60 0x7fdadc06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:08.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.753+0000 7fdad7fff640 1 -- 172.21.15.3:0/2621402698 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fdae408b9f0 con 0x7fdaf013d4d0 2024-05-09T06:37:08.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.753+0000 7fdaef7fe640 1 --2- 172.21.15.3:0/2621402698 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fdadc06be60 0x7fdadc06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:08.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.754+0000 7fdaef7fe640 1 --2- 172.21.15.3:0/2621402698 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fdadc06be60 0x7fdadc06e320 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fdae0033000 tx=0x7fdae0009030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:08.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.757+0000 7fdad7fff640 1 -- 172.21.15.3:0/2621402698 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fdae4019030 con 0x7fdaf013d4d0 2024-05-09T06:37:08.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.969+0000 7fdaf60a5640 1 -- 172.21.15.3:0/2621402698 --> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvol46440", "group_name": "subvol_grp46030", "target": ["mon-mgr", ""]}) -- 0x7fdaf00b3ce0 con 0x7fdadc06be60 2024-05-09T06:37:08.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.991+0000 7fdad7fff640 1 -- 172.21.15.3:0/2621402698 <== mgr.4105 v2:172.21.15.190:6832/975893321 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fdaf00b3ce0 con 0x7fdadc06be60 2024-05-09T06:37:08.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.993+0000 7fdaf60a5640 1 -- 172.21.15.3:0/2621402698 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fdadc06be60 msgr2=0x7fdadc06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:08.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.993+0000 7fdaf60a5640 1 --2- 172.21.15.3:0/2621402698 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fdadc06be60 0x7fdadc06e320 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fdae0033000 tx=0x7fdae0009030 comp rx=0 tx=0).stop 2024-05-09T06:37:08.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.993+0000 7fdaf60a5640 1 -- 172.21.15.3:0/2621402698 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdaf013d4d0 msgr2=0x7fdaf013d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:08.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.993+0000 7fdaf60a5640 1 --2- 172.21.15.3:0/2621402698 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdaf013d4d0 0x7fdaf013d950 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fdae4011860 tx=0x7fdae4011d20 comp rx=0 tx=0).stop 2024-05-09T06:37:08.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.993+0000 7fdaf60a5640 1 -- 172.21.15.3:0/2621402698 shutdown_connections 2024-05-09T06:37:08.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.993+0000 7fdaf60a5640 1 --2- 172.21.15.3:0/2621402698 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fdadc06be60 0x7fdadc06e320 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:08.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.993+0000 7fdaf60a5640 1 --2- 172.21.15.3:0/2621402698 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdaf00b0770 0x7fdaf00b0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:08.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.993+0000 7fdaf60a5640 1 --2- 172.21.15.3:0/2621402698 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdaf013d4d0 0x7fdaf013d950 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:08.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.993+0000 7fdaf60a5640 1 --2- 172.21.15.3:0/2621402698 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdad80039b0 0x7fdaf013ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:08.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.994+0000 7fdaf60a5640 1 -- 172.21.15.3:0/2621402698 >> 172.21.15.3:0/2621402698 conn(0x7fdaf006b640 msgr2=0x7fdaf006ba20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:08.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.994+0000 7fdaf60a5640 1 -- 172.21.15.3:0/2621402698 shutdown_connections 2024-05-09T06:37:08.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:08.994+0000 7fdaf60a5640 1 -- 172.21.15.3:0/2621402698 wait complete. 2024-05-09T06:37:08.985 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs subvol43941 --group_name subvol_grp46030 2024-05-09T06:37:09.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:09.114+0000 7f0f6c6fc640 1 Processor -- start 2024-05-09T06:37:09.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:09.114+0000 7f0f6c6fc640 1 -- start start 2024-05-09T06:37:09.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:09.115+0000 7f0f6c6fc640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0f640b32b0 0x7f0f640b71d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:09.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:09.115+0000 7f0f6c6fc640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0f640d2620 con 0x7f0f640cee30 2024-05-09T06:37:09.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:09.115+0000 7f0f6c6fc640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0f640d2900 con 0x7f0f6415e400 2024-05-09T06:37:09.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:09.115+0000 7f0f6c6fc640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f0f640d2f50 con 0x7f0f640b32b0 2024-05-09T06:37:09.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:09.115+0000 7f0f6ac99640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f0f640cee30 0x7f0f640cf2c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:58112/0 (socket says 172.21.15.3:58112) 2024-05-09T06:37:09.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:09.115+0000 7f0f6ac99640 1 -- 172.21.15.3:0/779100405 learned_addr learned my addr 172.21.15.3:0/779100405 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-05-09T06:37:09.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:09.116+0000 7f0f69c97640 1 --2- 172.21.15.3:0/779100405 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0f640b32b0 0x7f0f640b71d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:09.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:09.116+0000 7f0f69496640 1 -- 172.21.15.3:0/779100405 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1745646566 0 0) 0x7f0f640d2620 con 0x7f0f640cee30 2024-05-09T06:37:09.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:09.116+0000 7f0f69496640 1 -- 172.21.15.3:0/779100405 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0f4c003520 con 0x7f0f640cee30 2024-05-09T06:37:09.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:09.116+0000 7f0f69496640 1 -- 172.21.15.3:0/779100405 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1477005438 0 0) 0x7f0f640d2900 con 0x7f0f6415e400 2024-05-09T06:37:09.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:09.116+0000 7f0f69496640 1 -- 172.21.15.3:0/779100405 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0f640d2620 con 0x7f0f6415e400 2024-05-09T06:37:09.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:09.116+0000 7f0f69496640 1 -- 172.21.15.3:0/779100405 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2803444716 0 0) 0x7f0f4c003520 con 0x7f0f640cee30 2024-05-09T06:37:09.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:09.116+0000 7f0f69496640 1 -- 172.21.15.3:0/779100405 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0f640b32b0 msgr2=0x7f0f640b71d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:37:09.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:09.116+0000 7f0f69496640 1 --2- 172.21.15.3:0/779100405 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0f640b32b0 0x7f0f640b71d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:09.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:09.116+0000 7f0f69496640 1 -- 172.21.15.3:0/779100405 >> v1:172.21.15.190:6789/0 conn(0x7f0f6415e400 legacy=0x7f0f640b99c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:09.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:09.116+0000 7f0f69496640 1 -- 172.21.15.3:0/779100405 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f640d2be0 con 0x7f0f640cee30 2024-05-09T06:37:09.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:09.116+0000 7f0f69496640 1 -- 172.21.15.3:0/779100405 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 563312031 0 0) 0x7f0f540019f0 con 0x7f0f640cee30 2024-05-09T06:37:09.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:09.116+0000 7f0f69496640 1 -- 172.21.15.3:0/779100405 >> v1:172.21.15.3:6789/0 conn(0x7f0f640cee30 legacy=0x7f0f640cf2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:09.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:09.116+0000 7f0f69496640 1 --2- 172.21.15.3:0/779100405 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0f4c003b70 0x7f0f4c003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:09.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:09.116+0000 7f0f69496640 1 --2- 172.21.15.3:0/779100405 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0f640b32b0 0x7f0f4c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:09.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:09.116+0000 7f0f69496640 1 --2- 172.21.15.3:0/779100405 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0f4c004d40 0x7f0f4c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:09.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:09.116+0000 7f0f69496640 1 -- 172.21.15.3:0/779100405 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f0f640d2f50 con 0x7f0f4c004d40 2024-05-09T06:37:09.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:09.116+0000 7f0f69496640 1 -- 172.21.15.3:0/779100405 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f0f4c00ba70 con 0x7f0f4c003b70 2024-05-09T06:37:09.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:09.116+0000 7f0f69496640 1 -- 172.21.15.3:0/779100405 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f0f4c00bc20 con 0x7f0f640b32b0 2024-05-09T06:37:09.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:09.116+0000 7f0f6ac99640 1 --2- 172.21.15.3:0/779100405 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0f4c004d40 0x7f0f4c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:09.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:09.116+0000 7f0f69c97640 1 --2- 172.21.15.3:0/779100405 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0f640b32b0 0x7f0f4c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:09.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:09.116+0000 7f0f6a498640 1 --2- 172.21.15.3:0/779100405 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0f4c003b70 0x7f0f4c003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:09.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:09.117+0000 7f0f6ac99640 1 -- 172.21.15.3:0/779100405 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0f640b32b0 msgr2=0x7f0f4c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:37:09.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:09.117+0000 7f0f6ac99640 1 --2- 172.21.15.3:0/779100405 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0f640b32b0 0x7f0f4c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:09.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:09.117+0000 7f0f6ac99640 1 -- 172.21.15.3:0/779100405 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0f4c003b70 msgr2=0x7f0f4c003f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:37:09.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:09.117+0000 7f0f6ac99640 1 --2- 172.21.15.3:0/779100405 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0f4c003b70 0x7f0f4c003f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:09.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:09.117+0000 7f0f6ac99640 1 -- 172.21.15.3:0/779100405 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f4c003520 con 0x7f0f4c004d40 2024-05-09T06:37:09.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:09.117+0000 7f0f6ac99640 1 --2- 172.21.15.3:0/779100405 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0f4c004d40 0x7f0f4c0051c0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f0f5400a800 tx=0x7f0f4c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b16566442aa71ecf server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:09.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:09.117+0000 7f0f69496640 1 -- 172.21.15.3:0/779100405 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f5400a430 con 0x7f0f4c004d40 2024-05-09T06:37:09.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:09.117+0000 7f0f69496640 1 -- 172.21.15.3:0/779100405 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f5402f070 con 0x7f0f4c004d40 2024-05-09T06:37:09.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:09.117+0000 7f0f69496640 1 -- 172.21.15.3:0/779100405 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f5402bc30 con 0x7f0f4c004d40 2024-05-09T06:37:09.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:09.118+0000 7f0f6c6fc640 1 -- 172.21.15.3:0/779100405 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0f4c004d40 msgr2=0x7f0f4c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:09.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:09.118+0000 7f0f6c6fc640 1 --2- 172.21.15.3:0/779100405 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0f4c004d40 0x7f0f4c0051c0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f0f5400a800 tx=0x7f0f4c005830 comp rx=0 tx=0).stop 2024-05-09T06:37:09.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:09.118+0000 7f0f6ac99640 1 -- 172.21.15.3:0/779100405 reap_dead start 2024-05-09T06:37:09.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:09.118+0000 7f0f6c6fc640 1 -- 172.21.15.3:0/779100405 shutdown_connections 2024-05-09T06:37:09.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:09.118+0000 7f0f6c6fc640 1 -- 172.21.15.3:0/779100405 >> 172.21.15.3:0/779100405 conn(0x7f0f640675c0 msgr2=0x7f0f640d0be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:09.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:09.118+0000 7f0f6c6fc640 1 -- 172.21.15.3:0/779100405 shutdown_connections 2024-05-09T06:37:09.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:09.118+0000 7f0f6c6fc640 1 -- 172.21.15.3:0/779100405 wait complete. 2024-05-09T06:37:09.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:09.119+0000 7f0f6c6fc640 1 Processor -- start 2024-05-09T06:37:09.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:09.119+0000 7f0f6c6fc640 1 -- start start 2024-05-09T06:37:09.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:09.120+0000 7f0f6c6fc640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0f4c004d40 0x7f0f6414c910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:09.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:09.120+0000 7f0f6c6fc640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0f640cceb0 0x7f0f640cd330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:09.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:09.120+0000 7f0f6c6fc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0f640b81b0 0x7f0f640b8630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:09.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:09.120+0000 7f0f6c6fc640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f0f640b75a0 con 0x7f0f640b81b0 2024-05-09T06:37:09.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:09.120+0000 7f0f6c6fc640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f0f640b7420 con 0x7f0f4c004d40 2024-05-09T06:37:09.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:09.120+0000 7f0f6c6fc640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f0f640b7720 con 0x7f0f640cceb0 2024-05-09T06:37:09.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:09.120+0000 7f0f6ac99640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0f640b81b0 0x7f0f640b8630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:09.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:09.120+0000 7f0f6ac99640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0f640b81b0 0x7f0f640b8630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:39648/0 (socket says 172.21.15.3:39648) 2024-05-09T06:37:09.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:09.120+0000 7f0f6ac99640 1 -- 172.21.15.3:0/2069812106 learned_addr learned my addr 172.21.15.3:0/2069812106 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:37:09.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:09.120+0000 7f0f6ac99640 1 -- 172.21.15.3:0/2069812106 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0f640cceb0 msgr2=0x7f0f640cd330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:09.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:09.120+0000 7f0f6ac99640 1 --2- 172.21.15.3:0/2069812106 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0f640cceb0 0x7f0f640cd330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:09.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:09.120+0000 7f0f6ac99640 1 -- 172.21.15.3:0/2069812106 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0f4c004d40 msgr2=0x7f0f6414c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:09.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:09.120+0000 7f0f6ac99640 1 --2- 172.21.15.3:0/2069812106 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0f4c004d40 0x7f0f6414c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:09.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:09.120+0000 7f0f6ac99640 1 -- 172.21.15.3:0/2069812106 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f6414ce50 con 0x7f0f640b81b0 2024-05-09T06:37:09.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:09.121+0000 7f0f6ac99640 1 --2- 172.21.15.3:0/2069812106 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0f640b81b0 0x7f0f640b8630 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f0f5402b4a0 tx=0x7f0f54001d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:09.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:09.121+0000 7f0f62ffd640 1 -- 172.21.15.3:0/2069812106 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f5400a430 con 0x7f0f640b81b0 2024-05-09T06:37:09.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:09.121+0000 7f0f62ffd640 1 -- 172.21.15.3:0/2069812106 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f5402bc00 con 0x7f0f640b81b0 2024-05-09T06:37:09.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:09.121+0000 7f0f6c6fc640 1 -- 172.21.15.3:0/2069812106 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f641f7430 con 0x7f0f640b81b0 2024-05-09T06:37:09.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:09.121+0000 7f0f62ffd640 1 -- 172.21.15.3:0/2069812106 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f54001f90 con 0x7f0f640b81b0 2024-05-09T06:37:09.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:09.121+0000 7f0f6c6fc640 1 -- 172.21.15.3:0/2069812106 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f641f79f0 con 0x7f0f640b81b0 2024-05-09T06:37:09.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:09.122+0000 7f0f6c6fc640 1 -- 172.21.15.3:0/2069812106 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f64069550 con 0x7f0f640b81b0 2024-05-09T06:37:09.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:09.122+0000 7f0f62ffd640 1 -- 172.21.15.3:0/2069812106 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0f54034470 con 0x7f0f640b81b0 2024-05-09T06:37:09.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:09.123+0000 7f0f62ffd640 1 --2- 172.21.15.3:0/2069812106 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f0f5006be60 0x7f0f5006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:09.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:09.123+0000 7f0f62ffd640 1 -- 172.21.15.3:0/2069812106 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f0f540aa5f0 con 0x7f0f640b81b0 2024-05-09T06:37:09.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:09.123+0000 7f0f6a498640 1 --2- 172.21.15.3:0/2069812106 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f0f5006be60 0x7f0f5006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:09.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:09.124+0000 7f0f6a498640 1 --2- 172.21.15.3:0/2069812106 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f0f5006be60 0x7f0f5006e320 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f0f640cab60 tx=0x7f0f5c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:09.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:09.127+0000 7f0f62ffd640 1 -- 172.21.15.3:0/2069812106 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f0f54079030 con 0x7f0f640b81b0 2024-05-09T06:37:09.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:09.338+0000 7f0f6c6fc640 1 -- 172.21.15.3:0/2069812106 --> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvol43941", "group_name": "subvol_grp46030", "target": ["mon-mgr", ""]}) -- 0x7f0f640cf780 con 0x7f0f5006be60 2024-05-09T06:37:09.346 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:09.365+0000 7f0f62ffd640 1 -- 172.21.15.3:0/2069812106 <== mgr.4105 v2:172.21.15.190:6832/975893321 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f0f640cf780 con 0x7f0f5006be60 2024-05-09T06:37:09.348 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:09.367+0000 7f0f6c6fc640 1 -- 172.21.15.3:0/2069812106 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f0f5006be60 msgr2=0x7f0f5006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:09.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:09.367+0000 7f0f6c6fc640 1 --2- 172.21.15.3:0/2069812106 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f0f5006be60 0x7f0f5006e320 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f0f640cab60 tx=0x7f0f5c009040 comp rx=0 tx=0).stop 2024-05-09T06:37:09.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:09.367+0000 7f0f6c6fc640 1 -- 172.21.15.3:0/2069812106 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0f640b81b0 msgr2=0x7f0f640b8630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:09.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:09.367+0000 7f0f6c6fc640 1 --2- 172.21.15.3:0/2069812106 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0f640b81b0 0x7f0f640b8630 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f0f5402b4a0 tx=0x7f0f54001d40 comp rx=0 tx=0).stop 2024-05-09T06:37:09.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:09.367+0000 7f0f6c6fc640 1 -- 172.21.15.3:0/2069812106 shutdown_connections 2024-05-09T06:37:09.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:09.367+0000 7f0f6c6fc640 1 --2- 172.21.15.3:0/2069812106 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f0f5006be60 0x7f0f5006e320 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:09.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:09.368+0000 7f0f6c6fc640 1 --2- 172.21.15.3:0/2069812106 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0f640b81b0 0x7f0f640b8630 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:09.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:09.368+0000 7f0f6c6fc640 1 --2- 172.21.15.3:0/2069812106 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0f640cceb0 0x7f0f640cd330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:09.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:09.368+0000 7f0f6c6fc640 1 --2- 172.21.15.3:0/2069812106 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0f4c004d40 0x7f0f6414c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:09.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:09.368+0000 7f0f6c6fc640 1 -- 172.21.15.3:0/2069812106 >> 172.21.15.3:0/2069812106 conn(0x7f0f640675c0 msgr2=0x7f0f640b50a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:09.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:09.368+0000 7f0f6c6fc640 1 -- 172.21.15.3:0/2069812106 shutdown_connections 2024-05-09T06:37:09.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:09.368+0000 7f0f6c6fc640 1 -- 172.21.15.3:0/2069812106 wait complete. 2024-05-09T06:37:09.359 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs subvol3942 --group_name subvol_grp46030 2024-05-09T06:37:09.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:09.488+0000 7fba5a0ee640 1 Processor -- start 2024-05-09T06:37:09.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:09.488+0000 7fba5a0ee640 1 -- start start 2024-05-09T06:37:09.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:09.489+0000 7fba5a0ee640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fba540b32b0 0x7fba540b71d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:09.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:09.489+0000 7fba5a0ee640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fba540cee30 0x7fba540cf2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:09.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:09.489+0000 7fba5a0ee640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fba540cf800 con 0x7fba5415e400 2024-05-09T06:37:09.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:09.489+0000 7fba5a0ee640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7fba540d2f50 con 0x7fba540b32b0 2024-05-09T06:37:09.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:09.489+0000 7fba5a0ee640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fba540d2620 con 0x7fba540cee30 2024-05-09T06:37:09.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:09.489+0000 7fba53fff640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fba540cee30 0x7fba540cf2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:09.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:09.489+0000 7fba52ffd640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fba540b32b0 0x7fba540b71d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:09.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:09.489+0000 7fba53fff640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fba540cee30 0x7fba540cf2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.3:42848/0 (socket says 172.21.15.3:42848) 2024-05-09T06:37:09.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:09.489+0000 7fba52ffd640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fba540b32b0 0x7fba540b71d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.3:52080/0 (socket says 172.21.15.3:52080) 2024-05-09T06:37:09.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:09.489+0000 7fba53fff640 1 -- 172.21.15.3:0/1521495656 learned_addr learned my addr 172.21.15.3:0/1521495656 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:37:09.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:09.490+0000 7fba52ffd640 1 -- 172.21.15.3:0/1521495656 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fba540cee30 msgr2=0x7fba540cf2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:37:09.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:09.490+0000 7fba52ffd640 1 --2- 172.21.15.3:0/1521495656 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fba540cee30 0x7fba540cf2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:09.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:09.490+0000 7fba52ffd640 1 -- 172.21.15.3:0/1521495656 >> v1:172.21.15.190:6789/0 conn(0x7fba5415e400 legacy=0x7fba540b99c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:09.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:09.490+0000 7fba52ffd640 1 -- 172.21.15.3:0/1521495656 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba540d2900 con 0x7fba540b32b0 2024-05-09T06:37:09.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:09.490+0000 7fba52ffd640 1 --2- 172.21.15.3:0/1521495656 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fba540b32b0 0x7fba540b71d0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fba480097a0 tx=0x7fba4802a210 comp rx=0 tx=0).ready entity=mon.1 client_cookie=259df8f3360ef920 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:09.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:09.491+0000 7fba527fc640 1 -- 172.21.15.3:0/1521495656 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba48037070 con 0x7fba540b32b0 2024-05-09T06:37:09.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:09.491+0000 7fba527fc640 1 -- 172.21.15.3:0/1521495656 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba48032440 con 0x7fba540b32b0 2024-05-09T06:37:09.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:09.491+0000 7fba527fc640 1 -- 172.21.15.3:0/1521495656 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba48032810 con 0x7fba540b32b0 2024-05-09T06:37:09.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:09.491+0000 7fba5a0ee640 1 -- 172.21.15.3:0/1521495656 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fba540b32b0 msgr2=0x7fba540b71d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:09.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:09.491+0000 7fba5a0ee640 1 --2- 172.21.15.3:0/1521495656 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fba540b32b0 0x7fba540b71d0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fba480097a0 tx=0x7fba4802a210 comp rx=0 tx=0).stop 2024-05-09T06:37:09.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:09.491+0000 7fba5a0ee640 1 -- 172.21.15.3:0/1521495656 shutdown_connections 2024-05-09T06:37:09.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:09.491+0000 7fba5a0ee640 1 --2- 172.21.15.3:0/1521495656 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fba540cee30 0x7fba540cf2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:09.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:09.491+0000 7fba5a0ee640 1 --2- 172.21.15.3:0/1521495656 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fba540b32b0 0x7fba540b71d0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:09.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:09.491+0000 7fba5a0ee640 1 -- 172.21.15.3:0/1521495656 >> 172.21.15.3:0/1521495656 conn(0x7fba540675c0 msgr2=0x7fba540d0be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:09.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:09.491+0000 7fba5a0ee640 1 -- 172.21.15.3:0/1521495656 shutdown_connections 2024-05-09T06:37:09.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:09.491+0000 7fba5a0ee640 1 -- 172.21.15.3:0/1521495656 wait complete. 2024-05-09T06:37:09.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:09.492+0000 7fba5a0ee640 1 Processor -- start 2024-05-09T06:37:09.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:09.493+0000 7fba5a0ee640 1 -- start start 2024-05-09T06:37:09.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:09.493+0000 7fba5a0ee640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fba540b32b0 0x7fba541612d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:09.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:09.493+0000 7fba5a0ee640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fba540cee30 0x7fba540cdc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:09.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:09.493+0000 7fba5a0ee640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fba5415e400 0x7fba540cc470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:09.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:09.493+0000 7fba5a0ee640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fba540b7890 con 0x7fba540cee30 2024-05-09T06:37:09.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:09.493+0000 7fba5a0ee640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fba540b7710 con 0x7fba5415e400 2024-05-09T06:37:09.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:09.493+0000 7fba5a0ee640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fba540d33e0 con 0x7fba540b32b0 2024-05-09T06:37:09.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:09.493+0000 7fba52ffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fba540cee30 0x7fba540cdc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:09.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:09.494+0000 7fba52ffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fba540cee30 0x7fba540cdc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:39662/0 (socket says 172.21.15.3:39662) 2024-05-09T06:37:09.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:09.494+0000 7fba52ffd640 1 -- 172.21.15.3:0/3829896304 learned_addr learned my addr 172.21.15.3:0/3829896304 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:37:09.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:09.494+0000 7fba53fff640 1 --2- 172.21.15.3:0/3829896304 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fba5415e400 0x7fba540cc470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:09.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:09.494+0000 7fba537fe640 1 --2- 172.21.15.3:0/3829896304 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fba540b32b0 0x7fba541612d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:09.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:09.494+0000 7fba52ffd640 1 -- 172.21.15.3:0/3829896304 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fba540b32b0 msgr2=0x7fba541612d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:09.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:09.494+0000 7fba52ffd640 1 --2- 172.21.15.3:0/3829896304 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fba540b32b0 0x7fba541612d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:09.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:09.494+0000 7fba52ffd640 1 -- 172.21.15.3:0/3829896304 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fba5415e400 msgr2=0x7fba540cc470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:09.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:09.494+0000 7fba52ffd640 1 --2- 172.21.15.3:0/3829896304 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fba5415e400 0x7fba540cc470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:09.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:09.494+0000 7fba52ffd640 1 -- 172.21.15.3:0/3829896304 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba5415e960 con 0x7fba540cee30 2024-05-09T06:37:09.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:09.494+0000 7fba52ffd640 1 --2- 172.21.15.3:0/3829896304 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fba540cee30 0x7fba540cdc30 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fba4802a250 tx=0x7fba480046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:09.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:09.494+0000 7fba3bfff640 1 -- 172.21.15.3:0/3829896304 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba48041020 con 0x7fba540cee30 2024-05-09T06:37:09.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:09.494+0000 7fba3bfff640 1 -- 172.21.15.3:0/3829896304 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba48030700 con 0x7fba540cee30 2024-05-09T06:37:09.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:09.494+0000 7fba5a0ee640 1 -- 172.21.15.3:0/3829896304 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba5415ec40 con 0x7fba540cee30 2024-05-09T06:37:09.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:09.494+0000 7fba3bfff640 1 -- 172.21.15.3:0/3829896304 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba48037040 con 0x7fba540cee30 2024-05-09T06:37:09.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:09.494+0000 7fba5a0ee640 1 -- 172.21.15.3:0/3829896304 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba5415f200 con 0x7fba540cee30 2024-05-09T06:37:09.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:09.495+0000 7fba5a0ee640 1 -- 172.21.15.3:0/3829896304 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba54069550 con 0x7fba540cee30 2024-05-09T06:37:09.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:09.496+0000 7fba3bfff640 1 -- 172.21.15.3:0/3829896304 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fba48044020 con 0x7fba540cee30 2024-05-09T06:37:09.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:09.496+0000 7fba3bfff640 1 --2- 172.21.15.3:0/3829896304 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fba4006be30 0x7fba4006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:09.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:09.496+0000 7fba3bfff640 1 -- 172.21.15.3:0/3829896304 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fba480ab550 con 0x7fba540cee30 2024-05-09T06:37:09.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:09.496+0000 7fba537fe640 1 --2- 172.21.15.3:0/3829896304 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fba4006be30 0x7fba4006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:09.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:09.497+0000 7fba537fe640 1 --2- 172.21.15.3:0/3829896304 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fba4006be30 0x7fba4006e2f0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fba440016e0 tx=0x7fba44009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:09.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:09.500+0000 7fba3bfff640 1 -- 172.21.15.3:0/3829896304 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fba4804b030 con 0x7fba540cee30 2024-05-09T06:37:09.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:09.610+0000 7fba3bfff640 1 -- 172.21.15.3:0/3829896304 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mgrmap(e 6) ==== 90549+0+0 (secure 0 0 0) 0x7fba48075f20 con 0x7fba540cee30 2024-05-09T06:37:09.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:09.712+0000 7fba5a0ee640 1 -- 172.21.15.3:0/3829896304 --> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvol3942", "group_name": "subvol_grp46030", "target": ["mon-mgr", ""]}) -- 0x7fba540ce170 con 0x7fba4006be30 2024-05-09T06:37:09.720 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:09.740+0000 7fba3bfff640 1 -- 172.21.15.3:0/3829896304 <== mgr.4105 v2:172.21.15.190:6832/975893321 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fba44001380 con 0x7fba4006be30 2024-05-09T06:37:09.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:09.742+0000 7fba5a0ee640 1 -- 172.21.15.3:0/3829896304 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fba4006be30 msgr2=0x7fba4006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:09.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:09.742+0000 7fba5a0ee640 1 --2- 172.21.15.3:0/3829896304 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fba4006be30 0x7fba4006e2f0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fba440016e0 tx=0x7fba44009040 comp rx=0 tx=0).stop 2024-05-09T06:37:09.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:09.742+0000 7fba5a0ee640 1 -- 172.21.15.3:0/3829896304 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fba540cee30 msgr2=0x7fba540cdc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:09.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:09.742+0000 7fba5a0ee640 1 --2- 172.21.15.3:0/3829896304 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fba540cee30 0x7fba540cdc30 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fba4802a250 tx=0x7fba480046c0 comp rx=0 tx=0).stop 2024-05-09T06:37:09.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:09.742+0000 7fba5a0ee640 1 -- 172.21.15.3:0/3829896304 shutdown_connections 2024-05-09T06:37:09.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:09.742+0000 7fba5a0ee640 1 --2- 172.21.15.3:0/3829896304 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fba4006be30 0x7fba4006e2f0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:09.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:09.742+0000 7fba5a0ee640 1 --2- 172.21.15.3:0/3829896304 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fba5415e400 0x7fba540cc470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:09.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:09.742+0000 7fba5a0ee640 1 --2- 172.21.15.3:0/3829896304 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fba540cee30 0x7fba540cdc30 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:09.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:09.742+0000 7fba5a0ee640 1 --2- 172.21.15.3:0/3829896304 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fba540b32b0 0x7fba541612d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:09.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:09.742+0000 7fba5a0ee640 1 -- 172.21.15.3:0/3829896304 >> 172.21.15.3:0/3829896304 conn(0x7fba540675c0 msgr2=0x7fba540b48f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:09.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:09.742+0000 7fba5a0ee640 1 -- 172.21.15.3:0/3829896304 shutdown_connections 2024-05-09T06:37:09.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:09.742+0000 7fba5a0ee640 1 -- 172.21.15.3:0/3829896304 wait complete. 2024-05-09T06:37:09.734 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvol46440 subvol_grp46030 2024-05-09T06:37:09.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:09.861+0000 7f55753f9640 1 Processor -- start 2024-05-09T06:37:09.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:09.861+0000 7f55753f9640 1 -- start start 2024-05-09T06:37:09.844 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:09.862+0000 7f55753f9640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f55700b56f0 0x7f55700ae490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:09.844 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:09.862+0000 7f55753f9640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f557006b250 0x7f557006b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:09.844 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:09.862+0000 7f55753f9640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f55700adb40 con 0x7f55700ae0b0 2024-05-09T06:37:09.844 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:09.862+0000 7f55753f9640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f557006cc80 con 0x7f557006b250 2024-05-09T06:37:09.844 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:09.862+0000 7f55753f9640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f55700b5260 con 0x7f55700b56f0 2024-05-09T06:37:09.844 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:09.862+0000 7f556e7fc640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f55700b56f0 0x7f55700ae490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:09.844 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:09.862+0000 7f556f7fe640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f557006b250 0x7f557006b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:09.844 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:09.863+0000 7f556effd640 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7f55700ae0b0 0x7f557015f280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.3:37784/0 (socket says 172.21.15.3:37784) 2024-05-09T06:37:09.844 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:09.863+0000 7f556e7fc640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f55700b56f0 0x7f55700ae490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.3:42872/0 (socket says 172.21.15.3:42872) 2024-05-09T06:37:09.844 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:09.863+0000 7f556effd640 1 -- 172.21.15.3:0/546545711 learned_addr learned my addr 172.21.15.3:0/546545711 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-05-09T06:37:09.844 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:09.863+0000 7f556f7fe640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f557006b250 0x7f557006b6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.3:52098/0 (socket says 172.21.15.3:52098) 2024-05-09T06:37:09.844 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:09.863+0000 7f556e7fc640 1 -- 172.21.15.3:0/546545711 >> v2:172.21.15.190:3300/0 conn(0x7f557006b250 msgr2=0x7f557006b6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:37:09.844 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:09.863+0000 7f556e7fc640 1 --2- 172.21.15.3:0/546545711 >> v2:172.21.15.190:3300/0 conn(0x7f557006b250 0x7f557006b6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:09.845 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:09.863+0000 7f556e7fc640 1 -- 172.21.15.3:0/546545711 >> v1:172.21.15.190:6789/0 conn(0x7f55700ae0b0 legacy=0x7f557015f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:09.845 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:09.863+0000 7f556e7fc640 1 -- 172.21.15.3:0/546545711 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55700ae9d0 con 0x7f55700b56f0 2024-05-09T06:37:09.845 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:09.863+0000 7f556f7fe640 1 --2- 172.21.15.3:0/546545711 >> v2:172.21.15.190:3300/0 conn(0x7f557006b250 0x7f557006b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T06:37:09.846 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:09.864+0000 7f556e7fc640 1 --2- 172.21.15.3:0/546545711 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f55700b56f0 0x7f55700ae490 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f55580097a0 tx=0x7f555802a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a1bf8fd3c7fb80bd server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:09.846 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:09.864+0000 7f556dffb640 1 -- 172.21.15.3:0/546545711 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5558037070 con 0x7f55700b56f0 2024-05-09T06:37:09.846 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:09.864+0000 7f556dffb640 1 -- 172.21.15.3:0/546545711 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f555802ae50 con 0x7f55700b56f0 2024-05-09T06:37:09.846 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:09.864+0000 7f556dffb640 1 -- 172.21.15.3:0/546545711 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5558032660 con 0x7f55700b56f0 2024-05-09T06:37:09.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:09.864+0000 7f55753f9640 1 -- 172.21.15.3:0/546545711 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f55700b56f0 msgr2=0x7f55700ae490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:09.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:09.864+0000 7f55753f9640 1 --2- 172.21.15.3:0/546545711 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f55700b56f0 0x7f55700ae490 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f55580097a0 tx=0x7f555802a100 comp rx=0 tx=0).stop 2024-05-09T06:37:09.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:09.864+0000 7f55753f9640 1 -- 172.21.15.3:0/546545711 shutdown_connections 2024-05-09T06:37:09.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:09.864+0000 7f55753f9640 1 --2- 172.21.15.3:0/546545711 >> v2:172.21.15.190:3300/0 conn(0x7f557006b250 0x7f557006b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:09.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:09.864+0000 7f55753f9640 1 --2- 172.21.15.3:0/546545711 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f55700b56f0 0x7f55700ae490 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:09.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:09.864+0000 7f55753f9640 1 -- 172.21.15.3:0/546545711 >> 172.21.15.3:0/546545711 conn(0x7f55700d05c0 msgr2=0x7f55700d09d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:09.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:09.864+0000 7f55753f9640 1 -- 172.21.15.3:0/546545711 shutdown_connections 2024-05-09T06:37:09.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:09.864+0000 7f55753f9640 1 -- 172.21.15.3:0/546545711 wait complete. 2024-05-09T06:37:09.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:09.866+0000 7f55753f9640 1 Processor -- start 2024-05-09T06:37:09.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:09.866+0000 7f55753f9640 1 -- start start 2024-05-09T06:37:09.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:09.866+0000 7f55753f9640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f557006b250 0x7f557015d320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:09.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:09.867+0000 7f55753f9640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f55700ae0b0 0x7f55701f6890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:09.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:09.867+0000 7f55753f9640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f55700b56f0 0x7f55701f4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:09.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:09.867+0000 7f55753f9640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f55700b4c10 con 0x7f55700ae0b0 2024-05-09T06:37:09.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:09.867+0000 7f55753f9640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f55700b4930 con 0x7f55700b56f0 2024-05-09T06:37:09.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:09.867+0000 7f55753f9640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f55700b4ef0 con 0x7f557006b250 2024-05-09T06:37:09.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:09.867+0000 7f556e7fc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f55700ae0b0 0x7f55701f6890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:09.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:09.867+0000 7f556e7fc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f55700ae0b0 0x7f55701f6890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:39666/0 (socket says 172.21.15.3:39666) 2024-05-09T06:37:09.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:09.867+0000 7f556e7fc640 1 -- 172.21.15.3:0/1166637409 learned_addr learned my addr 172.21.15.3:0/1166637409 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:37:09.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:09.867+0000 7f556effd640 1 --2- 172.21.15.3:0/1166637409 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f557006b250 0x7f557015d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:09.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:09.867+0000 7f556f7fe640 1 --2- 172.21.15.3:0/1166637409 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f55700b56f0 0x7f55701f4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:09.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:09.867+0000 7f556e7fc640 1 -- 172.21.15.3:0/1166637409 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f557006b250 msgr2=0x7f557015d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:09.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:09.867+0000 7f556e7fc640 1 --2- 172.21.15.3:0/1166637409 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f557006b250 0x7f557015d320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:09.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:09.867+0000 7f556e7fc640 1 -- 172.21.15.3:0/1166637409 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f55700b56f0 msgr2=0x7f55701f4db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:09.851 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:09.867+0000 7f556e7fc640 1 --2- 172.21.15.3:0/1166637409 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f55700b56f0 0x7f55701f4db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:09.851 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:09.867+0000 7f556e7fc640 1 -- 172.21.15.3:0/1166637409 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55701f57f0 con 0x7f55700ae0b0 2024-05-09T06:37:09.851 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:09.867+0000 7f556e7fc640 1 --2- 172.21.15.3:0/1166637409 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f55700ae0b0 0x7f55701f6890 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f55580097a0 tx=0x7f55580046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:09.851 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:09.867+0000 7f55677fe640 1 -- 172.21.15.3:0/1166637409 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5558042070 con 0x7f55700ae0b0 2024-05-09T06:37:09.851 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:09.867+0000 7f55677fe640 1 -- 172.21.15.3:0/1166637409 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f555803e460 con 0x7f55700ae0b0 2024-05-09T06:37:09.851 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:09.867+0000 7f55753f9640 1 -- 172.21.15.3:0/1166637409 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55701f5ad0 con 0x7f55700ae0b0 2024-05-09T06:37:09.851 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:09.867+0000 7f55677fe640 1 -- 172.21.15.3:0/1166637409 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5558037040 con 0x7f55700ae0b0 2024-05-09T06:37:09.851 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:09.868+0000 7f55753f9640 1 -- 172.21.15.3:0/1166637409 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55701f6370 con 0x7f55700ae0b0 2024-05-09T06:37:09.851 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:09.868+0000 7f55753f9640 1 -- 172.21.15.3:0/1166637409 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55701f5ec0 con 0x7f55700ae0b0 2024-05-09T06:37:09.851 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:09.869+0000 7f55677fe640 1 -- 172.21.15.3:0/1166637409 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 6) ==== 90549+0+0 (secure 0 0 0) 0x7f5558045070 con 0x7f55700ae0b0 2024-05-09T06:37:09.852 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:09.869+0000 7f55677fe640 1 --2- 172.21.15.3:0/1166637409 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f555006bec0 0x7f555006e380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:09.852 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:09.870+0000 7f55677fe640 1 -- 172.21.15.3:0/1166637409 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f55580ab7e0 con 0x7f55700ae0b0 2024-05-09T06:37:09.852 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:09.870+0000 7f556effd640 1 --2- 172.21.15.3:0/1166637409 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f555006bec0 0x7f555006e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:09.852 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:09.870+0000 7f556effd640 1 --2- 172.21.15.3:0/1166637409 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f555006bec0 0x7f555006e380 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f55600012e0 tx=0x7f5560001990 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:09.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:09.874+0000 7f55677fe640 1 -- 172.21.15.3:0/1166637409 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f5558079c70 con 0x7f55700ae0b0 2024-05-09T06:37:10.065 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:10.085+0000 7f55753f9640 1 -- 172.21.15.3:0/1166637409 --> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvol46440", "group_name": "subvol_grp46030", "target": ["mon-mgr", ""]}) -- 0x7f5570069550 con 0x7f555006bec0 2024-05-09T06:37:10.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:10.097+0000 7f55677fe640 1 -- 172.21.15.3:0/1166637409 <== mgr.4105 v2:172.21.15.190:6832/975893321 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f5570069550 con 0x7f555006bec0 2024-05-09T06:37:10.080 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:10.099+0000 7f55753f9640 1 -- 172.21.15.3:0/1166637409 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f555006bec0 msgr2=0x7f555006e380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:10.081 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:10.099+0000 7f55753f9640 1 --2- 172.21.15.3:0/1166637409 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f555006bec0 0x7f555006e380 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f55600012e0 tx=0x7f5560001990 comp rx=0 tx=0).stop 2024-05-09T06:37:10.081 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:10.099+0000 7f55753f9640 1 -- 172.21.15.3:0/1166637409 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f55700ae0b0 msgr2=0x7f55701f6890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:10.081 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:10.099+0000 7f55753f9640 1 --2- 172.21.15.3:0/1166637409 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f55700ae0b0 0x7f55701f6890 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f55580097a0 tx=0x7f55580046c0 comp rx=0 tx=0).stop 2024-05-09T06:37:10.081 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:10.100+0000 7f55753f9640 1 -- 172.21.15.3:0/1166637409 shutdown_connections 2024-05-09T06:37:10.081 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:10.100+0000 7f55753f9640 1 --2- 172.21.15.3:0/1166637409 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f555006bec0 0x7f555006e380 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:10.081 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:10.100+0000 7f55753f9640 1 --2- 172.21.15.3:0/1166637409 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f55700b56f0 0x7f55701f4db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:10.081 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:10.100+0000 7f55753f9640 1 --2- 172.21.15.3:0/1166637409 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f55700ae0b0 0x7f55701f6890 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:10.081 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:10.100+0000 7f55753f9640 1 --2- 172.21.15.3:0/1166637409 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f557006b250 0x7f557015d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:10.081 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:10.100+0000 7f55753f9640 1 -- 172.21.15.3:0/1166637409 >> 172.21.15.3:0/1166637409 conn(0x7f55700d05c0 msgr2=0x7f557015c7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:10.081 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:10.100+0000 7f55753f9640 1 -- 172.21.15.3:0/1166637409 shutdown_connections 2024-05-09T06:37:10.081 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:10.100+0000 7f55753f9640 1 -- 172.21.15.3:0/1166637409 wait complete. 2024-05-09T06:37:10.082 INFO:tasks.ceph.mgr.x.smithi190.stderr:2024-05-09T06:37:10.102+0000 7faa89227640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-09T06:37:10.082 INFO:tasks.ceph.mgr.x.smithi190.stderr:2024-05-09T06:37:10.102+0000 7faa89227640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-09T06:37:10.082 INFO:tasks.ceph.mgr.x.smithi190.stderr:2024-05-09T06:37:10.102+0000 7faa89227640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-09T06:37:10.082 INFO:tasks.ceph.mgr.x.smithi190.stderr:2024-05-09T06:37:10.102+0000 7faa89227640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-09T06:37:10.082 INFO:tasks.ceph.mgr.x.smithi190.stderr:2024-05-09T06:37:10.102+0000 7faa89227640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-09T06:37:10.093 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvol43941 subvol_grp46030 2024-05-09T06:37:10.099 INFO:tasks.ceph.mgr.x.smithi190.stderr:2024-05-09T06:37:10.119+0000 7faa88a26640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-09T06:37:10.099 INFO:tasks.ceph.mgr.x.smithi190.stderr:2024-05-09T06:37:10.119+0000 7faa88a26640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-09T06:37:10.099 INFO:tasks.ceph.mgr.x.smithi190.stderr:2024-05-09T06:37:10.119+0000 7faa88a26640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-09T06:37:10.099 INFO:tasks.ceph.mgr.x.smithi190.stderr:2024-05-09T06:37:10.119+0000 7faa88a26640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-09T06:37:10.099 INFO:tasks.ceph.mgr.x.smithi190.stderr:2024-05-09T06:37:10.119+0000 7faa88a26640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-09T06:37:10.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:10.222+0000 7f6cb9de7640 1 Processor -- start 2024-05-09T06:37:10.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:10.222+0000 7f6cb9de7640 1 -- start start 2024-05-09T06:37:10.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:10.223+0000 7f6cb9de7640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6cb4141180 0x7f6cb406b250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:10.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:10.223+0000 7f6cb9de7640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6cb40cd540 con 0x7f6cb406b790 2024-05-09T06:37:10.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:10.223+0000 7f6cb9de7640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6cb40cd820 con 0x7f6cb40d0110 2024-05-09T06:37:10.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:10.223+0000 7f6cb9de7640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f6cb40cf200 con 0x7f6cb4141180 2024-05-09T06:37:10.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:10.223+0000 7f6cb2ffd640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f6cb406b790 0x7f6cb40cfa00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:58114/0 (socket says 172.21.15.3:58114) 2024-05-09T06:37:10.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:10.223+0000 7f6cb2ffd640 1 -- 172.21.15.3:0/1516628738 learned_addr learned my addr 172.21.15.3:0/1516628738 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-05-09T06:37:10.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:10.223+0000 7f6cb37fe640 1 --2- 172.21.15.3:0/1516628738 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6cb4141180 0x7f6cb406b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:10.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:10.224+0000 7f6cb27fc640 1 -- 172.21.15.3:0/1516628738 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2590744964 0 0) 0x7f6cb40cd540 con 0x7f6cb406b790 2024-05-09T06:37:10.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:10.224+0000 7f6cb27fc640 1 -- 172.21.15.3:0/1516628738 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6c9c003520 con 0x7f6cb406b790 2024-05-09T06:37:10.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:10.224+0000 7f6cb27fc640 1 -- 172.21.15.3:0/1516628738 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1467634608 0 0) 0x7f6cb40cd820 con 0x7f6cb40d0110 2024-05-09T06:37:10.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:10.224+0000 7f6cb27fc640 1 -- 172.21.15.3:0/1516628738 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6cb40cd540 con 0x7f6cb40d0110 2024-05-09T06:37:10.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:10.224+0000 7f6cb37fe640 1 -- 172.21.15.3:0/1516628738 >> v1:172.21.15.190:6789/0 conn(0x7f6cb40d0110 legacy=0x7f6cb40cc810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:10.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:10.224+0000 7f6cb37fe640 1 -- 172.21.15.3:0/1516628738 >> v1:172.21.15.3:6789/0 conn(0x7f6cb406b790 legacy=0x7f6cb40cfa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:10.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:10.224+0000 7f6cb27fc640 1 -- 172.21.15.3:0/1516628738 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3201657079 0 0) 0x7f6c9c003520 con 0x7f6cb406b790 2024-05-09T06:37:10.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:10.224+0000 7f6cb37fe640 1 -- 172.21.15.3:0/1516628738 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cb40cf4e0 con 0x7f6cb4141180 2024-05-09T06:37:10.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:10.225+0000 7f6cb37fe640 1 --2- 172.21.15.3:0/1516628738 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6cb4141180 0x7f6cb406b250 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f6ca40097d0 tx=0x7f6ca402a1a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fa26e60b36fa348e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:10.207 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:10.225+0000 7f6cb27fc640 1 -- 172.21.15.3:0/1516628738 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ca4037070 con 0x7f6cb4141180 2024-05-09T06:37:10.207 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:10.225+0000 7f6cb27fc640 1 -- 172.21.15.3:0/1516628738 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ca402ac30 con 0x7f6cb4141180 2024-05-09T06:37:10.207 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:10.225+0000 7f6cb27fc640 1 -- 172.21.15.3:0/1516628738 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ca8001670 con 0x7f6cb4141180 2024-05-09T06:37:10.207 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:10.225+0000 7f6cb9de7640 1 -- 172.21.15.3:0/1516628738 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6cb4141180 msgr2=0x7f6cb406b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:10.207 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:10.225+0000 7f6cb9de7640 1 --2- 172.21.15.3:0/1516628738 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6cb4141180 0x7f6cb406b250 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f6ca40097d0 tx=0x7f6ca402a1a0 comp rx=0 tx=0).stop 2024-05-09T06:37:10.207 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:10.225+0000 7f6cb9de7640 1 -- 172.21.15.3:0/1516628738 shutdown_connections 2024-05-09T06:37:10.208 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:10.225+0000 7f6cb9de7640 1 --2- 172.21.15.3:0/1516628738 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6cb4141180 0x7f6cb406b250 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:10.208 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:10.225+0000 7f6cb9de7640 1 -- 172.21.15.3:0/1516628738 >> 172.21.15.3:0/1516628738 conn(0x7f6cb40d05c0 msgr2=0x7f6cb40d09d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:10.208 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:10.225+0000 7f6cb9de7640 1 -- 172.21.15.3:0/1516628738 shutdown_connections 2024-05-09T06:37:10.208 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:10.225+0000 7f6cb9de7640 1 -- 172.21.15.3:0/1516628738 wait complete. 2024-05-09T06:37:10.208 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:10.227+0000 7f6cb9de7640 1 Processor -- start 2024-05-09T06:37:10.208 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:10.227+0000 7f6cb9de7640 1 -- start start 2024-05-09T06:37:10.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:10.227+0000 7f6cb9de7640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6cb406b790 0x7f6cb41f2380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:10.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:10.227+0000 7f6cb9de7640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6cb40d0110 0x7f6cb413b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:10.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:10.228+0000 7f6cb9de7640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6cb4141180 0x7f6cb4159a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:10.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:10.228+0000 7f6cb9de7640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f6cb40cd0a0 con 0x7f6cb406b790 2024-05-09T06:37:10.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:10.228+0000 7f6cb9de7640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f6cb40ccf20 con 0x7f6cb40d0110 2024-05-09T06:37:10.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:10.228+0000 7f6cb9de7640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f6cb40cd220 con 0x7f6cb4141180 2024-05-09T06:37:10.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:10.228+0000 7f6cb37fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6cb406b790 0x7f6cb41f2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:10.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:10.228+0000 7f6cb37fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6cb406b790 0x7f6cb41f2380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:39672/0 (socket says 172.21.15.3:39672) 2024-05-09T06:37:10.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:10.228+0000 7f6cb37fe640 1 -- 172.21.15.3:0/1562425713 learned_addr learned my addr 172.21.15.3:0/1562425713 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:37:10.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:10.228+0000 7f6cb2ffd640 1 --2- 172.21.15.3:0/1562425713 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6cb40d0110 0x7f6cb413b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:10.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:10.228+0000 7f6cb3fff640 1 --2- 172.21.15.3:0/1562425713 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6cb4141180 0x7f6cb4159a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:10.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:10.228+0000 7f6cb37fe640 1 -- 172.21.15.3:0/1562425713 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6cb4141180 msgr2=0x7f6cb4159a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:10.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:10.228+0000 7f6cb37fe640 1 --2- 172.21.15.3:0/1562425713 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6cb4141180 0x7f6cb4159a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:10.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:10.228+0000 7f6cb37fe640 1 -- 172.21.15.3:0/1562425713 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6cb40d0110 msgr2=0x7f6cb413b6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:10.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:10.228+0000 7f6cb37fe640 1 --2- 172.21.15.3:0/1562425713 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6cb40d0110 0x7f6cb413b6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:10.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:10.228+0000 7f6cb37fe640 1 -- 172.21.15.3:0/1562425713 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cb41f2ca0 con 0x7f6cb406b790 2024-05-09T06:37:10.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:10.228+0000 7f6cb37fe640 1 --2- 172.21.15.3:0/1562425713 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6cb406b790 0x7f6cb41f2380 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f6ca4035040 tx=0x7f6ca40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:10.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:10.228+0000 7f6c9bfff640 1 -- 172.21.15.3:0/1562425713 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ca4037050 con 0x7f6cb406b790 2024-05-09T06:37:10.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:10.228+0000 7f6c9bfff640 1 -- 172.21.15.3:0/1562425713 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ca4043070 con 0x7f6cb406b790 2024-05-09T06:37:10.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:10.228+0000 7f6cb9de7640 1 -- 172.21.15.3:0/1562425713 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6cb41589a0 con 0x7f6cb406b790 2024-05-09T06:37:10.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:10.228+0000 7f6c9bfff640 1 -- 172.21.15.3:0/1562425713 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ca4006e10 con 0x7f6cb406b790 2024-05-09T06:37:10.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:10.229+0000 7f6cb9de7640 1 -- 172.21.15.3:0/1562425713 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6cb4159240 con 0x7f6cb406b790 2024-05-09T06:37:10.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:10.229+0000 7f6cb9de7640 1 -- 172.21.15.3:0/1562425713 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6cb4069550 con 0x7f6cb406b790 2024-05-09T06:37:10.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:10.230+0000 7f6c9bfff640 1 -- 172.21.15.3:0/1562425713 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 6) ==== 90549+0+0 (secure 0 0 0) 0x7f6ca403ca30 con 0x7f6cb406b790 2024-05-09T06:37:10.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:10.230+0000 7f6c9bfff640 1 --2- 172.21.15.3:0/1562425713 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f6ca006bec0 0x7f6ca006e380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:10.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:10.230+0000 7f6c9bfff640 1 -- 172.21.15.3:0/1562425713 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f6ca40ab7f0 con 0x7f6cb406b790 2024-05-09T06:37:10.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:10.231+0000 7f6cb2ffd640 1 --2- 172.21.15.3:0/1562425713 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f6ca006bec0 0x7f6ca006e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:10.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:10.231+0000 7f6cb2ffd640 1 --2- 172.21.15.3:0/1562425713 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f6ca006bec0 0x7f6ca006e380 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f6cb415bd60 tx=0x7f6ca8001650 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:10.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:10.235+0000 7f6c9bfff640 1 -- 172.21.15.3:0/1562425713 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f6ca4079bf0 con 0x7f6cb406b790 2024-05-09T06:37:10.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:10.446+0000 7f6cb9de7640 1 -- 172.21.15.3:0/1562425713 --> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvol43941", "group_name": "subvol_grp46030", "target": ["mon-mgr", ""]}) -- 0x7f6cb406d9b0 con 0x7f6ca006bec0 2024-05-09T06:37:10.452 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:10.471+0000 7f6c9bfff640 1 -- 172.21.15.3:0/1562425713 <== mgr.4105 v2:172.21.15.190:6832/975893321 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f6cb406d9b0 con 0x7f6ca006bec0 2024-05-09T06:37:10.455 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:10.474+0000 7f6cb9de7640 1 -- 172.21.15.3:0/1562425713 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f6ca006bec0 msgr2=0x7f6ca006e380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:10.455 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:10.474+0000 7f6cb9de7640 1 --2- 172.21.15.3:0/1562425713 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f6ca006bec0 0x7f6ca006e380 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f6cb415bd60 tx=0x7f6ca8001650 comp rx=0 tx=0).stop 2024-05-09T06:37:10.455 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:10.474+0000 7f6cb9de7640 1 -- 172.21.15.3:0/1562425713 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6cb406b790 msgr2=0x7f6cb41f2380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:10.455 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:10.474+0000 7f6cb9de7640 1 --2- 172.21.15.3:0/1562425713 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6cb406b790 0x7f6cb41f2380 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f6ca4035040 tx=0x7f6ca40046c0 comp rx=0 tx=0).stop 2024-05-09T06:37:10.456 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:10.474+0000 7f6cb9de7640 1 -- 172.21.15.3:0/1562425713 shutdown_connections 2024-05-09T06:37:10.456 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:10.474+0000 7f6cb9de7640 1 --2- 172.21.15.3:0/1562425713 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f6ca006bec0 0x7f6ca006e380 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:10.456 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:10.474+0000 7f6cb9de7640 1 --2- 172.21.15.3:0/1562425713 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6cb4141180 0x7f6cb4159a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:10.456 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:10.474+0000 7f6cb9de7640 1 --2- 172.21.15.3:0/1562425713 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6cb40d0110 0x7f6cb413b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:10.456 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:10.474+0000 7f6cb9de7640 1 --2- 172.21.15.3:0/1562425713 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6cb406b790 0x7f6cb41f2380 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:10.456 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:10.474+0000 7f6cb9de7640 1 -- 172.21.15.3:0/1562425713 >> 172.21.15.3:0/1562425713 conn(0x7f6cb40d05c0 msgr2=0x7f6cb406f370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:10.456 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:10.474+0000 7f6cb9de7640 1 -- 172.21.15.3:0/1562425713 shutdown_connections 2024-05-09T06:37:10.456 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:10.474+0000 7f6cb9de7640 1 -- 172.21.15.3:0/1562425713 wait complete. 2024-05-09T06:37:10.467 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvol3942 subvol_grp46030 2024-05-09T06:37:10.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:10.594+0000 7f4f82c15640 1 Processor -- start 2024-05-09T06:37:10.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:10.595+0000 7f4f82c15640 1 -- start start 2024-05-09T06:37:10.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:10.595+0000 7f4f82c15640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4f7c0ae0b0 0x7f4f7c15f280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:10.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:10.595+0000 7f4f82c15640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f4f7c0b56f0 0x7f4f7c0ae490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:10.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:10.595+0000 7f4f82c15640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4f7c15f7c0 con 0x7f4f7c06b250 2024-05-09T06:37:10.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:10.595+0000 7f4f82c15640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f4f7c06cc80 con 0x7f4f7c0b56f0 2024-05-09T06:37:10.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:10.595+0000 7f4f82c15640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f4f7c0b5260 con 0x7f4f7c0ae0b0 2024-05-09T06:37:10.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:10.596+0000 7f4f7bfff640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f4f7c0b56f0 0x7f4f7c0ae490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:10.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:10.596+0000 7f4f811b2640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f4f7c06b250 0x7f4f7c06b6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:45884/0 (socket says 172.21.15.3:45884) 2024-05-09T06:37:10.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:10.596+0000 7f4f811b2640 1 -- 172.21.15.3:0/1752007986 learned_addr learned my addr 172.21.15.3:0/1752007986 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-05-09T06:37:10.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:10.596+0000 7f4f809b1640 1 --2- 172.21.15.3:0/1752007986 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4f7c0ae0b0 0x7f4f7c15f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:10.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:10.596+0000 7f4f7bfff640 1 -- 172.21.15.3:0/1752007986 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4f7c0ae0b0 msgr2=0x7f4f7c15f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:37:10.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:10.596+0000 7f4f7bfff640 1 --2- 172.21.15.3:0/1752007986 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4f7c0ae0b0 0x7f4f7c15f280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:10.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:10.596+0000 7f4f7bfff640 1 -- 172.21.15.3:0/1752007986 >> v1:172.21.15.3:6789/0 conn(0x7f4f7c06b250 legacy=0x7f4f7c06b6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:10.578 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:10.596+0000 7f4f7bfff640 1 -- 172.21.15.3:0/1752007986 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f7c0adb40 con 0x7f4f7c0b56f0 2024-05-09T06:37:10.578 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:10.596+0000 7f4f7b7fe640 1 -- 172.21.15.3:0/1752007986 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1364516745 0 0) 0x7f4f64001110 con 0x7f4f7c06b250 2024-05-09T06:37:10.579 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:10.597+0000 7f4f7bfff640 1 --2- 172.21.15.3:0/1752007986 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4f7c0b56f0 0x7f4f7c0ae490 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f4f6c009d90 tx=0x7f4f6c02a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0876ebdd4d505c8 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:10.579 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:10.597+0000 7f4f7b7fe640 1 -- 172.21.15.3:0/1752007986 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f6c031600 con 0x7f4f7c0b56f0 2024-05-09T06:37:10.579 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:10.597+0000 7f4f7b7fe640 1 -- 172.21.15.3:0/1752007986 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f6c0317a0 con 0x7f4f7c0b56f0 2024-05-09T06:37:10.579 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:10.597+0000 7f4f7b7fe640 1 -- 172.21.15.3:0/1752007986 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f6c031b70 con 0x7f4f7c0b56f0 2024-05-09T06:37:10.579 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:10.597+0000 7f4f82c15640 1 -- 172.21.15.3:0/1752007986 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4f7c0b56f0 msgr2=0x7f4f7c0ae490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:10.580 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:10.597+0000 7f4f82c15640 1 --2- 172.21.15.3:0/1752007986 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4f7c0b56f0 0x7f4f7c0ae490 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f4f6c009d90 tx=0x7f4f6c02a5d0 comp rx=0 tx=0).stop 2024-05-09T06:37:10.580 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:10.597+0000 7f4f82c15640 1 -- 172.21.15.3:0/1752007986 shutdown_connections 2024-05-09T06:37:10.580 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:10.597+0000 7f4f82c15640 1 --2- 172.21.15.3:0/1752007986 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4f7c0b56f0 0x7f4f7c0ae490 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:10.580 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:10.597+0000 7f4f82c15640 1 --2- 172.21.15.3:0/1752007986 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4f7c0ae0b0 0x7f4f7c15f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:10.580 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:10.597+0000 7f4f82c15640 1 -- 172.21.15.3:0/1752007986 >> 172.21.15.3:0/1752007986 conn(0x7f4f7c0d05c0 msgr2=0x7f4f7c0d09d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:10.580 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:10.597+0000 7f4f82c15640 1 -- 172.21.15.3:0/1752007986 shutdown_connections 2024-05-09T06:37:10.580 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:10.597+0000 7f4f82c15640 1 -- 172.21.15.3:0/1752007986 wait complete. 2024-05-09T06:37:10.580 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:10.599+0000 7f4f82c15640 1 Processor -- start 2024-05-09T06:37:10.580 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:10.599+0000 7f4f82c15640 1 -- start start 2024-05-09T06:37:10.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:10.599+0000 7f4f82c15640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4f7c06b250 0x7f4f7c15b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:10.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:10.599+0000 7f4f82c15640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4f7c0ae0b0 0x7f4f7c149ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:10.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:10.599+0000 7f4f82c15640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4f7c0b56f0 0x7f4f7c156440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:10.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:10.599+0000 7f4f82c15640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f4f7c0b4930 con 0x7f4f7c06b250 2024-05-09T06:37:10.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:10.599+0000 7f4f82c15640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f4f7c0b4650 con 0x7f4f7c0ae0b0 2024-05-09T06:37:10.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:10.599+0000 7f4f82c15640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f4f7c0b4c10 con 0x7f4f7c0b56f0 2024-05-09T06:37:10.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:10.600+0000 7f4f809b1640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4f7c06b250 0x7f4f7c15b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:10.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:10.600+0000 7f4f809b1640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4f7c06b250 0x7f4f7c15b4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50832/0 (socket says 172.21.15.3:50832) 2024-05-09T06:37:10.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:10.600+0000 7f4f809b1640 1 -- 172.21.15.3:0/2563554400 learned_addr learned my addr 172.21.15.3:0/2563554400 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:37:10.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:10.600+0000 7f4f7bfff640 1 --2- 172.21.15.3:0/2563554400 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4f7c0ae0b0 0x7f4f7c149ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:10.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:10.600+0000 7f4f811b2640 1 --2- 172.21.15.3:0/2563554400 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4f7c0b56f0 0x7f4f7c156440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:10.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:10.600+0000 7f4f809b1640 1 -- 172.21.15.3:0/2563554400 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4f7c0b56f0 msgr2=0x7f4f7c156440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:10.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:10.600+0000 7f4f809b1640 1 --2- 172.21.15.3:0/2563554400 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4f7c0b56f0 0x7f4f7c156440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:10.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:10.600+0000 7f4f809b1640 1 -- 172.21.15.3:0/2563554400 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4f7c0ae0b0 msgr2=0x7f4f7c149ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:10.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:10.600+0000 7f4f809b1640 1 --2- 172.21.15.3:0/2563554400 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4f7c0ae0b0 0x7f4f7c149ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:10.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:10.600+0000 7f4f809b1640 1 -- 172.21.15.3:0/2563554400 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f7c1553a0 con 0x7f4f7c06b250 2024-05-09T06:37:10.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:10.600+0000 7f4f7bfff640 1 --2- 172.21.15.3:0/2563554400 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4f7c0ae0b0 0x7f4f7c149ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:37:10.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:10.600+0000 7f4f809b1640 1 --2- 172.21.15.3:0/2563554400 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4f7c06b250 0x7f4f7c15b4b0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f4f7400d5f0 tx=0x7f4f7400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:10.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:10.600+0000 7f4f797fa640 1 -- 172.21.15.3:0/2563554400 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f740125f0 con 0x7f4f7c06b250 2024-05-09T06:37:10.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:10.600+0000 7f4f797fa640 1 -- 172.21.15.3:0/2563554400 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f74019070 con 0x7f4f7c06b250 2024-05-09T06:37:10.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:10.600+0000 7f4f82c15640 1 -- 172.21.15.3:0/2563554400 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f7c155680 con 0x7f4f7c06b250 2024-05-09T06:37:10.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:10.600+0000 7f4f797fa640 1 -- 172.21.15.3:0/2563554400 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f740129d0 con 0x7f4f7c06b250 2024-05-09T06:37:10.584 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:10.600+0000 7f4f82c15640 1 -- 172.21.15.3:0/2563554400 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f7c155f20 con 0x7f4f7c06b250 2024-05-09T06:37:10.584 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:10.601+0000 7f4f82c15640 1 -- 172.21.15.3:0/2563554400 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f7c155a70 con 0x7f4f7c06b250 2024-05-09T06:37:10.584 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:10.602+0000 7f4f797fa640 1 -- 172.21.15.3:0/2563554400 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 6) ==== 90549+0+0 (secure 0 0 0) 0x7f4f74012b70 con 0x7f4f7c06b250 2024-05-09T06:37:10.584 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:10.602+0000 7f4f797fa640 1 --2- 172.21.15.3:0/2563554400 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f4f5c06bec0 0x7f4f5c06e380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:10.584 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:10.602+0000 7f4f797fa640 1 -- 172.21.15.3:0/2563554400 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f4f7408b390 con 0x7f4f7c06b250 2024-05-09T06:37:10.585 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:10.603+0000 7f4f7bfff640 1 --2- 172.21.15.3:0/2563554400 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f4f5c06bec0 0x7f4f5c06e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:10.585 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:10.603+0000 7f4f7bfff640 1 --2- 172.21.15.3:0/2563554400 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f4f5c06bec0 0x7f4f5c06e380 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f4f6c034000 tx=0x7f4f6c02f000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:10.587 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:10.606+0000 7f4f797fa640 1 -- 172.21.15.3:0/2563554400 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f4f74059c50 con 0x7f4f7c06b250 2024-05-09T06:37:10.796 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:10.816+0000 7f4f82c15640 1 -- 172.21.15.3:0/2563554400 --> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvol3942", "group_name": "subvol_grp46030", "target": ["mon-mgr", ""]}) -- 0x7f4f7c0682e0 con 0x7f4f5c06bec0 2024-05-09T06:37:10.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:10.833+0000 7f4f797fa640 1 -- 172.21.15.3:0/2563554400 <== mgr.4105 v2:172.21.15.190:6832/975893321 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f4f7c0682e0 con 0x7f4f5c06bec0 2024-05-09T06:37:10.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:10.835+0000 7f4f82c15640 1 -- 172.21.15.3:0/2563554400 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f4f5c06bec0 msgr2=0x7f4f5c06e380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:10.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:10.835+0000 7f4f82c15640 1 --2- 172.21.15.3:0/2563554400 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f4f5c06bec0 0x7f4f5c06e380 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f4f6c034000 tx=0x7f4f6c02f000 comp rx=0 tx=0).stop 2024-05-09T06:37:10.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:10.835+0000 7f4f82c15640 1 -- 172.21.15.3:0/2563554400 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4f7c06b250 msgr2=0x7f4f7c15b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:10.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:10.835+0000 7f4f82c15640 1 --2- 172.21.15.3:0/2563554400 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4f7c06b250 0x7f4f7c15b4b0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f4f7400d5f0 tx=0x7f4f7400dab0 comp rx=0 tx=0).stop 2024-05-09T06:37:10.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:10.836+0000 7f4f82c15640 1 -- 172.21.15.3:0/2563554400 shutdown_connections 2024-05-09T06:37:10.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:10.836+0000 7f4f82c15640 1 --2- 172.21.15.3:0/2563554400 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f4f5c06bec0 0x7f4f5c06e380 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:10.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:10.836+0000 7f4f82c15640 1 --2- 172.21.15.3:0/2563554400 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4f7c0b56f0 0x7f4f7c156440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:10.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:10.836+0000 7f4f82c15640 1 --2- 172.21.15.3:0/2563554400 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4f7c0ae0b0 0x7f4f7c149ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:10.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:10.836+0000 7f4f82c15640 1 --2- 172.21.15.3:0/2563554400 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4f7c06b250 0x7f4f7c15b4b0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:10.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:10.836+0000 7f4f82c15640 1 -- 172.21.15.3:0/2563554400 >> 172.21.15.3:0/2563554400 conn(0x7f4f7c0d05c0 msgr2=0x7f4f7c0b0d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:10.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:10.836+0000 7f4f82c15640 1 -- 172.21.15.3:0/2563554400 shutdown_connections 2024-05-09T06:37:10.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:10.836+0000 7f4f82c15640 1 -- 172.21.15.3:0/2563554400 wait complete. 2024-05-09T06:37:10.830 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup rm cephfs subvol_grp46030 2024-05-09T06:37:10.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:10.956+0000 7fa698589640 1 Processor -- start 2024-05-09T06:37:10.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:10.956+0000 7fa698589640 1 -- start start 2024-05-09T06:37:10.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:10.957+0000 7fa698589640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa69015c0f0 0x7fa690160730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:10.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:10.957+0000 7fa698589640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fa6900b4e30 0x7fa69006e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:10.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:10.957+0000 7fa698589640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa690160c70 con 0x7fa69006e790 2024-05-09T06:37:10.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:10.957+0000 7fa698589640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fa69006ebc0 con 0x7fa69015c0f0 2024-05-09T06:37:10.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:10.957+0000 7fa698589640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7fa6900b40f0 con 0x7fa6900b4e30 2024-05-09T06:37:10.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:10.957+0000 7fa696325640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa69015c0f0 0x7fa690160730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:10.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:10.957+0000 7fa696325640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa69015c0f0 0x7fa690160730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50840/0 (socket says 172.21.15.3:50840) 2024-05-09T06:37:10.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:10.957+0000 7fa696325640 1 -- 172.21.15.3:0/4136626119 learned_addr learned my addr 172.21.15.3:0/4136626119 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:37:10.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:10.958+0000 7fa696325640 1 -- 172.21.15.3:0/4136626119 >> v2:172.21.15.190:3300/0 conn(0x7fa6900b4e30 msgr2=0x7fa69006e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:37:10.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:10.958+0000 7fa695b24640 1 --2- 172.21.15.3:0/4136626119 >> v2:172.21.15.190:3300/0 conn(0x7fa6900b4e30 0x7fa69006e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:10.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:10.958+0000 7fa696325640 1 --2- 172.21.15.3:0/4136626119 >> v2:172.21.15.190:3300/0 conn(0x7fa6900b4e30 0x7fa69006e250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:10.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:10.958+0000 7fa696325640 1 -- 172.21.15.3:0/4136626119 >> v1:172.21.15.190:6789/0 conn(0x7fa69006e790 legacy=0x7fa6900cd5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:37:10.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:10.958+0000 7fa696325640 1 -- 172.21.15.3:0/4136626119 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6900b3050 con 0x7fa69015c0f0 2024-05-09T06:37:10.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:10.959+0000 7fa696325640 1 --2- 172.21.15.3:0/4136626119 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa69015c0f0 0x7fa690160730 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fa690073500 tx=0x7fa68802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1ff3b1b6f09a0c1 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:10.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:10.959+0000 7fa695323640 1 -- 172.21.15.3:0/4136626119 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa688037070 con 0x7fa69015c0f0 2024-05-09T06:37:10.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:10.959+0000 7fa695323640 1 -- 172.21.15.3:0/4136626119 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa688032440 con 0x7fa69015c0f0 2024-05-09T06:37:10.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:10.959+0000 7fa695323640 1 -- 172.21.15.3:0/4136626119 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa688032810 con 0x7fa69015c0f0 2024-05-09T06:37:10.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:10.959+0000 7fa698589640 1 -- 172.21.15.3:0/4136626119 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa69015c0f0 msgr2=0x7fa690160730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:10.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:10.959+0000 7fa698589640 1 --2- 172.21.15.3:0/4136626119 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa69015c0f0 0x7fa690160730 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fa690073500 tx=0x7fa68802a1a0 comp rx=0 tx=0).stop 2024-05-09T06:37:10.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:10.959+0000 7fa698589640 1 -- 172.21.15.3:0/4136626119 shutdown_connections 2024-05-09T06:37:10.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:10.959+0000 7fa698589640 1 --2- 172.21.15.3:0/4136626119 >> v2:172.21.15.190:3300/0 conn(0x7fa6900b4e30 0x7fa69006e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:10.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:10.959+0000 7fa698589640 1 --2- 172.21.15.3:0/4136626119 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa69015c0f0 0x7fa690160730 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:10.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:10.959+0000 7fa698589640 1 -- 172.21.15.3:0/4136626119 >> 172.21.15.3:0/4136626119 conn(0x7fa6900b1c20 msgr2=0x7fa6900b2030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:10.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:10.959+0000 7fa698589640 1 -- 172.21.15.3:0/4136626119 shutdown_connections 2024-05-09T06:37:10.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:10.959+0000 7fa698589640 1 -- 172.21.15.3:0/4136626119 wait complete. 2024-05-09T06:37:10.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:10.961+0000 7fa698589640 1 Processor -- start 2024-05-09T06:37:10.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:10.961+0000 7fa698589640 1 -- start start 2024-05-09T06:37:10.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:10.961+0000 7fa698589640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa69006e790 0x7fa69015e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:10.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:10.961+0000 7fa698589640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa6900b4e30 0x7fa69015ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:10.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:10.961+0000 7fa698589640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa69015c0f0 0x7fa690148540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:10.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:10.961+0000 7fa698589640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fa6900b38f0 con 0x7fa69015c0f0 2024-05-09T06:37:10.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:10.961+0000 7fa698589640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fa6900b3610 con 0x7fa69006e790 2024-05-09T06:37:10.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:10.961+0000 7fa698589640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fa6900b3bd0 con 0x7fa6900b4e30 2024-05-09T06:37:10.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:10.962+0000 7fa696b26640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa69015c0f0 0x7fa690148540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:10.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:10.962+0000 7fa696b26640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa69015c0f0 0x7fa690148540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50846/0 (socket says 172.21.15.3:50846) 2024-05-09T06:37:10.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:10.962+0000 7fa696b26640 1 -- 172.21.15.3:0/1345696621 learned_addr learned my addr 172.21.15.3:0/1345696621 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:37:10.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:10.962+0000 7fa696325640 1 --2- 172.21.15.3:0/1345696621 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa69006e790 0x7fa69015e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:10.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:10.962+0000 7fa695b24640 1 --2- 172.21.15.3:0/1345696621 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa6900b4e30 0x7fa69015ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:10.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:10.962+0000 7fa696b26640 1 -- 172.21.15.3:0/1345696621 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa6900b4e30 msgr2=0x7fa69015ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:10.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:10.962+0000 7fa696b26640 1 --2- 172.21.15.3:0/1345696621 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa6900b4e30 0x7fa69015ebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:10.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:10.962+0000 7fa696b26640 1 -- 172.21.15.3:0/1345696621 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa69006e790 msgr2=0x7fa69015e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:10.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:10.962+0000 7fa696b26640 1 --2- 172.21.15.3:0/1345696621 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa69006e790 0x7fa69015e6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:10.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:10.962+0000 7fa696b26640 1 -- 172.21.15.3:0/1345696621 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6901492c0 con 0x7fa69015c0f0 2024-05-09T06:37:10.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:10.962+0000 7fa696b26640 1 --2- 172.21.15.3:0/1345696621 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa69015c0f0 0x7fa690148540 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fa68000c780 tx=0x7fa68000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:10.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:10.962+0000 7fa68effd640 1 -- 172.21.15.3:0/1345696621 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa680017070 con 0x7fa69015c0f0 2024-05-09T06:37:10.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:10.962+0000 7fa68effd640 1 -- 172.21.15.3:0/1345696621 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa680003da0 con 0x7fa69015c0f0 2024-05-09T06:37:10.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:10.962+0000 7fa68effd640 1 -- 172.21.15.3:0/1345696621 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6800052b0 con 0x7fa69015c0f0 2024-05-09T06:37:10.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:10.962+0000 7fa698589640 1 -- 172.21.15.3:0/1345696621 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6901495a0 con 0x7fa69015c0f0 2024-05-09T06:37:10.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:10.962+0000 7fa698589640 1 -- 172.21.15.3:0/1345696621 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa690149b60 con 0x7fa69015c0f0 2024-05-09T06:37:10.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:10.963+0000 7fa698589640 1 -- 172.21.15.3:0/1345696621 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa690069550 con 0x7fa69015c0f0 2024-05-09T06:37:10.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:10.964+0000 7fa68effd640 1 -- 172.21.15.3:0/1345696621 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 6) ==== 90549+0+0 (secure 0 0 0) 0x7fa68001a070 con 0x7fa69015c0f0 2024-05-09T06:37:10.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:10.964+0000 7fa68effd640 1 --2- 172.21.15.3:0/1345696621 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fa67c06bec0 0x7fa67c06e380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:10.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:10.964+0000 7fa68effd640 1 -- 172.21.15.3:0/1345696621 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fa68008a550 con 0x7fa69015c0f0 2024-05-09T06:37:10.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:10.965+0000 7fa696325640 1 --2- 172.21.15.3:0/1345696621 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fa67c06bec0 0x7fa67c06e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:10.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:10.965+0000 7fa696325640 1 --2- 172.21.15.3:0/1345696621 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fa67c06bec0 0x7fa67c06e380 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fa688034000 tx=0x7fa68802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:10.949 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:10.969+0000 7fa68effd640 1 -- 172.21.15.3:0/1345696621 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fa6800587c0 con 0x7fa69015c0f0 2024-05-09T06:37:11.158 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:11.177+0000 7fa698589640 1 -- 172.21.15.3:0/1345696621 --> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup rm", "vol_name": "cephfs", "group_name": "subvol_grp46030", "target": ["mon-mgr", ""]}) -- 0x7fa69015b410 con 0x7fa67c06bec0 2024-05-09T06:37:11.163 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:11.183+0000 7fa68effd640 1 -- 172.21.15.3:0/1345696621 <== mgr.4105 v2:172.21.15.190:6832/975893321 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fa69015b410 con 0x7fa67c06bec0 2024-05-09T06:37:11.166 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:11.185+0000 7fa698589640 1 -- 172.21.15.3:0/1345696621 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fa67c06bec0 msgr2=0x7fa67c06e380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:11.166 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:11.185+0000 7fa698589640 1 --2- 172.21.15.3:0/1345696621 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fa67c06bec0 0x7fa67c06e380 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fa688034000 tx=0x7fa68802e000 comp rx=0 tx=0).stop 2024-05-09T06:37:11.167 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:11.185+0000 7fa698589640 1 -- 172.21.15.3:0/1345696621 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa69015c0f0 msgr2=0x7fa690148540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:11.167 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:11.185+0000 7fa698589640 1 --2- 172.21.15.3:0/1345696621 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa69015c0f0 0x7fa690148540 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fa68000c780 tx=0x7fa68000cc40 comp rx=0 tx=0).stop 2024-05-09T06:37:11.167 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:11.185+0000 7fa698589640 1 -- 172.21.15.3:0/1345696621 shutdown_connections 2024-05-09T06:37:11.167 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:11.185+0000 7fa698589640 1 --2- 172.21.15.3:0/1345696621 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fa67c06bec0 0x7fa67c06e380 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:11.167 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:11.185+0000 7fa698589640 1 --2- 172.21.15.3:0/1345696621 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa69015c0f0 0x7fa690148540 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:11.167 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:11.185+0000 7fa698589640 1 --2- 172.21.15.3:0/1345696621 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa6900b4e30 0x7fa69015ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:11.167 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:11.185+0000 7fa698589640 1 --2- 172.21.15.3:0/1345696621 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa69006e790 0x7fa69015e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:11.167 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:11.185+0000 7fa698589640 1 -- 172.21.15.3:0/1345696621 >> 172.21.15.3:0/1345696621 conn(0x7fa6900b1c20 msgr2=0x7fa690151160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:11.167 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:11.185+0000 7fa698589640 1 -- 172.21.15.3:0/1345696621 shutdown_connections 2024-05-09T06:37:11.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:11.185+0000 7fa698589640 1 -- 172.21.15.3:0/1345696621 wait complete. 2024-05-09T06:37:11.177 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2024-05-09T06:37:11.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:11 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-09T06:37:11.232 INFO:teuthology.orchestra.run.smithi003.stdout:2 2024-05-09T06:37:11.232 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T06:37:11.238 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-05-09T06:37:11.239 DEBUG:teuthology.orchestra.run.smithi003:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-09T06:37:11.279 DEBUG:teuthology.orchestra.run.smithi190:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-09T06:37:11.325 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-05-09T06:37:11.325 INFO:tasks.cephfs.mount:Terminating background process 2024-05-09T06:37:11.326 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-05-09T06:37:11.326 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T06:37:11.342 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-05-09T06:37:11.342 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T06:37:11.403 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T06:37:11.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:11.568+0000 7f7879b5a640 1 Processor -- start 2024-05-09T06:37:11.550 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:11.569+0000 7f7879b5a640 1 -- start start 2024-05-09T06:37:11.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:11.569+0000 7f7879b5a640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f787415e480 0x7f78740d1dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:11.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:11.569+0000 7f7879b5a640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f78740d2ed0 0x7f78741683a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:11.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:11.569+0000 7f7879b5a640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f78740d4e50 con 0x7f78740d2310 2024-05-09T06:37:11.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:11.569+0000 7f7879b5a640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f78740d0270 con 0x7f787415e480 2024-05-09T06:37:11.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:11.569+0000 7f7879b5a640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f78740cf1d0 con 0x7f78740d2ed0 2024-05-09T06:37:11.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:11.570+0000 7f7873fff640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f78740d2ed0 0x7f78741683a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:11.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:11.570+0000 7f78737fe640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f787415e480 0x7f78740d1dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:11.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:11.570+0000 7f7873fff640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f78740d2ed0 0x7f78741683a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.3:38712/0 (socket says 172.21.15.3:38712) 2024-05-09T06:37:11.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:11.570+0000 7f7873fff640 1 -- 172.21.15.3:0/3874845099 learned_addr learned my addr 172.21.15.3:0/3874845099 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:37:11.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:11.570+0000 7f7873fff640 1 -- 172.21.15.3:0/3874845099 >> v2:172.21.15.190:3300/0 conn(0x7f787415e480 msgr2=0x7f78740d1dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:37:11.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:11.570+0000 7f7873fff640 1 --2- 172.21.15.3:0/3874845099 >> v2:172.21.15.190:3300/0 conn(0x7f787415e480 0x7f78740d1dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:11.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:11.570+0000 7f7873fff640 1 -- 172.21.15.3:0/3874845099 >> v1:172.21.15.190:6789/0 conn(0x7f78740d2310 legacy=0x7f78740d27c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:11.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:11.570+0000 7f7873fff640 1 -- 172.21.15.3:0/3874845099 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78740cf4b0 con 0x7f78740d2ed0 2024-05-09T06:37:11.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:11.570+0000 7f78737fe640 1 --2- 172.21.15.3:0/3874845099 >> v2:172.21.15.190:3300/0 conn(0x7f787415e480 0x7f78740d1dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T06:37:11.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:11.571+0000 7f7873fff640 1 --2- 172.21.15.3:0/3874845099 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f78740d2ed0 0x7f78741683a0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f786c009e50 tx=0x7f786c02a710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d19f67752a082e54 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:11.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:11.571+0000 7f78727fc640 1 -- 172.21.15.3:0/3874845099 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f786c036070 con 0x7f78740d2ed0 2024-05-09T06:37:11.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:11.571+0000 7f78727fc640 1 -- 172.21.15.3:0/3874845099 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f786c031600 con 0x7f78740d2ed0 2024-05-09T06:37:11.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:11.571+0000 7f78727fc640 1 -- 172.21.15.3:0/3874845099 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f786c0319d0 con 0x7f78740d2ed0 2024-05-09T06:37:11.554 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:11.571+0000 7f7879b5a640 1 -- 172.21.15.3:0/3874845099 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f78740d2ed0 msgr2=0x7f78741683a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:11.554 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:11.571+0000 7f7879b5a640 1 --2- 172.21.15.3:0/3874845099 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f78740d2ed0 0x7f78741683a0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f786c009e50 tx=0x7f786c02a710 comp rx=0 tx=0).stop 2024-05-09T06:37:11.554 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:11.572+0000 7f7879b5a640 1 -- 172.21.15.3:0/3874845099 shutdown_connections 2024-05-09T06:37:11.554 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:11.572+0000 7f7879b5a640 1 --2- 172.21.15.3:0/3874845099 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f78740d2ed0 0x7f78741683a0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:11.554 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:11.572+0000 7f7879b5a640 1 --2- 172.21.15.3:0/3874845099 >> v2:172.21.15.190:3300/0 conn(0x7f787415e480 0x7f78740d1dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:11.554 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:11.572+0000 7f7879b5a640 1 -- 172.21.15.3:0/3874845099 >> 172.21.15.3:0/3874845099 conn(0x7f787409d4d0 msgr2=0x7f78740d4200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:11.554 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:11.572+0000 7f7879b5a640 1 -- 172.21.15.3:0/3874845099 shutdown_connections 2024-05-09T06:37:11.554 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:11.572+0000 7f7879b5a640 1 -- 172.21.15.3:0/3874845099 wait complete. 2024-05-09T06:37:11.557 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:11.573+0000 7f7879b5a640 1 Processor -- start 2024-05-09T06:37:11.557 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:11.573+0000 7f7879b5a640 1 -- start start 2024-05-09T06:37:11.557 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:11.574+0000 7f7879b5a640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f78740d2310 0x7f787415b110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:11.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:11.574+0000 7f7879b5a640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f78740d2ed0 0x7f787406cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:11.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:11.574+0000 7f7879b5a640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f787415e480 0x7f787406d320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:11.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:11.574+0000 7f7879b5a640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f78740cfd50 con 0x7f78740d2ed0 2024-05-09T06:37:11.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:11.574+0000 7f7879b5a640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f78740cfa70 con 0x7f78740d2310 2024-05-09T06:37:11.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:11.574+0000 7f7879b5a640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f78740b5ab0 con 0x7f787415e480 2024-05-09T06:37:11.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:11.574+0000 7f7872ffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f78740d2ed0 0x7f787406cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:11.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:11.574+0000 7f7872ffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f78740d2ed0 0x7f787406cde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50856/0 (socket says 172.21.15.3:50856) 2024-05-09T06:37:11.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:11.574+0000 7f7872ffd640 1 -- 172.21.15.3:0/139463360 learned_addr learned my addr 172.21.15.3:0/139463360 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:37:11.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:11.574+0000 7f78737fe640 1 --2- 172.21.15.3:0/139463360 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f78740d2310 0x7f787415b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:11.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:11.574+0000 7f7873fff640 1 --2- 172.21.15.3:0/139463360 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f787415e480 0x7f787406d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:11.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:11.574+0000 7f7872ffd640 1 -- 172.21.15.3:0/139463360 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f787415e480 msgr2=0x7f787406d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:11.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:11.574+0000 7f7872ffd640 1 --2- 172.21.15.3:0/139463360 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f787415e480 0x7f787406d320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:11.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:11.574+0000 7f7872ffd640 1 -- 172.21.15.3:0/139463360 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f78740d2310 msgr2=0x7f787415b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:11.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:11.574+0000 7f7872ffd640 1 --2- 172.21.15.3:0/139463360 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f78740d2310 0x7f787415b110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:11.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:11.574+0000 7f7872ffd640 1 -- 172.21.15.3:0/139463360 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78740b3710 con 0x7f78740d2ed0 2024-05-09T06:37:11.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:11.574+0000 7f7872ffd640 1 --2- 172.21.15.3:0/139463360 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f78740d2ed0 0x7f787406cde0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f786800c470 tx=0x7f786800c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:11.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:11.575+0000 7f785bfff640 1 -- 172.21.15.3:0/139463360 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78680052d0 con 0x7f78740d2ed0 2024-05-09T06:37:11.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:11.575+0000 7f785bfff640 1 -- 172.21.15.3:0/139463360 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7868005490 con 0x7f78740d2ed0 2024-05-09T06:37:11.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:11.575+0000 7f785bfff640 1 -- 172.21.15.3:0/139463360 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7868007260 con 0x7f78740d2ed0 2024-05-09T06:37:11.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:11.575+0000 7f7879b5a640 1 -- 172.21.15.3:0/139463360 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78740b93f0 con 0x7f78740d2ed0 2024-05-09T06:37:11.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:11.575+0000 7f7879b5a640 1 -- 172.21.15.3:0/139463360 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78740b9690 con 0x7f78740d2ed0 2024-05-09T06:37:11.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:11.576+0000 7f7879b5a640 1 -- 172.21.15.3:0/139463360 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f787404f660 con 0x7f78740d2ed0 2024-05-09T06:37:11.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:11.576+0000 7f785bfff640 1 -- 172.21.15.3:0/139463360 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 6) ==== 90549+0+0 (secure 0 0 0) 0x7f7868007400 con 0x7f78740d2ed0 2024-05-09T06:37:11.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:11.577+0000 7f785bfff640 1 --2- 172.21.15.3:0/139463360 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f786006bec0 0x7f786006e380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:11.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:11.577+0000 7f785bfff640 1 -- 172.21.15.3:0/139463360 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f786808a270 con 0x7f78740d2ed0 2024-05-09T06:37:11.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:11.577+0000 7f78737fe640 1 --2- 172.21.15.3:0/139463360 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f786006bec0 0x7f786006e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:11.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:11.577+0000 7f78737fe640 1 --2- 172.21.15.3:0/139463360 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f786006bec0 0x7f786006e380 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f7864009560 tx=0x7f7864009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:11.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:11.581+0000 7f785bfff640 1 -- 172.21.15.3:0/139463360 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f78740d4e50 con 0x7f78740d2ed0 2024-05-09T06:37:11.596 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:11.616+0000 7f785bfff640 1 -- 172.21.15.3:0/139463360 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mgrmap(e 7) ==== 90651+0+0 (secure 0 0 0) 0x7f7868054c50 con 0x7f78740d2ed0 2024-05-09T06:37:11.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:11.766+0000 7f7879b5a640 1 -- 172.21.15.3:0/139463360 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f78740d2b20 con 0x7f78740d2ed0 2024-05-09T06:37:11.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:11.768+0000 7f785bfff640 1 -- 172.21.15.3:0/139463360 <== mon.0 v2:172.21.15.3:3300/0 8 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) ==== 74+0+16585 (secure 0 0 0) 0x7f786805f040 con 0x7f78740d2ed0 2024-05-09T06:37:11.749 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-09T06:37:11.749 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":33,"fsid":"16ed737d-2665-431a-8ac6-ef242be3975a","created":"2024-05-09T06:35:42.541368+0000","modified":"2024-05-09T06:36:59.564366+0000","last_up_change":"2024-05-09T06:35:53.778274+0000","last_in_change":"2024-05-09T06:35:44.780586+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T06:35:54.819999+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-05-09T06:36:56.494700+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-05-09T06:36:56.900006+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fae3a8db-4826-4f3c-b33a-12c09bb5df6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":1897849827},{"type":"v1","addr":"172.21.15.3:6817","nonce":1897849827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":1897849827},{"type":"v1","addr":"172.21.15.3:6819","nonce":1897849827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":1897849827},{"type":"v1","addr":"172.21.15.3:6823","nonce":1897849827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":1897849827},{"type":"v1","addr":"172.21.15.3:6821","nonce":1897849827}]},"public_addr":"172.21.15.3:6817/1897849827","cluster_addr":"172.21.15.3:6819/1897849827","heartbeat_back_addr":"172.21.15.3:6823/1897849827","heartbeat_front_addr":"172.21.15.3:6821/1897849827","state":["exists","up"]},{"osd":1,"uuid":"646e882d-4ce4-4720-8c84-f71df563a35e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":926805498},{"type":"v1","addr":"172.21.15.3:6801","nonce":926805498}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":926805498},{"type":"v1","addr":"172.21.15.3:6803","nonce":926805498}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":926805498},{"type":"v1","addr":"172.21.15.3:6807","nonce":926805498}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":926805498},{"type":"v1","addr":"172.21.15.3:6805","nonce":926805498}]},"public_addr":"172.21.15.3:6801/926805498","cluster_addr":"172.21.15.3:6803/926805498","heartbeat_back_addr":"172.21.15.3:6807/926805498","heartbeat_front_addr":"172.21.15.3:6805/926805498","state":["exists","up"]},{"osd":2,"uuid":"1abbadb5-a570-4359-9643-acff3ede35a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":801295451},{"type":"v1","addr":"172.21.15.3:6809","nonce":801295451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":801295451},{"type":"v1","addr":"172.21.15.3:6811","nonce":801295451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":801295451},{"type":"v1","addr":"172.21.15.3:6815","nonce":801295451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":801295451},{"type":"v1","addr":"172.21.15.3:6813","nonce":801295451}]},"public_addr":"172.21.15.3:6809/801295451","cluster_addr":"172.21.15.3:6811/801295451","heartbeat_back_addr":"172.21.15.3:6815/801295451","heartbeat_front_addr":"172.21.15.3:6813/801295451","state":["exists","up"]},{"osd":3,"uuid":"62ac4dc1-2371-4dd2-acd3-e2dbb11aedf5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":1903389857},{"type":"v1","addr":"172.21.15.3:6825","nonce":1903389857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":1903389857},{"type":"v1","addr":"172.21.15.3:6827","nonce":1903389857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":1903389857},{"type":"v1","addr":"172.21.15.3:6831","nonce":1903389857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":1903389857},{"type":"v1","addr":"172.21.15.3:6829","nonce":1903389857}]},"public_addr":"172.21.15.3:6825/1903389857","cluster_addr":"172.21.15.3:6827/1903389857","heartbeat_back_addr":"172.21.15.3:6831/1903389857","heartbeat_front_addr":"172.21.15.3:6829/1903389857","state":["exists","up"]},{"osd":4,"uuid":"ae808398-8326-4d7c-9319-a2a0eafffb59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":2143624576},{"type":"v1","addr":"172.21.15.190:6801","nonce":2143624576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":2143624576},{"type":"v1","addr":"172.21.15.190:6803","nonce":2143624576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":2143624576},{"type":"v1","addr":"172.21.15.190:6807","nonce":2143624576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":2143624576},{"type":"v1","addr":"172.21.15.190:6805","nonce":2143624576}]},"public_addr":"172.21.15.190:6801/2143624576","cluster_addr":"172.21.15.190:6803/2143624576","heartbeat_back_addr":"172.21.15.190:6807/2143624576","heartbeat_front_addr":"172.21.15.190:6805/2143624576","state":["exists","up"]},{"osd":5,"uuid":"918adf50-c440-48ac-9a8d-4837562d7275","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":1766923126},{"type":"v1","addr":"172.21.15.190:6814","nonce":1766923126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":1766923126},{"type":"v1","addr":"172.21.15.190:6818","nonce":1766923126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":1766923126},{"type":"v1","addr":"172.21.15.190:6823","nonce":1766923126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":1766923126},{"type":"v1","addr":"172.21.15.190:6821","nonce":1766923126}]},"public_addr":"172.21.15.190:6814/1766923126","cluster_addr":"172.21.15.190:6818/1766923126","heartbeat_back_addr":"172.21.15.190:6823/1766923126","heartbeat_front_addr":"172.21.15.190:6821/1766923126","state":["exists","up"]},{"osd":6,"uuid":"ac2882bc-2a7c-463c-8f6c-76ff49828adc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":342414248},{"type":"v1","addr":"172.21.15.190:6825","nonce":342414248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":342414248},{"type":"v1","addr":"172.21.15.190:6827","nonce":342414248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":342414248},{"type":"v1","addr":"172.21.15.190:6831","nonce":342414248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":342414248},{"type":"v1","addr":"172.21.15.190:6829","nonce":342414248}]},"public_addr":"172.21.15.190:6825/342414248","cluster_addr":"172.21.15.190:6827/342414248","heartbeat_back_addr":"172.21.15.190:6831/342414248","heartbeat_front_addr":"172.21.15.190:6829/342414248","state":["exists","up"]},{"osd":7,"uuid":"dd4e767a-4730-4c70-8bd5-3f088140a5a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":537849891},{"type":"v1","addr":"172.21.15.190:6809","nonce":537849891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":537849891},{"type":"v1","addr":"172.21.15.190:6811","nonce":537849891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6817","nonce":537849891},{"type":"v1","addr":"172.21.15.190:6819","nonce":537849891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6813","nonce":537849891},{"type":"v1","addr":"172.21.15.190:6815","nonce":537849891}]},"public_addr":"172.21.15.190:6809/537849891","cluster_addr":"172.21.15.190:6811/537849891","heartbeat_back_addr":"172.21.15.190:6819/537849891","heartbeat_front_addr":"172.21.15.190:6815/537849891","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.889784+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.974002+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.973549+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.955396+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.928471+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.902418+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.958696+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.926474+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T06:37:11.751 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:11.769+0000 7f7879b5a640 1 -- 172.21.15.3:0/139463360 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f786006bec0 msgr2=0x7f786006e380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:11.751 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:11.769+0000 7f7879b5a640 1 --2- 172.21.15.3:0/139463360 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f786006bec0 0x7f786006e380 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f7864009560 tx=0x7f7864009190 comp rx=0 tx=0).stop 2024-05-09T06:37:11.751 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:11.769+0000 7f7879b5a640 1 -- 172.21.15.3:0/139463360 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f78740d2ed0 msgr2=0x7f787406cde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:11.751 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:11.769+0000 7f7879b5a640 1 --2- 172.21.15.3:0/139463360 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f78740d2ed0 0x7f787406cde0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f786800c470 tx=0x7f786800c930 comp rx=0 tx=0).stop 2024-05-09T06:37:11.751 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:11.769+0000 7f7879b5a640 1 -- 172.21.15.3:0/139463360 shutdown_connections 2024-05-09T06:37:11.751 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:11.769+0000 7f7879b5a640 1 --2- 172.21.15.3:0/139463360 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f786006bec0 0x7f786006e380 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:11.751 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:11.769+0000 7f7879b5a640 1 --2- 172.21.15.3:0/139463360 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f787415e480 0x7f787406d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:11.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:11.769+0000 7f7879b5a640 1 --2- 172.21.15.3:0/139463360 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f78740d2ed0 0x7f787406cde0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:11.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:11.769+0000 7f7879b5a640 1 --2- 172.21.15.3:0/139463360 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f78740d2310 0x7f787415b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:11.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:11.769+0000 7f7879b5a640 1 -- 172.21.15.3:0/139463360 >> 172.21.15.3:0/139463360 conn(0x7f787409d4d0 msgr2=0x7f78740c9d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:11.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:11.770+0000 7f7879b5a640 1 -- 172.21.15.3:0/139463360 shutdown_connections 2024-05-09T06:37:11.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:11.770+0000 7f7879b5a640 1 -- 172.21.15.3:0/139463360 wait complete. 2024-05-09T06:37:11.761 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-09T06:35:54.819999+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-09T06:36:56.494700+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '32', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2024-05-09T06:36:56.900006+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '33', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-09T06:37:11.762 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-09T06:37:11.869 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:11.888+0000 7f97045c1640 1 Processor -- start 2024-05-09T06:37:11.870 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:11.888+0000 7f97045c1640 1 -- start start 2024-05-09T06:37:11.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:11.889+0000 7f97045c1640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f96fc15c0f0 0x7f96fc160730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:11.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:11.889+0000 7f97045c1640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f96fc0b4e30 0x7f96fc06e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:11.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:11.890+0000 7f97045c1640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f96fc160c70 con 0x7f96fc06e790 2024-05-09T06:37:11.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:11.890+0000 7f97045c1640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f96fc06ec00 con 0x7f96fc15c0f0 2024-05-09T06:37:11.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:11.890+0000 7f97045c1640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f96fc0b40f0 con 0x7f96fc0b4e30 2024-05-09T06:37:11.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:11.890+0000 7f970235d640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f96fc15c0f0 0x7f96fc160730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:11.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:11.890+0000 7f970235d640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f96fc15c0f0 0x7f96fc160730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50870/0 (socket says 172.21.15.3:50870) 2024-05-09T06:37:11.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:11.890+0000 7f970235d640 1 -- 172.21.15.3:0/1674028928 learned_addr learned my addr 172.21.15.3:0/1674028928 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:37:11.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:11.890+0000 7f970235d640 1 -- 172.21.15.3:0/1674028928 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f96fc0b4e30 msgr2=0x7f96fc06e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:37:11.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:11.890+0000 7f9701b5c640 1 --2- 172.21.15.3:0/1674028928 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f96fc0b4e30 0x7f96fc06e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:11.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:11.890+0000 7f970235d640 1 --2- 172.21.15.3:0/1674028928 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f96fc0b4e30 0x7f96fc06e250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:11.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:11.890+0000 7f970235d640 1 -- 172.21.15.3:0/1674028928 >> v1:172.21.15.190:6789/0 conn(0x7f96fc06e790 legacy=0x7f96fc0cd5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:11.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:11.890+0000 7f970235d640 1 -- 172.21.15.3:0/1674028928 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96fc0b3050 con 0x7f96fc15c0f0 2024-05-09T06:37:11.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:11.890+0000 7f9701b5c640 1 --2- 172.21.15.3:0/1674028928 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f96fc0b4e30 0x7f96fc06e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:37:11.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:11.891+0000 7f970235d640 1 --2- 172.21.15.3:0/1674028928 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f96fc15c0f0 0x7f96fc160730 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f96f8009870 tx=0x7f96f802a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46d455b8213f3e70 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:11.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:11.891+0000 7f970135b640 1 -- 172.21.15.3:0/1674028928 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96f8037070 con 0x7f96fc15c0f0 2024-05-09T06:37:11.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:11.891+0000 7f970135b640 1 -- 172.21.15.3:0/1674028928 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96f8032440 con 0x7f96fc15c0f0 2024-05-09T06:37:11.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:11.891+0000 7f970135b640 1 -- 172.21.15.3:0/1674028928 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96f8032810 con 0x7f96fc15c0f0 2024-05-09T06:37:11.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:11.891+0000 7f97045c1640 1 -- 172.21.15.3:0/1674028928 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f96fc15c0f0 msgr2=0x7f96fc160730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:11.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:11.891+0000 7f97045c1640 1 --2- 172.21.15.3:0/1674028928 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f96fc15c0f0 0x7f96fc160730 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f96f8009870 tx=0x7f96f802a1c0 comp rx=0 tx=0).stop 2024-05-09T06:37:11.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:11.892+0000 7f97045c1640 1 -- 172.21.15.3:0/1674028928 shutdown_connections 2024-05-09T06:37:11.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:11.892+0000 7f97045c1640 1 --2- 172.21.15.3:0/1674028928 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f96fc0b4e30 0x7f96fc06e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:11.883 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:11.892+0000 7f97045c1640 1 --2- 172.21.15.3:0/1674028928 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f96fc15c0f0 0x7f96fc160730 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:11.883 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:11.892+0000 7f97045c1640 1 -- 172.21.15.3:0/1674028928 >> 172.21.15.3:0/1674028928 conn(0x7f96fc0b1c20 msgr2=0x7f96fc0b2030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:11.883 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:11.892+0000 7f97045c1640 1 -- 172.21.15.3:0/1674028928 shutdown_connections 2024-05-09T06:37:11.883 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:11.892+0000 7f97045c1640 1 -- 172.21.15.3:0/1674028928 wait complete. 2024-05-09T06:37:11.883 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:11.893+0000 7f97045c1640 1 Processor -- start 2024-05-09T06:37:11.883 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:11.893+0000 7f97045c1640 1 -- start start 2024-05-09T06:37:11.884 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:11.893+0000 7f97045c1640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f96fc06e790 0x7f96fc14abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:11.884 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:11.893+0000 7f97045c1640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f96fc0b4e30 0x7f96fc14ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:11.884 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:11.894+0000 7f97045c1640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f96fc15c0f0 0x7f96fc14cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:11.884 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:11.894+0000 7f97045c1640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f96fc0b38f0 con 0x7f96fc15c0f0 2024-05-09T06:37:11.884 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:11.894+0000 7f97045c1640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f96fc0b3610 con 0x7f96fc06e790 2024-05-09T06:37:11.884 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:11.894+0000 7f97045c1640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f96fc0b3bd0 con 0x7f96fc0b4e30 2024-05-09T06:37:11.884 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:11.894+0000 7f9702b5e640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f96fc15c0f0 0x7f96fc14cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:11.885 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:11.894+0000 7f9702b5e640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f96fc15c0f0 0x7f96fc14cfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50874/0 (socket says 172.21.15.3:50874) 2024-05-09T06:37:11.885 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:11.894+0000 7f9702b5e640 1 -- 172.21.15.3:0/2409501801 learned_addr learned my addr 172.21.15.3:0/2409501801 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:37:11.885 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:11.894+0000 7f970235d640 1 --2- 172.21.15.3:0/2409501801 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f96fc06e790 0x7f96fc14abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:11.885 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:11.894+0000 7f9701b5c640 1 --2- 172.21.15.3:0/2409501801 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f96fc0b4e30 0x7f96fc14ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:11.885 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:11.894+0000 7f9702b5e640 1 -- 172.21.15.3:0/2409501801 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f96fc0b4e30 msgr2=0x7f96fc14ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:11.885 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:11.894+0000 7f9702b5e640 1 --2- 172.21.15.3:0/2409501801 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f96fc0b4e30 0x7f96fc14ca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:11.886 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:11.894+0000 7f9702b5e640 1 -- 172.21.15.3:0/2409501801 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f96fc06e790 msgr2=0x7f96fc14abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:11.886 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:11.894+0000 7f9702b5e640 1 --2- 172.21.15.3:0/2409501801 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f96fc06e790 0x7f96fc14abf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:11.886 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:11.894+0000 7f9702b5e640 1 -- 172.21.15.3:0/2409501801 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96fc14d910 con 0x7f96fc15c0f0 2024-05-09T06:37:11.886 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:11.894+0000 7f9702b5e640 1 --2- 172.21.15.3:0/2409501801 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f96fc15c0f0 0x7f96fc14cfd0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f96f400c7a0 tx=0x7f96f400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:11.886 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:11.894+0000 7f96f2ffd640 1 -- 172.21.15.3:0/2409501801 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96f4003b70 con 0x7f96fc15c0f0 2024-05-09T06:37:11.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:11.894+0000 7f96f2ffd640 1 -- 172.21.15.3:0/2409501801 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96f4003d30 con 0x7f96fc15c0f0 2024-05-09T06:37:11.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:11.894+0000 7f97045c1640 1 -- 172.21.15.3:0/2409501801 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96fc14dbf0 con 0x7f96fc15c0f0 2024-05-09T06:37:11.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:11.894+0000 7f96f2ffd640 1 -- 172.21.15.3:0/2409501801 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96f400aa90 con 0x7f96fc15c0f0 2024-05-09T06:37:11.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:11.894+0000 7f97045c1640 1 -- 172.21.15.3:0/2409501801 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96fc13ac40 con 0x7f96fc15c0f0 2024-05-09T06:37:11.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:11.895+0000 7f97045c1640 1 -- 172.21.15.3:0/2409501801 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96fc069550 con 0x7f96fc15c0f0 2024-05-09T06:37:11.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:11.896+0000 7f96f2ffd640 1 -- 172.21.15.3:0/2409501801 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 7) ==== 90651+0+0 (secure 0 0 0) 0x7f96f400ac30 con 0x7f96fc15c0f0 2024-05-09T06:37:11.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:11.897+0000 7f96f2ffd640 1 --2- 172.21.15.3:0/2409501801 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f96e006bfe0 0x7f96e006e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:11.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:11.897+0000 7f96f2ffd640 1 -- 172.21.15.3:0/2409501801 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f96f408b090 con 0x7f96fc15c0f0 2024-05-09T06:37:11.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:11.897+0000 7f970235d640 1 --2- 172.21.15.3:0/2409501801 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f96e006bfe0 0x7f96e006e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:11.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:11.897+0000 7f970235d640 1 --2- 172.21.15.3:0/2409501801 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f96e006bfe0 0x7f96e006e4a0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f96f8034000 tx=0x7f96f802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:11.889 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:11.901+0000 7f96f2ffd640 1 -- 172.21.15.3:0/2409501801 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f96fc160c70 con 0x7f96fc15c0f0 2024-05-09T06:37:12.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:12.086+0000 7f97045c1640 1 -- 172.21.15.3:0/2409501801 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f96fc0b42e0 con 0x7f96fc15c0f0 2024-05-09T06:37:12.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:12.087+0000 7f96f2ffd640 1 -- 172.21.15.3:0/2409501801 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v33) ==== 93+0+10 (secure 0 0 0) 0x7f96f4059850 con 0x7f96fc15c0f0 2024-05-09T06:37:12.067 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 1 2024-05-09T06:37:12.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:12.089+0000 7f97045c1640 1 -- 172.21.15.3:0/2409501801 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f96e006bfe0 msgr2=0x7f96e006e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:12.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:12.089+0000 7f97045c1640 1 --2- 172.21.15.3:0/2409501801 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f96e006bfe0 0x7f96e006e4a0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f96f8034000 tx=0x7f96f802e000 comp rx=0 tx=0).stop 2024-05-09T06:37:12.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:12.089+0000 7f97045c1640 1 -- 172.21.15.3:0/2409501801 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f96fc15c0f0 msgr2=0x7f96fc14cfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:12.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:12.089+0000 7f97045c1640 1 --2- 172.21.15.3:0/2409501801 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f96fc15c0f0 0x7f96fc14cfd0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f96f400c7a0 tx=0x7f96f400cc60 comp rx=0 tx=0).stop 2024-05-09T06:37:12.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:12.089+0000 7f97045c1640 1 -- 172.21.15.3:0/2409501801 shutdown_connections 2024-05-09T06:37:12.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:12.089+0000 7f97045c1640 1 --2- 172.21.15.3:0/2409501801 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f96e006bfe0 0x7f96e006e4a0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:12.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:12.089+0000 7f97045c1640 1 --2- 172.21.15.3:0/2409501801 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f96fc15c0f0 0x7f96fc14cfd0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:12.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:12.089+0000 7f97045c1640 1 --2- 172.21.15.3:0/2409501801 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f96fc0b4e30 0x7f96fc14ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:12.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:12.089+0000 7f97045c1640 1 --2- 172.21.15.3:0/2409501801 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f96fc06e790 0x7f96fc14abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:12.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:12.089+0000 7f97045c1640 1 -- 172.21.15.3:0/2409501801 >> 172.21.15.3:0/2409501801 conn(0x7f96fc0b1c20 msgr2=0x7f96fc1584b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:12.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:12.089+0000 7f97045c1640 1 -- 172.21.15.3:0/2409501801 shutdown_connections 2024-05-09T06:37:12.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:12.089+0000 7f97045c1640 1 -- 172.21.15.3:0/2409501801 wait complete. 2024-05-09T06:37:12.081 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-09T06:37:12.189 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:12.209+0000 7ff9271c8640 1 Processor -- start 2024-05-09T06:37:12.190 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:12.209+0000 7ff9271c8640 1 -- start start 2024-05-09T06:37:12.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:12.209+0000 7ff9271c8640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff92015c0f0 0x7ff920160730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:12.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:12.210+0000 7ff9271c8640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7ff9200b4e30 0x7ff92006e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:12.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:12.210+0000 7ff9271c8640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff92006e790 0x7ff9200cd5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:12.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:12.210+0000 7ff9271c8640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7ff92006ec00 con 0x7ff92015c0f0 2024-05-09T06:37:12.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:12.210+0000 7ff9271c8640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7ff920160c70 con 0x7ff9200b4e30 2024-05-09T06:37:12.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:12.210+0000 7ff9271c8640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7ff9200b40f0 con 0x7ff92006e790 2024-05-09T06:37:12.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:12.210+0000 7ff924f64640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff92015c0f0 0x7ff920160730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:12.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:12.210+0000 7ff924f64640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff92015c0f0 0x7ff920160730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50878/0 (socket says 172.21.15.3:50878) 2024-05-09T06:37:12.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:12.210+0000 7ff924f64640 1 -- 172.21.15.3:0/207179762 learned_addr learned my addr 172.21.15.3:0/207179762 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:37:12.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:12.210+0000 7ff91ffff640 1 --2- 172.21.15.3:0/207179762 >> v2:172.21.15.190:3300/0 conn(0x7ff9200b4e30 0x7ff92006e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:12.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:12.210+0000 7ff925765640 1 --2- 172.21.15.3:0/207179762 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff92006e790 0x7ff9200cd5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:12.192 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:12.210+0000 7ff924f64640 1 -- 172.21.15.3:0/207179762 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff92006e790 msgr2=0x7ff9200cd5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:37:12.192 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:12.210+0000 7ff924f64640 1 --2- 172.21.15.3:0/207179762 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff92006e790 0x7ff9200cd5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:12.192 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:12.210+0000 7ff924f64640 1 -- 172.21.15.3:0/207179762 >> v2:172.21.15.190:3300/0 conn(0x7ff9200b4e30 msgr2=0x7ff92006e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:37:12.192 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:12.210+0000 7ff924f64640 1 --2- 172.21.15.3:0/207179762 >> v2:172.21.15.190:3300/0 conn(0x7ff9200b4e30 0x7ff92006e250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:12.192 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:12.210+0000 7ff924f64640 1 -- 172.21.15.3:0/207179762 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9200b3050 con 0x7ff92015c0f0 2024-05-09T06:37:12.192 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:12.210+0000 7ff91ffff640 1 --2- 172.21.15.3:0/207179762 >> v2:172.21.15.190:3300/0 conn(0x7ff9200b4e30 0x7ff92006e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:37:12.192 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:12.210+0000 7ff925765640 1 --2- 172.21.15.3:0/207179762 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff92006e790 0x7ff9200cd5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:37:12.194 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:12.211+0000 7ff924f64640 1 --2- 172.21.15.3:0/207179762 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff92015c0f0 0x7ff920160730 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7ff920073500 tx=0x7ff91802a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48e286517c37d96b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:12.194 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:12.211+0000 7ff91f7fe640 1 -- 172.21.15.3:0/207179762 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff918037070 con 0x7ff92015c0f0 2024-05-09T06:37:12.194 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:12.211+0000 7ff91f7fe640 1 -- 172.21.15.3:0/207179762 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff918032440 con 0x7ff92015c0f0 2024-05-09T06:37:12.194 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:12.211+0000 7ff91f7fe640 1 -- 172.21.15.3:0/207179762 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff918032810 con 0x7ff92015c0f0 2024-05-09T06:37:12.194 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:12.211+0000 7ff9271c8640 1 -- 172.21.15.3:0/207179762 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff92015c0f0 msgr2=0x7ff920160730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:12.194 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:12.211+0000 7ff9271c8640 1 --2- 172.21.15.3:0/207179762 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff92015c0f0 0x7ff920160730 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7ff920073500 tx=0x7ff91802a140 comp rx=0 tx=0).stop 2024-05-09T06:37:12.194 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:12.211+0000 7ff9271c8640 1 -- 172.21.15.3:0/207179762 shutdown_connections 2024-05-09T06:37:12.194 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:12.211+0000 7ff9271c8640 1 --2- 172.21.15.3:0/207179762 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff92006e790 0x7ff9200cd5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:12.194 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:12.211+0000 7ff9271c8640 1 --2- 172.21.15.3:0/207179762 >> v2:172.21.15.190:3300/0 conn(0x7ff9200b4e30 0x7ff92006e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:12.194 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:12.211+0000 7ff9271c8640 1 --2- 172.21.15.3:0/207179762 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff92015c0f0 0x7ff920160730 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:12.194 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:12.211+0000 7ff9271c8640 1 -- 172.21.15.3:0/207179762 >> 172.21.15.3:0/207179762 conn(0x7ff9200b1c20 msgr2=0x7ff9200b2030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:12.194 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:12.211+0000 7ff9271c8640 1 -- 172.21.15.3:0/207179762 shutdown_connections 2024-05-09T06:37:12.194 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:12.211+0000 7ff9271c8640 1 -- 172.21.15.3:0/207179762 wait complete. 2024-05-09T06:37:12.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:12.213+0000 7ff9271c8640 1 Processor -- start 2024-05-09T06:37:12.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:12.213+0000 7ff9271c8640 1 -- start start 2024-05-09T06:37:12.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:12.214+0000 7ff9271c8640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff92006e790 0x7ff92014ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:12.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:12.214+0000 7ff9271c8640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff9200b4e30 0x7ff92015cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:12.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:12.214+0000 7ff9271c8640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff92015c0f0 0x7ff92015d090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:12.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:12.214+0000 7ff9271c8640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7ff9200b38f0 con 0x7ff92015c0f0 2024-05-09T06:37:12.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:12.214+0000 7ff9271c8640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7ff9200b3610 con 0x7ff9200b4e30 2024-05-09T06:37:12.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:12.214+0000 7ff9271c8640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7ff9200b3bd0 con 0x7ff92006e790 2024-05-09T06:37:12.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:12.214+0000 7ff925765640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff92015c0f0 0x7ff92015d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:12.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:12.214+0000 7ff925765640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff92015c0f0 0x7ff92015d090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50880/0 (socket says 172.21.15.3:50880) 2024-05-09T06:37:12.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:12.214+0000 7ff925765640 1 -- 172.21.15.3:0/2806953671 learned_addr learned my addr 172.21.15.3:0/2806953671 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:37:12.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:12.214+0000 7ff924f64640 1 --2- 172.21.15.3:0/2806953671 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff92006e790 0x7ff92014ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:12.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:12.214+0000 7ff91ffff640 1 --2- 172.21.15.3:0/2806953671 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff9200b4e30 0x7ff92015cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:12.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:12.214+0000 7ff925765640 1 -- 172.21.15.3:0/2806953671 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff92006e790 msgr2=0x7ff92014ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:12.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:12.214+0000 7ff925765640 1 --2- 172.21.15.3:0/2806953671 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff92006e790 0x7ff92014ed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:12.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:12.214+0000 7ff925765640 1 -- 172.21.15.3:0/2806953671 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff9200b4e30 msgr2=0x7ff92015cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:12.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:12.214+0000 7ff925765640 1 --2- 172.21.15.3:0/2806953671 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff9200b4e30 0x7ff92015cb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:12.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:12.214+0000 7ff925765640 1 -- 172.21.15.3:0/2806953671 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff92014f560 con 0x7ff92015c0f0 2024-05-09T06:37:12.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:12.214+0000 7ff925765640 1 --2- 172.21.15.3:0/2806953671 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff92015c0f0 0x7ff92015d090 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7ff90800c7f0 tx=0x7ff90800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:12.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:12.214+0000 7ff91d7fa640 1 -- 172.21.15.3:0/2806953671 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff908015070 con 0x7ff92015c0f0 2024-05-09T06:37:12.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:12.214+0000 7ff91d7fa640 1 -- 172.21.15.3:0/2806953671 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff908007820 con 0x7ff92015c0f0 2024-05-09T06:37:12.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:12.214+0000 7ff9271c8640 1 -- 172.21.15.3:0/2806953671 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff92014f840 con 0x7ff92015c0f0 2024-05-09T06:37:12.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:12.214+0000 7ff91d7fa640 1 -- 172.21.15.3:0/2806953671 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff908007c50 con 0x7ff92015c0f0 2024-05-09T06:37:12.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:12.215+0000 7ff9271c8640 1 -- 172.21.15.3:0/2806953671 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9200715b0 con 0x7ff92015c0f0 2024-05-09T06:37:12.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:12.215+0000 7ff9271c8640 1 -- 172.21.15.3:0/2806953671 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9200b4c50 con 0x7ff92015c0f0 2024-05-09T06:37:12.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:12.216+0000 7ff91d7fa640 1 -- 172.21.15.3:0/2806953671 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 7) ==== 90651+0+0 (secure 0 0 0) 0x7ff908018070 con 0x7ff92015c0f0 2024-05-09T06:37:12.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:12.216+0000 7ff91d7fa640 1 --2- 172.21.15.3:0/2806953671 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7ff90006bfe0 0x7ff90006e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:12.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:12.217+0000 7ff91d7fa640 1 -- 172.21.15.3:0/2806953671 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7ff90808b3c0 con 0x7ff92015c0f0 2024-05-09T06:37:12.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:12.217+0000 7ff924f64640 1 --2- 172.21.15.3:0/2806953671 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7ff90006bfe0 0x7ff90006e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:12.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:12.217+0000 7ff924f64640 1 --2- 172.21.15.3:0/2806953671 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7ff90006bfe0 0x7ff90006e4a0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7ff918034000 tx=0x7ff91802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:12.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:12.221+0000 7ff91d7fa640 1 -- 172.21.15.3:0/2806953671 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7ff9200b4c50 con 0x7ff92015c0f0 2024-05-09T06:37:12.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:12.405+0000 7ff9271c8640 1 -- 172.21.15.3:0/2806953671 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7ff9200717b0 con 0x7ff92015c0f0 2024-05-09T06:37:12.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:12.405+0000 7ff91d7fa640 1 -- 172.21.15.3:0/2806953671 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v33) ==== 104+0+11 (secure 0 0 0) 0x7ff9200717b0 con 0x7ff92015c0f0 2024-05-09T06:37:12.386 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 32 2024-05-09T06:37:12.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:12.407+0000 7ff9271c8640 1 -- 172.21.15.3:0/2806953671 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7ff90006bfe0 msgr2=0x7ff90006e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:12.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:12.407+0000 7ff9271c8640 1 --2- 172.21.15.3:0/2806953671 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7ff90006bfe0 0x7ff90006e4a0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7ff918034000 tx=0x7ff91802e000 comp rx=0 tx=0).stop 2024-05-09T06:37:12.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:12.407+0000 7ff9271c8640 1 -- 172.21.15.3:0/2806953671 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff92015c0f0 msgr2=0x7ff92015d090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:12.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:12.407+0000 7ff9271c8640 1 --2- 172.21.15.3:0/2806953671 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff92015c0f0 0x7ff92015d090 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7ff90800c7f0 tx=0x7ff90800ccb0 comp rx=0 tx=0).stop 2024-05-09T06:37:12.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:12.407+0000 7ff9271c8640 1 -- 172.21.15.3:0/2806953671 shutdown_connections 2024-05-09T06:37:12.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:12.407+0000 7ff9271c8640 1 --2- 172.21.15.3:0/2806953671 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7ff90006bfe0 0x7ff90006e4a0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:12.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:12.407+0000 7ff9271c8640 1 --2- 172.21.15.3:0/2806953671 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff92015c0f0 0x7ff92015d090 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:12.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:12.407+0000 7ff9271c8640 1 --2- 172.21.15.3:0/2806953671 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff9200b4e30 0x7ff92015cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:12.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:12.407+0000 7ff9271c8640 1 --2- 172.21.15.3:0/2806953671 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff92006e790 0x7ff92014ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:12.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:12.407+0000 7ff9271c8640 1 -- 172.21.15.3:0/2806953671 >> 172.21.15.3:0/2806953671 conn(0x7ff9200b1c20 msgr2=0x7ff9200b4540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:12.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:12.407+0000 7ff9271c8640 1 -- 172.21.15.3:0/2806953671 shutdown_connections 2024-05-09T06:37:12.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:12.407+0000 7ff9271c8640 1 -- 172.21.15.3:0/2806953671 wait complete. 2024-05-09T06:37:12.399 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-09T06:37:12.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:12.525+0000 7fd04629a640 1 Processor -- start 2024-05-09T06:37:12.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:12.525+0000 7fd04629a640 1 -- start start 2024-05-09T06:37:12.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:12.526+0000 7fd04629a640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fd040144a10 0x7fd0400cf8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:12.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:12.526+0000 7fd04629a640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fd0400b4130 0x7fd0400b45e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:12.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:12.526+0000 7fd04629a640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd040071860 con 0x7fd040072900 2024-05-09T06:37:12.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:12.526+0000 7fd04629a640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fd040072470 con 0x7fd040144a10 2024-05-09T06:37:12.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:12.526+0000 7fd04629a640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7fd040071b40 con 0x7fd0400b4130 2024-05-09T06:37:12.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:12.526+0000 7fd03ffff640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fd040144a10 0x7fd0400cf8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:12.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:12.526+0000 7fd03f7fe640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fd040072900 0x7fd040071080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:45892/0 (socket says 172.21.15.3:45892) 2024-05-09T06:37:12.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:12.526+0000 7fd03f7fe640 1 -- 172.21.15.3:0/3932545808 learned_addr learned my addr 172.21.15.3:0/3932545808 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-05-09T06:37:12.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:12.526+0000 7fd03ffff640 1 -- 172.21.15.3:0/3932545808 >> v2:172.21.15.190:3300/0 conn(0x7fd0400b4130 msgr2=0x7fd0400b45e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:37:12.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:12.526+0000 7fd03ffff640 1 --2- 172.21.15.3:0/3932545808 >> v2:172.21.15.190:3300/0 conn(0x7fd0400b4130 0x7fd0400b45e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:12.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:12.526+0000 7fd03ffff640 1 -- 172.21.15.3:0/3932545808 >> v1:172.21.15.3:6789/0 conn(0x7fd040072900 legacy=0x7fd040071080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:12.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:12.526+0000 7fd03ffff640 1 -- 172.21.15.3:0/3932545808 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd040071e20 con 0x7fd040144a10 2024-05-09T06:37:12.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:12.526+0000 7fd03effd640 1 -- 172.21.15.3:0/3932545808 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 854402614 0 0) 0x7fd040071860 con 0x7fd040072900 2024-05-09T06:37:12.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:12.527+0000 7fd03ffff640 1 --2- 172.21.15.3:0/3932545808 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd040144a10 0x7fd0400cf8a0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fd0400b2fb0 tx=0x7fd03002a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee8a145d9c598258 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:12.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:12.527+0000 7fd03effd640 1 -- 172.21.15.3:0/3932545808 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd030037070 con 0x7fd040144a10 2024-05-09T06:37:12.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:12.527+0000 7fd03effd640 1 -- 172.21.15.3:0/3932545808 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd030032440 con 0x7fd040144a10 2024-05-09T06:37:12.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:12.527+0000 7fd03effd640 1 -- 172.21.15.3:0/3932545808 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd03002ae60 con 0x7fd040144a10 2024-05-09T06:37:12.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:12.528+0000 7fd04629a640 1 -- 172.21.15.3:0/3932545808 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd040144a10 msgr2=0x7fd0400cf8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:12.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:12.528+0000 7fd04629a640 1 --2- 172.21.15.3:0/3932545808 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd040144a10 0x7fd0400cf8a0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fd0400b2fb0 tx=0x7fd03002a110 comp rx=0 tx=0).stop 2024-05-09T06:37:12.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:12.528+0000 7fd04629a640 1 -- 172.21.15.3:0/3932545808 shutdown_connections 2024-05-09T06:37:12.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:12.528+0000 7fd04629a640 1 --2- 172.21.15.3:0/3932545808 >> v2:172.21.15.190:3300/0 conn(0x7fd0400b4130 0x7fd0400b45e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:12.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:12.528+0000 7fd04629a640 1 --2- 172.21.15.3:0/3932545808 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd040144a10 0x7fd0400cf8a0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:12.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:12.528+0000 7fd04629a640 1 -- 172.21.15.3:0/3932545808 >> 172.21.15.3:0/3932545808 conn(0x7fd0400b1c20 msgr2=0x7fd0400b2030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:12.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:12.528+0000 7fd04629a640 1 -- 172.21.15.3:0/3932545808 shutdown_connections 2024-05-09T06:37:12.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:12.528+0000 7fd04629a640 1 -- 172.21.15.3:0/3932545808 wait complete. 2024-05-09T06:37:12.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:12.529+0000 7fd04629a640 1 Processor -- start 2024-05-09T06:37:12.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:12.529+0000 7fd04629a640 1 -- start start 2024-05-09T06:37:12.513 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:12.530+0000 7fd04629a640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd040072900 0x7fd0401e4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:12.513 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:12.530+0000 7fd04629a640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd0400b4130 0x7fd04015d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:12.513 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:12.530+0000 7fd04629a640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd040144a10 0x7fd040142ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:12.513 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:12.530+0000 7fd04629a640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fd04015b750 con 0x7fd0400b4130 2024-05-09T06:37:12.513 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:12.530+0000 7fd04629a640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fd04015b5d0 con 0x7fd040144a10 2024-05-09T06:37:12.513 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:12.530+0000 7fd04629a640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fd04015b8d0 con 0x7fd040072900 2024-05-09T06:37:12.513 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:12.530+0000 7fd03f7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd0400b4130 0x7fd04015d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:12.513 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:12.530+0000 7fd03f7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd0400b4130 0x7fd04015d660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50898/0 (socket says 172.21.15.3:50898) 2024-05-09T06:37:12.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:12.530+0000 7fd03f7fe640 1 -- 172.21.15.3:0/4222999633 learned_addr learned my addr 172.21.15.3:0/4222999633 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:37:12.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:12.530+0000 7fd03f7fe640 1 -- 172.21.15.3:0/4222999633 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd040072900 msgr2=0x7fd0401e4f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:12.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:12.530+0000 7fd03f7fe640 1 --2- 172.21.15.3:0/4222999633 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd040072900 0x7fd0401e4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:12.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:12.530+0000 7fd03f7fe640 1 -- 172.21.15.3:0/4222999633 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd040144a10 msgr2=0x7fd040142ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:12.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:12.530+0000 7fd044837640 1 --2- 172.21.15.3:0/4222999633 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd040144a10 0x7fd040142ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:12.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:12.531+0000 7fd03f7fe640 1 --2- 172.21.15.3:0/4222999633 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd040144a10 0x7fd040142ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:12.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:12.531+0000 7fd03f7fe640 1 -- 172.21.15.3:0/4222999633 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0401e55e0 con 0x7fd0400b4130 2024-05-09T06:37:12.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:12.531+0000 7fd044837640 1 --2- 172.21.15.3:0/4222999633 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd040144a10 0x7fd040142ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:37:12.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:12.531+0000 7fd03f7fe640 1 --2- 172.21.15.3:0/4222999633 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd0400b4130 0x7fd04015d660 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fd03400c4b0 tx=0x7fd03400c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:12.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:12.531+0000 7fd03cff9640 1 -- 172.21.15.3:0/4222999633 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd034018070 con 0x7fd0400b4130 2024-05-09T06:37:12.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:12.531+0000 7fd03cff9640 1 -- 172.21.15.3:0/4222999633 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd034007500 con 0x7fd0400b4130 2024-05-09T06:37:12.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:12.531+0000 7fd04629a640 1 -- 172.21.15.3:0/4222999633 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0401e37d0 con 0x7fd0400b4130 2024-05-09T06:37:12.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:12.531+0000 7fd03cff9640 1 -- 172.21.15.3:0/4222999633 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd034004a20 con 0x7fd0400b4130 2024-05-09T06:37:12.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:12.531+0000 7fd04629a640 1 -- 172.21.15.3:0/4222999633 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0401e3d90 con 0x7fd0400b4130 2024-05-09T06:37:12.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:12.532+0000 7fd04629a640 1 -- 172.21.15.3:0/4222999633 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd040069550 con 0x7fd0400b4130 2024-05-09T06:37:12.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:12.533+0000 7fd03cff9640 1 -- 172.21.15.3:0/4222999633 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 7) ==== 90651+0+0 (secure 0 0 0) 0x7fd03401b070 con 0x7fd0400b4130 2024-05-09T06:37:12.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:12.533+0000 7fd03cff9640 1 --2- 172.21.15.3:0/4222999633 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fd02c06bfe0 0x7fd02c06e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:12.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:12.533+0000 7fd03cff9640 1 -- 172.21.15.3:0/4222999633 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fd03408a4a0 con 0x7fd0400b4130 2024-05-09T06:37:12.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:12.533+0000 7fd03ffff640 1 --2- 172.21.15.3:0/4222999633 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fd02c06bfe0 0x7fd02c06e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:12.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:12.534+0000 7fd03ffff640 1 --2- 172.21.15.3:0/4222999633 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fd02c06bfe0 0x7fd02c06e4a0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fd030034000 tx=0x7fd03002c000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:12.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:12.537+0000 7fd03cff9640 1 -- 172.21.15.3:0/4222999633 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fd034058c90 con 0x7fd0400b4130 2024-05-09T06:37:12.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:12.729+0000 7fd04629a640 1 -- 172.21.15.3:0/4222999633 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fd04006d370 con 0x7fd0400b4130 2024-05-09T06:37:12.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:12.729+0000 7fd03cff9640 1 -- 172.21.15.3:0/4222999633 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v33) ==== 100+0+11 (secure 0 0 0) 0x7fd034058590 con 0x7fd0400b4130 2024-05-09T06:37:12.710 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 64 2024-05-09T06:37:12.712 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:12.731+0000 7fd04629a640 1 -- 172.21.15.3:0/4222999633 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fd02c06bfe0 msgr2=0x7fd02c06e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:12.712 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:12.731+0000 7fd04629a640 1 --2- 172.21.15.3:0/4222999633 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fd02c06bfe0 0x7fd02c06e4a0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fd030034000 tx=0x7fd03002c000 comp rx=0 tx=0).stop 2024-05-09T06:37:12.712 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:12.731+0000 7fd04629a640 1 -- 172.21.15.3:0/4222999633 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd0400b4130 msgr2=0x7fd04015d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:12.712 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:12.731+0000 7fd04629a640 1 --2- 172.21.15.3:0/4222999633 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd0400b4130 0x7fd04015d660 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fd03400c4b0 tx=0x7fd03400c970 comp rx=0 tx=0).stop 2024-05-09T06:37:12.712 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:12.731+0000 7fd04629a640 1 -- 172.21.15.3:0/4222999633 shutdown_connections 2024-05-09T06:37:12.712 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:12.731+0000 7fd04629a640 1 --2- 172.21.15.3:0/4222999633 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fd02c06bfe0 0x7fd02c06e4a0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:12.713 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:12.731+0000 7fd04629a640 1 --2- 172.21.15.3:0/4222999633 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd040144a10 0x7fd040142ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:12.713 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:12.731+0000 7fd04629a640 1 --2- 172.21.15.3:0/4222999633 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd0400b4130 0x7fd04015d660 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:12.713 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:12.731+0000 7fd04629a640 1 --2- 172.21.15.3:0/4222999633 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd040072900 0x7fd0401e4f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:12.713 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:12.731+0000 7fd04629a640 1 -- 172.21.15.3:0/4222999633 >> 172.21.15.3:0/4222999633 conn(0x7fd0400b1c20 msgr2=0x7fd04006eda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:12.713 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:12.731+0000 7fd04629a640 1 -- 172.21.15.3:0/4222999633 shutdown_connections 2024-05-09T06:37:12.713 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:12.731+0000 7fd04629a640 1 -- 172.21.15.3:0/4222999633 wait complete. 2024-05-09T06:37:12.723 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-09T06:37:12.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:12.848+0000 7f1e69dc1640 1 Processor -- start 2024-05-09T06:37:12.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:12.848+0000 7f1e69dc1640 1 -- start start 2024-05-09T06:37:12.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:12.848+0000 7f1e69dc1640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1e6415e5c0 0x7f1e640d2800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:12.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:12.848+0000 7f1e69dc1640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f1e640d1300 0x7f1e640b47e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:12.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:12.848+0000 7f1e69dc1640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1e6415e9a0 con 0x7f1e640d4c40 2024-05-09T06:37:12.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:12.849+0000 7f1e69dc1640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f1e640d31c0 con 0x7f1e640d1300 2024-05-09T06:37:12.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:12.849+0000 7f1e69dc1640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f1e640d50a0 con 0x7f1e6415e5c0 2024-05-09T06:37:12.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:12.849+0000 7f1e63fff640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f1e640d1300 0x7f1e640b47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:12.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:12.849+0000 7f1e63fff640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f1e640d1300 0x7f1e640b47e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50908/0 (socket says 172.21.15.3:50908) 2024-05-09T06:37:12.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:12.849+0000 7f1e63fff640 1 -- 172.21.15.3:0/612199547 learned_addr learned my addr 172.21.15.3:0/612199547 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:37:12.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:12.849+0000 7f1e637fe640 1 --2- 172.21.15.3:0/612199547 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1e6415e5c0 0x7f1e640d2800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:12.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:12.849+0000 7f1e63fff640 1 -- 172.21.15.3:0/612199547 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1e6415e5c0 msgr2=0x7f1e640d2800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:37:12.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:12.849+0000 7f1e63fff640 1 --2- 172.21.15.3:0/612199547 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1e6415e5c0 0x7f1e640d2800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:12.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:12.849+0000 7f1e63fff640 1 -- 172.21.15.3:0/612199547 >> v1:172.21.15.190:6789/0 conn(0x7f1e640d4c40 legacy=0x7f1e640d0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:37:12.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:12.849+0000 7f1e63fff640 1 -- 172.21.15.3:0/612199547 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e640d2ee0 con 0x7f1e640d1300 2024-05-09T06:37:12.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:12.850+0000 7f1e63fff640 1 --2- 172.21.15.3:0/612199547 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1e640d1300 0x7f1e640b47e0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f1e5c009e50 tx=0x7f1e5c02a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61295222bafb1161 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:12.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:12.850+0000 7f1e627fc640 1 -- 172.21.15.3:0/612199547 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e5c036070 con 0x7f1e640d1300 2024-05-09T06:37:12.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:12.850+0000 7f1e627fc640 1 -- 172.21.15.3:0/612199547 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e5c031600 con 0x7f1e640d1300 2024-05-09T06:37:12.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:12.850+0000 7f1e627fc640 1 -- 172.21.15.3:0/612199547 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e5c0319d0 con 0x7f1e640d1300 2024-05-09T06:37:12.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:12.850+0000 7f1e69dc1640 1 -- 172.21.15.3:0/612199547 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1e640d1300 msgr2=0x7f1e640b47e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:12.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:12.850+0000 7f1e69dc1640 1 --2- 172.21.15.3:0/612199547 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1e640d1300 0x7f1e640b47e0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f1e5c009e50 tx=0x7f1e5c02a6c0 comp rx=0 tx=0).stop 2024-05-09T06:37:12.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:12.850+0000 7f1e69dc1640 1 -- 172.21.15.3:0/612199547 shutdown_connections 2024-05-09T06:37:12.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:12.850+0000 7f1e69dc1640 1 --2- 172.21.15.3:0/612199547 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1e640d1300 0x7f1e640b47e0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:12.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:12.850+0000 7f1e69dc1640 1 --2- 172.21.15.3:0/612199547 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1e6415e5c0 0x7f1e640d2800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:12.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:12.850+0000 7f1e69dc1640 1 -- 172.21.15.3:0/612199547 >> 172.21.15.3:0/612199547 conn(0x7f1e6409d500 msgr2=0x7f1e640d4220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:12.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:12.850+0000 7f1e69dc1640 1 -- 172.21.15.3:0/612199547 shutdown_connections 2024-05-09T06:37:12.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:12.850+0000 7f1e69dc1640 1 -- 172.21.15.3:0/612199547 wait complete. 2024-05-09T06:37:12.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:12.852+0000 7f1e69dc1640 1 Processor -- start 2024-05-09T06:37:12.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:12.852+0000 7f1e69dc1640 1 -- start start 2024-05-09T06:37:12.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:12.853+0000 7f1e69dc1640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1e640d1300 0x7f1e640ca960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:12.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:12.853+0000 7f1e69dc1640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1e640d4c40 0x7f1e640ce060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:12.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:12.853+0000 7f1e69dc1640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1e6415e5c0 0x7f1e64160f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:12.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:12.853+0000 7f1e69dc1640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f1e640b42c0 con 0x7f1e640d4c40 2024-05-09T06:37:12.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:12.853+0000 7f1e69dc1640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f1e640b3fe0 con 0x7f1e6415e5c0 2024-05-09T06:37:12.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:12.853+0000 7f1e69dc1640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f1e640b4d20 con 0x7f1e640d1300 2024-05-09T06:37:12.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:12.853+0000 7f1e62ffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1e640d4c40 0x7f1e640ce060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:12.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:12.853+0000 7f1e62ffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1e640d4c40 0x7f1e640ce060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50912/0 (socket says 172.21.15.3:50912) 2024-05-09T06:37:12.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:12.853+0000 7f1e62ffd640 1 -- 172.21.15.3:0/3060070691 learned_addr learned my addr 172.21.15.3:0/3060070691 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:37:12.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:12.853+0000 7f1e63fff640 1 --2- 172.21.15.3:0/3060070691 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1e6415e5c0 0x7f1e64160f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:12.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:12.853+0000 7f1e637fe640 1 --2- 172.21.15.3:0/3060070691 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1e640d1300 0x7f1e640ca960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:12.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:12.853+0000 7f1e62ffd640 1 -- 172.21.15.3:0/3060070691 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1e640d1300 msgr2=0x7f1e640ca960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:12.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:12.853+0000 7f1e62ffd640 1 --2- 172.21.15.3:0/3060070691 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1e640d1300 0x7f1e640ca960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:12.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:12.853+0000 7f1e62ffd640 1 -- 172.21.15.3:0/3060070691 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1e6415e5c0 msgr2=0x7f1e64160f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:12.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:12.853+0000 7f1e62ffd640 1 --2- 172.21.15.3:0/3060070691 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1e6415e5c0 0x7f1e64160f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:12.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:12.853+0000 7f1e62ffd640 1 -- 172.21.15.3:0/3060070691 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e640ce5a0 con 0x7f1e640d4c40 2024-05-09T06:37:12.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:12.853+0000 7f1e63fff640 1 --2- 172.21.15.3:0/3060070691 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1e6415e5c0 0x7f1e64160f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:37:12.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:12.853+0000 7f1e637fe640 1 --2- 172.21.15.3:0/3060070691 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1e640d1300 0x7f1e640ca960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:37:12.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:12.853+0000 7f1e62ffd640 1 --2- 172.21.15.3:0/3060070691 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1e640d4c40 0x7f1e640ce060 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f1e5800c430 tx=0x7f1e5800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:12.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:12.854+0000 7f1e4bfff640 1 -- 172.21.15.3:0/3060070691 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e58011790 con 0x7f1e640d4c40 2024-05-09T06:37:12.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:12.854+0000 7f1e4bfff640 1 -- 172.21.15.3:0/3060070691 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e58017070 con 0x7f1e640d4c40 2024-05-09T06:37:12.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:12.854+0000 7f1e69dc1640 1 -- 172.21.15.3:0/3060070691 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e6415fbe0 con 0x7f1e640d4c40 2024-05-09T06:37:12.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:12.854+0000 7f1e4bfff640 1 -- 172.21.15.3:0/3060070691 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e58011bb0 con 0x7f1e640d4c40 2024-05-09T06:37:12.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:12.854+0000 7f1e69dc1640 1 -- 172.21.15.3:0/3060070691 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e641601a0 con 0x7f1e640d4c40 2024-05-09T06:37:12.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:12.855+0000 7f1e69dc1640 1 -- 172.21.15.3:0/3060070691 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e64069550 con 0x7f1e640d4c40 2024-05-09T06:37:12.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:12.855+0000 7f1e4bfff640 1 -- 172.21.15.3:0/3060070691 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 7) ==== 90651+0+0 (secure 0 0 0) 0x7f1e5801c460 con 0x7f1e640d4c40 2024-05-09T06:37:12.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:12.855+0000 7f1e4bfff640 1 --2- 172.21.15.3:0/3060070691 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f1e5006bfe0 0x7f1e5006e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:12.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:12.856+0000 7f1e4bfff640 1 -- 172.21.15.3:0/3060070691 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f1e5808afb0 con 0x7f1e640d4c40 2024-05-09T06:37:12.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:12.856+0000 7f1e637fe640 1 --2- 172.21.15.3:0/3060070691 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f1e5006bfe0 0x7f1e5006e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:12.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:12.856+0000 7f1e637fe640 1 --2- 172.21.15.3:0/3060070691 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f1e5006bfe0 0x7f1e5006e4a0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f1e54006fd0 tx=0x7f1e54008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:12.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:12.860+0000 7f1e4bfff640 1 -- 172.21.15.3:0/3060070691 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f1e58059770 con 0x7f1e640d4c40 2024-05-09T06:37:13.036 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.055+0000 7f1e69dc1640 1 -- 172.21.15.3:0/3060070691 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f1e640b9e70 con 0x7f1e640d4c40 2024-05-09T06:37:13.036 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.055+0000 7f1e4bfff640 1 -- 172.21.15.3:0/3060070691 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v33) ==== 80+0+0 (secure 0 0 0) 0x7f1e58059070 con 0x7f1e640d4c40 2024-05-09T06:37:13.036 INFO:teuthology.orchestra.run.smithi003.stderr:listed 0 entries 2024-05-09T06:37:13.038 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.057+0000 7f1e69dc1640 1 -- 172.21.15.3:0/3060070691 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f1e5006bfe0 msgr2=0x7f1e5006e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:13.038 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.057+0000 7f1e69dc1640 1 --2- 172.21.15.3:0/3060070691 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f1e5006bfe0 0x7f1e5006e4a0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f1e54006fd0 tx=0x7f1e54008040 comp rx=0 tx=0).stop 2024-05-09T06:37:13.038 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.057+0000 7f1e69dc1640 1 -- 172.21.15.3:0/3060070691 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1e640d4c40 msgr2=0x7f1e640ce060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:13.038 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.057+0000 7f1e69dc1640 1 --2- 172.21.15.3:0/3060070691 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1e640d4c40 0x7f1e640ce060 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f1e5800c430 tx=0x7f1e5800c8f0 comp rx=0 tx=0).stop 2024-05-09T06:37:13.038 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.057+0000 7f1e69dc1640 1 -- 172.21.15.3:0/3060070691 shutdown_connections 2024-05-09T06:37:13.039 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.057+0000 7f1e69dc1640 1 --2- 172.21.15.3:0/3060070691 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f1e5006bfe0 0x7f1e5006e4a0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:13.039 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.057+0000 7f1e69dc1640 1 --2- 172.21.15.3:0/3060070691 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1e6415e5c0 0x7f1e64160f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:13.039 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.057+0000 7f1e69dc1640 1 --2- 172.21.15.3:0/3060070691 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1e640d4c40 0x7f1e640ce060 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:13.039 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.057+0000 7f1e69dc1640 1 --2- 172.21.15.3:0/3060070691 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1e640d1300 0x7f1e640ca960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:13.039 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.057+0000 7f1e69dc1640 1 -- 172.21.15.3:0/3060070691 >> 172.21.15.3:0/3060070691 conn(0x7f1e6409d500 msgr2=0x7f1e640b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:13.039 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.057+0000 7f1e69dc1640 1 -- 172.21.15.3:0/3060070691 shutdown_connections 2024-05-09T06:37:13.039 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.057+0000 7f1e69dc1640 1 -- 172.21.15.3:0/3060070691 wait complete. 2024-05-09T06:37:13.049 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-05-09T06:37:13.049 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:37:13.049 DEBUG:teuthology.orchestra.run.smithi003:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-05-09T06:37:13.078 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi003.front.sepia.ceph.com 2024-05-09T06:37:13.078 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:37:13.079 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-09T06:37:13.135 INFO:tasks.cephfs.mount:Terminating background process 2024-05-09T06:37:13.135 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-05-09T06:37:13.135 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T06:37:13.155 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-05-09T06:37:13.155 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T06:37:13.214 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T06:37:13.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.339+0000 7f9fe344d640 1 Processor -- start 2024-05-09T06:37:13.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.339+0000 7f9fe344d640 1 -- start start 2024-05-09T06:37:13.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.340+0000 7f9fe344d640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f9fdc0ccf80 0x7f9fdc0cf720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:13.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.340+0000 7f9fe344d640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9fdc13e4a0 con 0x7f9fdc13e0c0 2024-05-09T06:37:13.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.340+0000 7f9fe344d640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9fdc0ce960 con 0x7f9fdc1408f0 2024-05-09T06:37:13.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.340+0000 7f9fe344d640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f9fdc141410 con 0x7f9fdc0ccf80 2024-05-09T06:37:13.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.340+0000 7f9fe11e9640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f9fdc13e0c0 0x7f9fdc0cc870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:45898/0 (socket says 172.21.15.3:45898) 2024-05-09T06:37:13.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.340+0000 7f9fe11e9640 1 -- 172.21.15.3:0/3443928701 learned_addr learned my addr 172.21.15.3:0/3443928701 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-05-09T06:37:13.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.340+0000 7f9fd3fff640 1 -- 172.21.15.3:0/3443928701 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3821543280 0 0) 0x7f9fd8001110 con 0x7f9fdc13e0c0 2024-05-09T06:37:13.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.340+0000 7f9fd3fff640 1 -- 172.21.15.3:0/3443928701 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9fc8003520 con 0x7f9fdc13e0c0 2024-05-09T06:37:13.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.341+0000 7f9fe09e8640 1 --2- 172.21.15.3:0/3443928701 >> v2:172.21.15.190:3300/0 conn(0x7f9fdc0ccf80 0x7f9fdc0cf720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:13.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.341+0000 7f9fd3fff640 1 -- 172.21.15.3:0/3443928701 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2286817903 0 0) 0x7f9fc8003520 con 0x7f9fdc13e0c0 2024-05-09T06:37:13.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.341+0000 7f9fd3fff640 1 -- 172.21.15.3:0/3443928701 >> v2:172.21.15.190:3300/0 conn(0x7f9fdc0ccf80 msgr2=0x7f9fdc0cf720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:37:13.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.341+0000 7f9fd3fff640 1 --2- 172.21.15.3:0/3443928701 >> v2:172.21.15.190:3300/0 conn(0x7f9fdc0ccf80 0x7f9fdc0cf720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:13.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.341+0000 7f9fd3fff640 1 -- 172.21.15.3:0/3443928701 >> v1:172.21.15.190:6789/0 conn(0x7f9fdc1408f0 legacy=0x7f9fdc140d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:13.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.341+0000 7f9fd3fff640 1 -- 172.21.15.3:0/3443928701 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fdc0cec40 con 0x7f9fdc13e0c0 2024-05-09T06:37:13.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.341+0000 7f9fd3fff640 1 -- 172.21.15.3:0/3443928701 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 563312031 0 0) 0x7f9fd8001840 con 0x7f9fdc13e0c0 2024-05-09T06:37:13.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.341+0000 7f9fe09e8640 1 --2- 172.21.15.3:0/3443928701 >> v2:172.21.15.190:3300/0 conn(0x7f9fdc0ccf80 0x7f9fdc0cf720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:37:13.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.341+0000 7f9fd3fff640 1 -- 172.21.15.3:0/3443928701 >> v1:172.21.15.3:6789/0 conn(0x7f9fdc13e0c0 legacy=0x7f9fdc0cc870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:13.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.341+0000 7f9fd3fff640 1 --2- 172.21.15.3:0/3443928701 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9fc80039b0 0x7f9fc8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:13.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.341+0000 7f9fd3fff640 1 --2- 172.21.15.3:0/3443928701 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9fc8004800 0x7f9fc8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:13.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.341+0000 7f9fd3fff640 1 --2- 172.21.15.3:0/3443928701 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9fc80051c0 0x7f9fc800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:13.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.341+0000 7f9fd3fff640 1 -- 172.21.15.3:0/3443928701 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f9fdc141410 con 0x7f9fc80051c0 2024-05-09T06:37:13.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.341+0000 7f9fd3fff640 1 -- 172.21.15.3:0/3443928701 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f9fc800c110 con 0x7f9fc8004800 2024-05-09T06:37:13.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.341+0000 7f9fd3fff640 1 -- 172.21.15.3:0/3443928701 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f9fc800c310 con 0x7f9fc80039b0 2024-05-09T06:37:13.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.341+0000 7f9fe19ea640 1 --2- 172.21.15.3:0/3443928701 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9fc80051c0 0x7f9fc800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:13.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.341+0000 7f9fe09e8640 1 --2- 172.21.15.3:0/3443928701 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9fc8004800 0x7f9fc8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:13.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.341+0000 7f9fe11e9640 1 --2- 172.21.15.3:0/3443928701 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9fc80039b0 0x7f9fc8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:13.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.341+0000 7f9fe19ea640 1 -- 172.21.15.3:0/3443928701 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9fc80039b0 msgr2=0x7f9fc8003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:37:13.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.341+0000 7f9fe19ea640 1 --2- 172.21.15.3:0/3443928701 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9fc80039b0 0x7f9fc8003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:13.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.341+0000 7f9fe19ea640 1 -- 172.21.15.3:0/3443928701 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9fc8004800 msgr2=0x7f9fc8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:37:13.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.341+0000 7f9fe19ea640 1 --2- 172.21.15.3:0/3443928701 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9fc8004800 0x7f9fc8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:13.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.341+0000 7f9fe19ea640 1 -- 172.21.15.3:0/3443928701 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fc8003520 con 0x7f9fc80051c0 2024-05-09T06:37:13.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.341+0000 7f9fe11e9640 1 --2- 172.21.15.3:0/3443928701 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9fc80039b0 0x7f9fc8003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:37:13.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.342+0000 7f9fe19ea640 1 -- 172.21.15.3:0/3443928701 reap_dead start 2024-05-09T06:37:13.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.342+0000 7f9fe19ea640 1 --2- 172.21.15.3:0/3443928701 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9fc80051c0 0x7f9fc800b9c0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f9fd4001e50 tx=0x7f9fc8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f26d19e867cf4183 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:13.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.342+0000 7f9fd3fff640 1 -- 172.21.15.3:0/3443928701 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fd4001a70 con 0x7f9fc80051c0 2024-05-09T06:37:13.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.342+0000 7f9fd3fff640 1 -- 172.21.15.3:0/3443928701 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9fd4003970 con 0x7f9fc80051c0 2024-05-09T06:37:13.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.342+0000 7f9fd3fff640 1 -- 172.21.15.3:0/3443928701 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fd4003d60 con 0x7f9fc80051c0 2024-05-09T06:37:13.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.343+0000 7f9fe344d640 1 -- 172.21.15.3:0/3443928701 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9fc80051c0 msgr2=0x7f9fc800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:13.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.343+0000 7f9fe344d640 1 --2- 172.21.15.3:0/3443928701 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9fc80051c0 0x7f9fc800b9c0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f9fd4001e50 tx=0x7f9fc8005830 comp rx=0 tx=0).stop 2024-05-09T06:37:13.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.343+0000 7f9fe344d640 1 -- 172.21.15.3:0/3443928701 shutdown_connections 2024-05-09T06:37:13.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.343+0000 7f9fe344d640 1 --2- 172.21.15.3:0/3443928701 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9fc80051c0 0x7f9fc800b9c0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:13.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.343+0000 7f9fe344d640 1 -- 172.21.15.3:0/3443928701 >> 172.21.15.3:0/3443928701 conn(0x7f9fdc06b430 msgr2=0x7f9fdc06b810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:13.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.343+0000 7f9fe344d640 1 -- 172.21.15.3:0/3443928701 shutdown_connections 2024-05-09T06:37:13.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.343+0000 7f9fe344d640 1 -- 172.21.15.3:0/3443928701 wait complete. 2024-05-09T06:37:13.330 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.344+0000 7f9fe344d640 1 Processor -- start 2024-05-09T06:37:13.330 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.344+0000 7f9fe344d640 1 -- start start 2024-05-09T06:37:13.330 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.345+0000 7f9fe344d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9fc80051c0 0x7f9fdc156240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:13.330 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.345+0000 7f9fe344d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9fdc13f9c0 0x7f9fdc1532e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:13.330 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.345+0000 7f9fe344d640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9fdc13eee0 0x7f9fdc1e79f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:13.330 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.345+0000 7f9fe344d640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f9fdc141660 con 0x7f9fdc13f9c0 2024-05-09T06:37:13.330 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.345+0000 7f9fe344d640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f9fdc0cf200 con 0x7f9fc80051c0 2024-05-09T06:37:13.330 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.345+0000 7f9fe344d640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f9fdc0cf720 con 0x7f9fdc13eee0 2024-05-09T06:37:13.330 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.345+0000 7f9fe09e8640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9fdc13f9c0 0x7f9fdc1532e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:13.330 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.345+0000 7f9fe09e8640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9fdc13f9c0 0x7f9fdc1532e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50936/0 (socket says 172.21.15.3:50936) 2024-05-09T06:37:13.330 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.345+0000 7f9fe09e8640 1 -- 172.21.15.3:0/1435728949 learned_addr learned my addr 172.21.15.3:0/1435728949 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:37:13.331 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.345+0000 7f9fe19ea640 1 --2- 172.21.15.3:0/1435728949 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9fdc13eee0 0x7f9fdc1e79f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:13.331 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.345+0000 7f9fe11e9640 1 --2- 172.21.15.3:0/1435728949 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9fc80051c0 0x7f9fdc156240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:13.331 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.345+0000 7f9fe09e8640 1 -- 172.21.15.3:0/1435728949 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9fdc13eee0 msgr2=0x7f9fdc1e79f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:13.331 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.345+0000 7f9fe09e8640 1 --2- 172.21.15.3:0/1435728949 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9fdc13eee0 0x7f9fdc1e79f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:13.331 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.345+0000 7f9fe09e8640 1 -- 172.21.15.3:0/1435728949 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9fc80051c0 msgr2=0x7f9fdc156240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:13.331 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.345+0000 7f9fe09e8640 1 --2- 172.21.15.3:0/1435728949 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9fc80051c0 0x7f9fdc156240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:13.331 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.345+0000 7f9fe09e8640 1 -- 172.21.15.3:0/1435728949 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fdc13e720 con 0x7f9fdc13f9c0 2024-05-09T06:37:13.331 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.345+0000 7f9fe11e9640 1 --2- 172.21.15.3:0/1435728949 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9fc80051c0 0x7f9fdc156240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:37:13.331 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.346+0000 7f9fe09e8640 1 --2- 172.21.15.3:0/1435728949 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9fdc13f9c0 0x7f9fdc1532e0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f9fcc011590 tx=0x7f9fcc011a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:13.331 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.346+0000 7f9fd1ffb640 1 -- 172.21.15.3:0/1435728949 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fcc019070 con 0x7f9fdc13f9c0 2024-05-09T06:37:13.331 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.346+0000 7f9fd1ffb640 1 -- 172.21.15.3:0/1435728949 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9fcc004220 con 0x7f9fdc13f9c0 2024-05-09T06:37:13.331 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.346+0000 7f9fe344d640 1 -- 172.21.15.3:0/1435728949 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9fdc140050 con 0x7f9fdc13f9c0 2024-05-09T06:37:13.331 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.346+0000 7f9fd1ffb640 1 -- 172.21.15.3:0/1435728949 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fcc00f040 con 0x7f9fdc13f9c0 2024-05-09T06:37:13.332 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.346+0000 7f9fe344d640 1 -- 172.21.15.3:0/1435728949 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9fdc140610 con 0x7f9fdc13f9c0 2024-05-09T06:37:13.332 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.347+0000 7f9fe344d640 1 -- 172.21.15.3:0/1435728949 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9fdc04f660 con 0x7f9fdc13f9c0 2024-05-09T06:37:13.332 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.347+0000 7f9fd1ffb640 1 -- 172.21.15.3:0/1435728949 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 7) ==== 90651+0+0 (secure 0 0 0) 0x7f9fcc015670 con 0x7f9fdc13f9c0 2024-05-09T06:37:13.332 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.348+0000 7f9fd1ffb640 1 --2- 172.21.15.3:0/1435728949 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f9fc006bfe0 0x7f9fc006e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:13.332 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.348+0000 7f9fd1ffb640 1 -- 172.21.15.3:0/1435728949 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f9fcc08b3d0 con 0x7f9fdc13f9c0 2024-05-09T06:37:13.332 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.348+0000 7f9fe11e9640 1 --2- 172.21.15.3:0/1435728949 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f9fc006bfe0 0x7f9fc006e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:13.332 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.349+0000 7f9fe11e9640 1 --2- 172.21.15.3:0/1435728949 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f9fc006bfe0 0x7f9fc006e4a0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f9fd80012f0 tx=0x7f9fd8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:13.333 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.352+0000 7f9fd1ffb640 1 -- 172.21.15.3:0/1435728949 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f9fcc059390 con 0x7f9fdc13f9c0 2024-05-09T06:37:13.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.536+0000 7f9fe344d640 1 -- 172.21.15.3:0/1435728949 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9fdc158310 con 0x7f9fdc13f9c0 2024-05-09T06:37:13.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.537+0000 7f9fd1ffb640 1 -- 172.21.15.3:0/1435728949 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) ==== 74+0+16585 (secure 0 0 0) 0x7f9fcc05d690 con 0x7f9fdc13f9c0 2024-05-09T06:37:13.518 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-09T06:37:13.518 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":33,"fsid":"16ed737d-2665-431a-8ac6-ef242be3975a","created":"2024-05-09T06:35:42.541368+0000","modified":"2024-05-09T06:36:59.564366+0000","last_up_change":"2024-05-09T06:35:53.778274+0000","last_in_change":"2024-05-09T06:35:44.780586+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T06:35:54.819999+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-05-09T06:36:56.494700+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-05-09T06:36:56.900006+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fae3a8db-4826-4f3c-b33a-12c09bb5df6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":1897849827},{"type":"v1","addr":"172.21.15.3:6817","nonce":1897849827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":1897849827},{"type":"v1","addr":"172.21.15.3:6819","nonce":1897849827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":1897849827},{"type":"v1","addr":"172.21.15.3:6823","nonce":1897849827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":1897849827},{"type":"v1","addr":"172.21.15.3:6821","nonce":1897849827}]},"public_addr":"172.21.15.3:6817/1897849827","cluster_addr":"172.21.15.3:6819/1897849827","heartbeat_back_addr":"172.21.15.3:6823/1897849827","heartbeat_front_addr":"172.21.15.3:6821/1897849827","state":["exists","up"]},{"osd":1,"uuid":"646e882d-4ce4-4720-8c84-f71df563a35e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":926805498},{"type":"v1","addr":"172.21.15.3:6801","nonce":926805498}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":926805498},{"type":"v1","addr":"172.21.15.3:6803","nonce":926805498}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":926805498},{"type":"v1","addr":"172.21.15.3:6807","nonce":926805498}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":926805498},{"type":"v1","addr":"172.21.15.3:6805","nonce":926805498}]},"public_addr":"172.21.15.3:6801/926805498","cluster_addr":"172.21.15.3:6803/926805498","heartbeat_back_addr":"172.21.15.3:6807/926805498","heartbeat_front_addr":"172.21.15.3:6805/926805498","state":["exists","up"]},{"osd":2,"uuid":"1abbadb5-a570-4359-9643-acff3ede35a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":801295451},{"type":"v1","addr":"172.21.15.3:6809","nonce":801295451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":801295451},{"type":"v1","addr":"172.21.15.3:6811","nonce":801295451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":801295451},{"type":"v1","addr":"172.21.15.3:6815","nonce":801295451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":801295451},{"type":"v1","addr":"172.21.15.3:6813","nonce":801295451}]},"public_addr":"172.21.15.3:6809/801295451","cluster_addr":"172.21.15.3:6811/801295451","heartbeat_back_addr":"172.21.15.3:6815/801295451","heartbeat_front_addr":"172.21.15.3:6813/801295451","state":["exists","up"]},{"osd":3,"uuid":"62ac4dc1-2371-4dd2-acd3-e2dbb11aedf5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":1903389857},{"type":"v1","addr":"172.21.15.3:6825","nonce":1903389857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":1903389857},{"type":"v1","addr":"172.21.15.3:6827","nonce":1903389857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":1903389857},{"type":"v1","addr":"172.21.15.3:6831","nonce":1903389857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":1903389857},{"type":"v1","addr":"172.21.15.3:6829","nonce":1903389857}]},"public_addr":"172.21.15.3:6825/1903389857","cluster_addr":"172.21.15.3:6827/1903389857","heartbeat_back_addr":"172.21.15.3:6831/1903389857","heartbeat_front_addr":"172.21.15.3:6829/1903389857","state":["exists","up"]},{"osd":4,"uuid":"ae808398-8326-4d7c-9319-a2a0eafffb59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":2143624576},{"type":"v1","addr":"172.21.15.190:6801","nonce":2143624576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":2143624576},{"type":"v1","addr":"172.21.15.190:6803","nonce":2143624576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":2143624576},{"type":"v1","addr":"172.21.15.190:6807","nonce":2143624576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":2143624576},{"type":"v1","addr":"172.21.15.190:6805","nonce":2143624576}]},"public_addr":"172.21.15.190:6801/2143624576","cluster_addr":"172.21.15.190:6803/2143624576","heartbeat_back_addr":"172.21.15.190:6807/2143624576","heartbeat_front_addr":"172.21.15.190:6805/2143624576","state":["exists","up"]},{"osd":5,"uuid":"918adf50-c440-48ac-9a8d-4837562d7275","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":1766923126},{"type":"v1","addr":"172.21.15.190:6814","nonce":1766923126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":1766923126},{"type":"v1","addr":"172.21.15.190:6818","nonce":1766923126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":1766923126},{"type":"v1","addr":"172.21.15.190:6823","nonce":1766923126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":1766923126},{"type":"v1","addr":"172.21.15.190:6821","nonce":1766923126}]},"public_addr":"172.21.15.190:6814/1766923126","cluster_addr":"172.21.15.190:6818/1766923126","heartbeat_back_addr":"172.21.15.190:6823/1766923126","heartbeat_front_addr":"172.21.15.190:6821/1766923126","state":["exists","up"]},{"osd":6,"uuid":"ac2882bc-2a7c-463c-8f6c-76ff49828adc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":342414248},{"type":"v1","addr":"172.21.15.190:6825","nonce":342414248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":342414248},{"type":"v1","addr":"172.21.15.190:6827","nonce":342414248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":342414248},{"type":"v1","addr":"172.21.15.190:6831","nonce":342414248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":342414248},{"type":"v1","addr":"172.21.15.190:6829","nonce":342414248}]},"public_addr":"172.21.15.190:6825/342414248","cluster_addr":"172.21.15.190:6827/342414248","heartbeat_back_addr":"172.21.15.190:6831/342414248","heartbeat_front_addr":"172.21.15.190:6829/342414248","state":["exists","up"]},{"osd":7,"uuid":"dd4e767a-4730-4c70-8bd5-3f088140a5a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":537849891},{"type":"v1","addr":"172.21.15.190:6809","nonce":537849891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":537849891},{"type":"v1","addr":"172.21.15.190:6811","nonce":537849891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6817","nonce":537849891},{"type":"v1","addr":"172.21.15.190:6819","nonce":537849891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6813","nonce":537849891},{"type":"v1","addr":"172.21.15.190:6815","nonce":537849891}]},"public_addr":"172.21.15.190:6809/537849891","cluster_addr":"172.21.15.190:6811/537849891","heartbeat_back_addr":"172.21.15.190:6819/537849891","heartbeat_front_addr":"172.21.15.190:6815/537849891","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.889784+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.974002+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.973549+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.955396+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.928471+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.902418+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.958696+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.926474+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T06:37:13.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.539+0000 7f9fe344d640 1 -- 172.21.15.3:0/1435728949 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f9fc006bfe0 msgr2=0x7f9fc006e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:13.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.539+0000 7f9fe344d640 1 --2- 172.21.15.3:0/1435728949 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f9fc006bfe0 0x7f9fc006e4a0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f9fd80012f0 tx=0x7f9fd8009040 comp rx=0 tx=0).stop 2024-05-09T06:37:13.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.539+0000 7f9fe344d640 1 -- 172.21.15.3:0/1435728949 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9fdc13f9c0 msgr2=0x7f9fdc1532e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:13.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.539+0000 7f9fe344d640 1 --2- 172.21.15.3:0/1435728949 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9fdc13f9c0 0x7f9fdc1532e0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f9fcc011590 tx=0x7f9fcc011a50 comp rx=0 tx=0).stop 2024-05-09T06:37:13.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.539+0000 7f9fe344d640 1 -- 172.21.15.3:0/1435728949 shutdown_connections 2024-05-09T06:37:13.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.539+0000 7f9fe344d640 1 --2- 172.21.15.3:0/1435728949 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f9fc006bfe0 0x7f9fc006e4a0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:13.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.539+0000 7f9fe344d640 1 --2- 172.21.15.3:0/1435728949 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9fdc13eee0 0x7f9fdc1e79f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:13.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.539+0000 7f9fe344d640 1 --2- 172.21.15.3:0/1435728949 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9fdc13f9c0 0x7f9fdc1532e0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:13.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.539+0000 7f9fe344d640 1 --2- 172.21.15.3:0/1435728949 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9fc80051c0 0x7f9fdc156240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:13.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.539+0000 7f9fe344d640 1 -- 172.21.15.3:0/1435728949 >> 172.21.15.3:0/1435728949 conn(0x7f9fdc06b430 msgr2=0x7f9fdc0b3eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:13.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.540+0000 7f9fe344d640 1 -- 172.21.15.3:0/1435728949 shutdown_connections 2024-05-09T06:37:13.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.540+0000 7f9fe344d640 1 -- 172.21.15.3:0/1435728949 wait complete. 2024-05-09T06:37:13.531 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-09T06:35:54.819999+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-09T06:36:56.494700+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '32', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2024-05-09T06:36:56.900006+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '33', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-09T06:37:13.532 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-09T06:37:13.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.660+0000 7f0df1163640 1 Processor -- start 2024-05-09T06:37:13.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.660+0000 7f0df1163640 1 -- start start 2024-05-09T06:37:13.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.661+0000 7f0df1163640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f0dec15e5c0 0x7f0dec0d2800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:13.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.661+0000 7f0df1163640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f0dec0d1300 0x7f0dec0b47e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:13.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.661+0000 7f0df1163640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0dec15e9a0 con 0x7f0dec0d4c40 2024-05-09T06:37:13.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.661+0000 7f0df1163640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f0dec0d50d0 con 0x7f0dec0d1300 2024-05-09T06:37:13.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.661+0000 7f0df1163640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f0dec0d31c0 con 0x7f0dec15e5c0 2024-05-09T06:37:13.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.661+0000 7f0deb59e640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f0dec0d1300 0x7f0dec0b47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:13.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.661+0000 7f0deb59e640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f0dec0d1300 0x7f0dec0b47e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50938/0 (socket says 172.21.15.3:50938) 2024-05-09T06:37:13.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.661+0000 7f0deb59e640 1 -- 172.21.15.3:0/155845150 learned_addr learned my addr 172.21.15.3:0/155845150 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:37:13.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.662+0000 7f0dead9d640 1 --2- 172.21.15.3:0/155845150 >> v2:172.21.15.190:3300/0 conn(0x7f0dec15e5c0 0x7f0dec0d2800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:13.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.662+0000 7f0deb59e640 1 -- 172.21.15.3:0/155845150 >> v2:172.21.15.190:3300/0 conn(0x7f0dec15e5c0 msgr2=0x7f0dec0d2800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:37:13.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.662+0000 7f0deb59e640 1 --2- 172.21.15.3:0/155845150 >> v2:172.21.15.190:3300/0 conn(0x7f0dec15e5c0 0x7f0dec0d2800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:13.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.662+0000 7f0deb59e640 1 -- 172.21.15.3:0/155845150 >> v1:172.21.15.190:6789/0 conn(0x7f0dec0d4c40 legacy=0x7f0dec0d0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:13.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.662+0000 7f0deb59e640 1 -- 172.21.15.3:0/155845150 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0dec0d2ee0 con 0x7f0dec0d1300 2024-05-09T06:37:13.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.662+0000 7f0deb59e640 1 --2- 172.21.15.3:0/155845150 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0dec0d1300 0x7f0dec0b47e0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f0ddc009e50 tx=0x7f0ddc02a760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cafa65314e2cb3db server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:13.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.663+0000 7f0de9d9b640 1 -- 172.21.15.3:0/155845150 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ddc036070 con 0x7f0dec0d1300 2024-05-09T06:37:13.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.663+0000 7f0de9d9b640 1 -- 172.21.15.3:0/155845150 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ddc031600 con 0x7f0dec0d1300 2024-05-09T06:37:13.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.663+0000 7f0de9d9b640 1 -- 172.21.15.3:0/155845150 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ddc0319d0 con 0x7f0dec0d1300 2024-05-09T06:37:13.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.663+0000 7f0df1163640 1 -- 172.21.15.3:0/155845150 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0dec0d1300 msgr2=0x7f0dec0b47e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:13.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.663+0000 7f0df1163640 1 --2- 172.21.15.3:0/155845150 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0dec0d1300 0x7f0dec0b47e0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f0ddc009e50 tx=0x7f0ddc02a760 comp rx=0 tx=0).stop 2024-05-09T06:37:13.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.663+0000 7f0df1163640 1 -- 172.21.15.3:0/155845150 shutdown_connections 2024-05-09T06:37:13.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.663+0000 7f0df1163640 1 --2- 172.21.15.3:0/155845150 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0dec0d1300 0x7f0dec0b47e0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:13.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.663+0000 7f0df1163640 1 --2- 172.21.15.3:0/155845150 >> v2:172.21.15.190:3300/0 conn(0x7f0dec15e5c0 0x7f0dec0d2800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:13.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.663+0000 7f0df1163640 1 -- 172.21.15.3:0/155845150 >> 172.21.15.3:0/155845150 conn(0x7f0dec09d500 msgr2=0x7f0dec0d4220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:13.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.663+0000 7f0df1163640 1 -- 172.21.15.3:0/155845150 shutdown_connections 2024-05-09T06:37:13.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.663+0000 7f0df1163640 1 -- 172.21.15.3:0/155845150 wait complete. 2024-05-09T06:37:13.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.665+0000 7f0df1163640 1 Processor -- start 2024-05-09T06:37:13.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.665+0000 7f0df1163640 1 -- start start 2024-05-09T06:37:13.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.666+0000 7f0df1163640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0dec0d1300 0x7f0dec0a1920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:13.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.666+0000 7f0df1163640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0dec0d4c40 0x7f0dec1f7c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:13.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.666+0000 7f0df1163640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0dec15e5c0 0x7f0dec06cff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:13.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.666+0000 7f0df1163640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f0dec0b42c0 con 0x7f0dec15e5c0 2024-05-09T06:37:13.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.666+0000 7f0df1163640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f0dec0b3fe0 con 0x7f0dec0d1300 2024-05-09T06:37:13.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.666+0000 7f0df1163640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f0dec0b4d20 con 0x7f0dec0d4c40 2024-05-09T06:37:13.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.666+0000 7f0deb59e640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0dec15e5c0 0x7f0dec06cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:13.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.666+0000 7f0deb59e640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0dec15e5c0 0x7f0dec06cff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50952/0 (socket says 172.21.15.3:50952) 2024-05-09T06:37:13.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.666+0000 7f0deb59e640 1 -- 172.21.15.3:0/2369986667 learned_addr learned my addr 172.21.15.3:0/2369986667 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:37:13.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.666+0000 7f0dead9d640 1 --2- 172.21.15.3:0/2369986667 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0dec0d1300 0x7f0dec0a1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:13.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.666+0000 7f0dea59c640 1 --2- 172.21.15.3:0/2369986667 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0dec0d4c40 0x7f0dec1f7c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:13.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.666+0000 7f0deb59e640 1 -- 172.21.15.3:0/2369986667 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0dec0d4c40 msgr2=0x7f0dec1f7c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:13.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.666+0000 7f0deb59e640 1 --2- 172.21.15.3:0/2369986667 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0dec0d4c40 0x7f0dec1f7c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:13.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.666+0000 7f0deb59e640 1 -- 172.21.15.3:0/2369986667 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0dec0d1300 msgr2=0x7f0dec0a1920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:13.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.666+0000 7f0deb59e640 1 --2- 172.21.15.3:0/2369986667 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0dec0d1300 0x7f0dec0a1920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:13.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.666+0000 7f0deb59e640 1 -- 172.21.15.3:0/2369986667 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0dec0a1e60 con 0x7f0dec15e5c0 2024-05-09T06:37:13.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.666+0000 7f0dea59c640 1 --2- 172.21.15.3:0/2369986667 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0dec0d4c40 0x7f0dec1f7c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:37:13.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.666+0000 7f0dead9d640 1 --2- 172.21.15.3:0/2369986667 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0dec0d1300 0x7f0dec0a1920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:37:13.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.666+0000 7f0deb59e640 1 --2- 172.21.15.3:0/2369986667 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0dec15e5c0 0x7f0dec06cff0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f0ddc034040 tx=0x7f0ddc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:13.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.666+0000 7f0de37fe640 1 -- 172.21.15.3:0/2369986667 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ddc036050 con 0x7f0dec15e5c0 2024-05-09T06:37:13.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.666+0000 7f0de37fe640 1 -- 172.21.15.3:0/2369986667 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ddc042070 con 0x7f0dec15e5c0 2024-05-09T06:37:13.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.666+0000 7f0df1163640 1 -- 172.21.15.3:0/2369986667 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0dec1f81b0 con 0x7f0dec15e5c0 2024-05-09T06:37:13.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.667+0000 7f0de37fe640 1 -- 172.21.15.3:0/2369986667 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ddc009510 con 0x7f0dec15e5c0 2024-05-09T06:37:13.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.667+0000 7f0df1163640 1 -- 172.21.15.3:0/2369986667 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0dec1f4160 con 0x7f0dec15e5c0 2024-05-09T06:37:13.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.667+0000 7f0df1163640 1 -- 172.21.15.3:0/2369986667 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0dec069550 con 0x7f0dec15e5c0 2024-05-09T06:37:13.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.668+0000 7f0de37fe640 1 -- 172.21.15.3:0/2369986667 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 7) ==== 90651+0+0 (secure 0 0 0) 0x7f0ddc007bd0 con 0x7f0dec15e5c0 2024-05-09T06:37:13.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.668+0000 7f0de37fe640 1 --2- 172.21.15.3:0/2369986667 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f0dcc06c010 0x7f0dcc06e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:13.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.669+0000 7f0de37fe640 1 -- 172.21.15.3:0/2369986667 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f0ddc0abff0 con 0x7f0dec15e5c0 2024-05-09T06:37:13.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.669+0000 7f0dead9d640 1 --2- 172.21.15.3:0/2369986667 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f0dcc06c010 0x7f0dcc06e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:13.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.669+0000 7f0dead9d640 1 --2- 172.21.15.3:0/2369986667 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f0dcc06c010 0x7f0dcc06e4d0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f0de400b9f0 tx=0x7f0de4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:13.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.673+0000 7f0de37fe640 1 -- 172.21.15.3:0/2369986667 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f0ddc045360 con 0x7f0dec15e5c0 2024-05-09T06:37:13.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.855+0000 7f0df1163640 1 -- 172.21.15.3:0/2369986667 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f0dec074fb0 con 0x7f0dec15e5c0 2024-05-09T06:37:13.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.855+0000 7f0de37fe640 1 -- 172.21.15.3:0/2369986667 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v33) ==== 93+0+10 (secure 0 0 0) 0x7f0ddc07a620 con 0x7f0dec15e5c0 2024-05-09T06:37:13.836 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 1 2024-05-09T06:37:13.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.857+0000 7f0df1163640 1 -- 172.21.15.3:0/2369986667 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f0dcc06c010 msgr2=0x7f0dcc06e4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:13.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.857+0000 7f0df1163640 1 --2- 172.21.15.3:0/2369986667 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f0dcc06c010 0x7f0dcc06e4d0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f0de400b9f0 tx=0x7f0de4008040 comp rx=0 tx=0).stop 2024-05-09T06:37:13.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.857+0000 7f0df1163640 1 -- 172.21.15.3:0/2369986667 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0dec15e5c0 msgr2=0x7f0dec06cff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:13.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.857+0000 7f0df1163640 1 --2- 172.21.15.3:0/2369986667 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0dec15e5c0 0x7f0dec06cff0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f0ddc034040 tx=0x7f0ddc004a40 comp rx=0 tx=0).stop 2024-05-09T06:37:13.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.857+0000 7f0df1163640 1 -- 172.21.15.3:0/2369986667 shutdown_connections 2024-05-09T06:37:13.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.857+0000 7f0df1163640 1 --2- 172.21.15.3:0/2369986667 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f0dcc06c010 0x7f0dcc06e4d0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:13.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.857+0000 7f0df1163640 1 --2- 172.21.15.3:0/2369986667 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0dec15e5c0 0x7f0dec06cff0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:13.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.857+0000 7f0df1163640 1 --2- 172.21.15.3:0/2369986667 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0dec0d4c40 0x7f0dec1f7c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:13.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.857+0000 7f0df1163640 1 --2- 172.21.15.3:0/2369986667 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0dec0d1300 0x7f0dec0a1920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:13.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.857+0000 7f0df1163640 1 -- 172.21.15.3:0/2369986667 >> 172.21.15.3:0/2369986667 conn(0x7f0dec09d500 msgr2=0x7f0dec0cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:13.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.857+0000 7f0df1163640 1 -- 172.21.15.3:0/2369986667 shutdown_connections 2024-05-09T06:37:13.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.857+0000 7f0df1163640 1 -- 172.21.15.3:0/2369986667 wait complete. 2024-05-09T06:37:13.849 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-09T06:37:13.959 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.978+0000 7fa33c36a640 1 Processor -- start 2024-05-09T06:37:13.959 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.978+0000 7fa33c36a640 1 -- start start 2024-05-09T06:37:13.960 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.979+0000 7fa33c36a640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa3340b4e30 0x7fa33406e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:13.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.979+0000 7fa33c36a640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa33406e790 0x7fa33415c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:13.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.979+0000 7fa33c36a640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa3340b40f0 con 0x7fa33415c0f0 2024-05-09T06:37:13.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.979+0000 7fa33c36a640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fa33406ec00 con 0x7fa3340b4e30 2024-05-09T06:37:13.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.979+0000 7fa33c36a640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fa3340b3050 con 0x7fa33406e790 2024-05-09T06:37:13.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.980+0000 7fa339905640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa3340b4e30 0x7fa33406e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:13.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.980+0000 7fa339905640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa3340b4e30 0x7fa33406e250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50954/0 (socket says 172.21.15.3:50954) 2024-05-09T06:37:13.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.980+0000 7fa339905640 1 -- 172.21.15.3:0/32108552 learned_addr learned my addr 172.21.15.3:0/32108552 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:37:13.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.980+0000 7fa33a907640 1 --2- 172.21.15.3:0/32108552 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa33406e790 0x7fa33415c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:13.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.980+0000 7fa339905640 1 -- 172.21.15.3:0/32108552 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa33406e790 msgr2=0x7fa33415c4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:37:13.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.980+0000 7fa339905640 1 --2- 172.21.15.3:0/32108552 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa33406e790 0x7fa33415c4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:13.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.980+0000 7fa339905640 1 -- 172.21.15.3:0/32108552 >> v1:172.21.15.190:6789/0 conn(0x7fa33415c0f0 legacy=0x7fa334160730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:37:13.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.980+0000 7fa339905640 1 -- 172.21.15.3:0/32108552 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3340b3330 con 0x7fa3340b4e30 2024-05-09T06:37:13.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.981+0000 7fa339905640 1 --2- 172.21.15.3:0/32108552 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa3340b4e30 0x7fa33406e250 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fa324009870 tx=0x7fa32402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb2cb2dcfa759811 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:13.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.981+0000 7fa339104640 1 -- 172.21.15.3:0/32108552 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa324037070 con 0x7fa3340b4e30 2024-05-09T06:37:13.963 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.981+0000 7fa339104640 1 -- 172.21.15.3:0/32108552 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa324032440 con 0x7fa3340b4e30 2024-05-09T06:37:13.963 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.981+0000 7fa339104640 1 -- 172.21.15.3:0/32108552 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa324032810 con 0x7fa3340b4e30 2024-05-09T06:37:13.963 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.981+0000 7fa33c36a640 1 -- 172.21.15.3:0/32108552 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa3340b4e30 msgr2=0x7fa33406e250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:13.963 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.981+0000 7fa33c36a640 1 --2- 172.21.15.3:0/32108552 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa3340b4e30 0x7fa33406e250 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fa324009870 tx=0x7fa32402a180 comp rx=0 tx=0).stop 2024-05-09T06:37:13.963 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.981+0000 7fa33c36a640 1 -- 172.21.15.3:0/32108552 shutdown_connections 2024-05-09T06:37:13.963 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.981+0000 7fa33c36a640 1 --2- 172.21.15.3:0/32108552 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa33406e790 0x7fa33415c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:13.963 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.981+0000 7fa33c36a640 1 --2- 172.21.15.3:0/32108552 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa3340b4e30 0x7fa33406e250 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:13.963 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.981+0000 7fa33c36a640 1 -- 172.21.15.3:0/32108552 >> 172.21.15.3:0/32108552 conn(0x7fa3340b1c20 msgr2=0x7fa3340b2030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:13.963 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.982+0000 7fa33c36a640 1 -- 172.21.15.3:0/32108552 shutdown_connections 2024-05-09T06:37:13.963 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.982+0000 7fa33c36a640 1 -- 172.21.15.3:0/32108552 wait complete. 2024-05-09T06:37:13.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.983+0000 7fa33c36a640 1 Processor -- start 2024-05-09T06:37:13.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.983+0000 7fa33c36a640 1 -- start start 2024-05-09T06:37:13.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.983+0000 7fa33c36a640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa33406e790 0x7fa3341489e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:13.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.983+0000 7fa33c36a640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa3340b4e30 0x7fa334155460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:13.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.983+0000 7fa33c36a640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa33415c0f0 0x7fa3341505f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:13.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.983+0000 7fa33c36a640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fa3340b3bd0 con 0x7fa3340b4e30 2024-05-09T06:37:13.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.983+0000 7fa33c36a640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fa3340b38f0 con 0x7fa33415c0f0 2024-05-09T06:37:13.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.983+0000 7fa33c36a640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fa33415ca10 con 0x7fa33406e790 2024-05-09T06:37:13.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.984+0000 7fa339905640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa3340b4e30 0x7fa334155460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:13.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.984+0000 7fa339905640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa3340b4e30 0x7fa334155460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50970/0 (socket says 172.21.15.3:50970) 2024-05-09T06:37:13.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.984+0000 7fa339905640 1 -- 172.21.15.3:0/2176641639 learned_addr learned my addr 172.21.15.3:0/2176641639 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:37:13.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.984+0000 7fa33a907640 1 --2- 172.21.15.3:0/2176641639 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa33415c0f0 0x7fa3341505f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:13.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.984+0000 7fa33a106640 1 --2- 172.21.15.3:0/2176641639 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa33406e790 0x7fa3341489e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:13.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.984+0000 7fa339905640 1 -- 172.21.15.3:0/2176641639 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa33406e790 msgr2=0x7fa3341489e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:13.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.984+0000 7fa339905640 1 --2- 172.21.15.3:0/2176641639 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa33406e790 0x7fa3341489e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:13.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.984+0000 7fa339905640 1 -- 172.21.15.3:0/2176641639 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa33415c0f0 msgr2=0x7fa3341505f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:13.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.984+0000 7fa339905640 1 --2- 172.21.15.3:0/2176641639 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa33415c0f0 0x7fa3341505f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:13.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.984+0000 7fa339905640 1 -- 172.21.15.3:0/2176641639 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa334148f20 con 0x7fa3340b4e30 2024-05-09T06:37:13.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.984+0000 7fa33a907640 1 --2- 172.21.15.3:0/2176641639 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa33415c0f0 0x7fa3341505f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:37:13.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.984+0000 7fa339905640 1 --2- 172.21.15.3:0/2176641639 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa3340b4e30 0x7fa334155460 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fa324009870 tx=0x7fa3240046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:13.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.984+0000 7fa32affd640 1 -- 172.21.15.3:0/2176641639 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa324041020 con 0x7fa3340b4e30 2024-05-09T06:37:13.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.984+0000 7fa32affd640 1 -- 172.21.15.3:0/2176641639 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa3240306c0 con 0x7fa3340b4e30 2024-05-09T06:37:13.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.984+0000 7fa32affd640 1 -- 172.21.15.3:0/2176641639 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa324037040 con 0x7fa3340b4e30 2024-05-09T06:37:13.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.984+0000 7fa33c36a640 1 -- 172.21.15.3:0/2176641639 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3341f6170 con 0x7fa3340b4e30 2024-05-09T06:37:13.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.984+0000 7fa33c36a640 1 -- 172.21.15.3:0/2176641639 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3341f6730 con 0x7fa3340b4e30 2024-05-09T06:37:13.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.985+0000 7fa33c36a640 1 -- 172.21.15.3:0/2176641639 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa334069550 con 0x7fa3340b4e30 2024-05-09T06:37:13.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.986+0000 7fa32affd640 1 -- 172.21.15.3:0/2176641639 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 7) ==== 90651+0+0 (secure 0 0 0) 0x7fa324044020 con 0x7fa3340b4e30 2024-05-09T06:37:13.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.986+0000 7fa32affd640 1 --2- 172.21.15.3:0/2176641639 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fa31806bfe0 0x7fa31806e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:13.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.987+0000 7fa32affd640 1 -- 172.21.15.3:0/2176641639 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fa3240ab880 con 0x7fa3340b4e30 2024-05-09T06:37:13.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.987+0000 7fa33a106640 1 --2- 172.21.15.3:0/2176641639 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fa31806bfe0 0x7fa31806e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:13.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.987+0000 7fa33a106640 1 --2- 172.21.15.3:0/2176641639 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fa31806bfe0 0x7fa31806e4a0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fa330006fd0 tx=0x7fa330009060 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:13.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:13.991+0000 7fa32affd640 1 -- 172.21.15.3:0/2176641639 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fa32404b030 con 0x7fa3340b4e30 2024-05-09T06:37:14.155 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:14.174+0000 7fa33c36a640 1 -- 172.21.15.3:0/2176641639 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fa3340b42e0 con 0x7fa3340b4e30 2024-05-09T06:37:14.155 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:14.174+0000 7fa32affd640 1 -- 172.21.15.3:0/2176641639 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v33) ==== 104+0+11 (secure 0 0 0) 0x7fa334148f20 con 0x7fa3340b4e30 2024-05-09T06:37:14.155 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 32 2024-05-09T06:37:14.157 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:14.176+0000 7fa33c36a640 1 -- 172.21.15.3:0/2176641639 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fa31806bfe0 msgr2=0x7fa31806e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:14.157 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:14.176+0000 7fa33c36a640 1 --2- 172.21.15.3:0/2176641639 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fa31806bfe0 0x7fa31806e4a0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fa330006fd0 tx=0x7fa330009060 comp rx=0 tx=0).stop 2024-05-09T06:37:14.157 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:14.176+0000 7fa33c36a640 1 -- 172.21.15.3:0/2176641639 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa3340b4e30 msgr2=0x7fa334155460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:14.157 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:14.176+0000 7fa33c36a640 1 --2- 172.21.15.3:0/2176641639 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa3340b4e30 0x7fa334155460 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fa324009870 tx=0x7fa3240046c0 comp rx=0 tx=0).stop 2024-05-09T06:37:14.157 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:14.176+0000 7fa33c36a640 1 -- 172.21.15.3:0/2176641639 shutdown_connections 2024-05-09T06:37:14.157 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:14.176+0000 7fa33c36a640 1 --2- 172.21.15.3:0/2176641639 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fa31806bfe0 0x7fa31806e4a0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:14.157 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:14.176+0000 7fa33c36a640 1 --2- 172.21.15.3:0/2176641639 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa33415c0f0 0x7fa3341505f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:14.157 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:14.176+0000 7fa33c36a640 1 --2- 172.21.15.3:0/2176641639 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa3340b4e30 0x7fa334155460 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:14.158 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:14.176+0000 7fa33c36a640 1 --2- 172.21.15.3:0/2176641639 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa33406e790 0x7fa3341489e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:14.158 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:14.176+0000 7fa33c36a640 1 -- 172.21.15.3:0/2176641639 >> 172.21.15.3:0/2176641639 conn(0x7fa3340b1c20 msgr2=0x7fa33415f1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:14.158 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:14.176+0000 7fa33c36a640 1 -- 172.21.15.3:0/2176641639 shutdown_connections 2024-05-09T06:37:14.158 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:14.176+0000 7fa33c36a640 1 -- 172.21.15.3:0/2176641639 wait complete. 2024-05-09T06:37:14.168 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-09T06:37:14.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:14.295+0000 7f24e3aa5640 1 Processor -- start 2024-05-09T06:37:14.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:14.295+0000 7f24e3aa5640 1 -- start start 2024-05-09T06:37:14.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:14.296+0000 7f24e3aa5640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f24dc072900 0x7f24dc0cf8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:14.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:14.296+0000 7f24e3aa5640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f24dc0b4130 0x7f24dc0b45e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:14.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:14.296+0000 7f24e3aa5640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f24dc071860 con 0x7f24dc144a10 2024-05-09T06:37:14.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:14.296+0000 7f24e3aa5640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f24dc072470 con 0x7f24dc072900 2024-05-09T06:37:14.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:14.296+0000 7f24e3aa5640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f24dc071b40 con 0x7f24dc0b4130 2024-05-09T06:37:14.278 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:14.296+0000 7f24e1040640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f24dc072900 0x7f24dc0cf8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:14.278 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:14.296+0000 7f24e2042640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f24dc0b4130 0x7f24dc0b45e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:14.278 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:14.297+0000 7f24e1040640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f24dc072900 0x7f24dc0cf8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.3:43362/0 (socket says 172.21.15.3:43362) 2024-05-09T06:37:14.278 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:14.297+0000 7f24e2042640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f24dc0b4130 0x7f24dc0b45e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.3:38848/0 (socket says 172.21.15.3:38848) 2024-05-09T06:37:14.278 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:14.297+0000 7f24e1040640 1 -- 172.21.15.3:0/598192434 learned_addr learned my addr 172.21.15.3:0/598192434 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:37:14.278 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:14.297+0000 7f24e1841640 1 --1- 172.21.15.3:0/598192434 >> v1:172.21.15.190:6789/0 conn(0x7f24dc144a10 0x7f24dc071080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.3:42446/0 (socket says 172.21.15.3:42446) 2024-05-09T06:37:14.278 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:14.297+0000 7f24e1040640 1 -- 172.21.15.3:0/598192434 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f24dc0b4130 msgr2=0x7f24dc0b45e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:37:14.279 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:14.297+0000 7f24e1040640 1 --2- 172.21.15.3:0/598192434 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f24dc0b4130 0x7f24dc0b45e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:14.279 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:14.297+0000 7f24e1040640 1 -- 172.21.15.3:0/598192434 >> v1:172.21.15.190:6789/0 conn(0x7f24dc144a10 legacy=0x7f24dc071080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:14.279 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:14.297+0000 7f24e1040640 1 -- 172.21.15.3:0/598192434 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24dc071e20 con 0x7f24dc072900 2024-05-09T06:37:14.279 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:14.297+0000 7f24e083f640 1 -- 172.21.15.3:0/598192434 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2139920648 0 0) 0x7f24dc071860 con 0x7f24dc144a10 2024-05-09T06:37:14.279 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:14.298+0000 7f24e1040640 1 --2- 172.21.15.3:0/598192434 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f24dc072900 0x7f24dc0cf8a0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f24d0009870 tx=0x7f24d002a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=847d462c67212029 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:14.280 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:14.298+0000 7f24e083f640 1 -- 172.21.15.3:0/598192434 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24d0037070 con 0x7f24dc072900 2024-05-09T06:37:14.280 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:14.298+0000 7f24e083f640 1 -- 172.21.15.3:0/598192434 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f24d0032440 con 0x7f24dc072900 2024-05-09T06:37:14.280 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:14.298+0000 7f24e083f640 1 -- 172.21.15.3:0/598192434 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24d0032810 con 0x7f24dc072900 2024-05-09T06:37:14.280 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:14.298+0000 7f24e3aa5640 1 -- 172.21.15.3:0/598192434 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f24dc072900 msgr2=0x7f24dc0cf8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:14.280 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:14.298+0000 7f24e3aa5640 1 --2- 172.21.15.3:0/598192434 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f24dc072900 0x7f24dc0cf8a0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f24d0009870 tx=0x7f24d002a1c0 comp rx=0 tx=0).stop 2024-05-09T06:37:14.281 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:14.298+0000 7f24e3aa5640 1 -- 172.21.15.3:0/598192434 shutdown_connections 2024-05-09T06:37:14.281 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:14.298+0000 7f24e3aa5640 1 --2- 172.21.15.3:0/598192434 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f24dc0b4130 0x7f24dc0b45e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:14.281 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:14.298+0000 7f24e3aa5640 1 --2- 172.21.15.3:0/598192434 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f24dc072900 0x7f24dc0cf8a0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:14.281 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:14.298+0000 7f24e3aa5640 1 -- 172.21.15.3:0/598192434 >> 172.21.15.3:0/598192434 conn(0x7f24dc0b1c20 msgr2=0x7f24dc0b2030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:14.281 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:14.298+0000 7f24e3aa5640 1 -- 172.21.15.3:0/598192434 shutdown_connections 2024-05-09T06:37:14.281 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:14.299+0000 7f24e3aa5640 1 -- 172.21.15.3:0/598192434 wait complete. 2024-05-09T06:37:14.281 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:14.300+0000 7f24e3aa5640 1 Processor -- start 2024-05-09T06:37:14.281 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:14.300+0000 7f24e3aa5640 1 -- start start 2024-05-09T06:37:14.283 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:14.300+0000 7f24e3aa5640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f24dc072900 0x7f24dc143e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:14.283 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:14.300+0000 7f24e3aa5640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f24dc0b4130 0x7f24dc136850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:14.283 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:14.300+0000 7f24e3aa5640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f24dc144a10 0x7f24dc15dca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:14.283 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:14.300+0000 7f24e3aa5640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f24dc15b750 con 0x7f24dc0b4130 2024-05-09T06:37:14.283 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:14.300+0000 7f24e3aa5640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f24dc15b5d0 con 0x7f24dc144a10 2024-05-09T06:37:14.283 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:14.300+0000 7f24e3aa5640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f24dc15b8d0 con 0x7f24dc072900 2024-05-09T06:37:14.283 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:14.301+0000 7f24e1040640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f24dc0b4130 0x7f24dc136850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:14.283 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:14.301+0000 7f24e1040640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f24dc0b4130 0x7f24dc136850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50980/0 (socket says 172.21.15.3:50980) 2024-05-09T06:37:14.283 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:14.301+0000 7f24e1040640 1 -- 172.21.15.3:0/153750872 learned_addr learned my addr 172.21.15.3:0/153750872 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:37:14.283 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:14.301+0000 7f24e1841640 1 --2- 172.21.15.3:0/153750872 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f24dc072900 0x7f24dc143e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:14.284 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:14.301+0000 7f24e2042640 1 --2- 172.21.15.3:0/153750872 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f24dc144a10 0x7f24dc15dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:14.284 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:14.301+0000 7f24e1040640 1 -- 172.21.15.3:0/153750872 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f24dc072900 msgr2=0x7f24dc143e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:14.284 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:14.301+0000 7f24e1040640 1 --2- 172.21.15.3:0/153750872 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f24dc072900 0x7f24dc143e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:14.284 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:14.301+0000 7f24e1040640 1 -- 172.21.15.3:0/153750872 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f24dc144a10 msgr2=0x7f24dc15dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:14.284 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:14.301+0000 7f24e1040640 1 --2- 172.21.15.3:0/153750872 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f24dc144a10 0x7f24dc15dca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:14.284 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:14.301+0000 7f24e1040640 1 -- 172.21.15.3:0/153750872 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24dc13af00 con 0x7f24dc0b4130 2024-05-09T06:37:14.284 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:14.301+0000 7f24e1040640 1 --2- 172.21.15.3:0/153750872 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f24dc0b4130 0x7f24dc136850 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f24d0035040 tx=0x7f24d00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:14.284 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:14.301+0000 7f24da7fc640 1 -- 172.21.15.3:0/153750872 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24d0037040 con 0x7f24dc0b4130 2024-05-09T06:37:14.284 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:14.301+0000 7f24da7fc640 1 -- 172.21.15.3:0/153750872 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f24d0004230 con 0x7f24dc0b4130 2024-05-09T06:37:14.284 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:14.301+0000 7f24e3aa5640 1 -- 172.21.15.3:0/153750872 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24dc13b1e0 con 0x7f24dc0b4130 2024-05-09T06:37:14.284 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:14.301+0000 7f24da7fc640 1 -- 172.21.15.3:0/153750872 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24d0003610 con 0x7f24dc0b4130 2024-05-09T06:37:14.284 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:14.301+0000 7f24e3aa5640 1 -- 172.21.15.3:0/153750872 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24dc13b7a0 con 0x7f24dc0b4130 2024-05-09T06:37:14.284 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:14.302+0000 7f24e3aa5640 1 -- 172.21.15.3:0/153750872 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24dc069550 con 0x7f24dc0b4130 2024-05-09T06:37:14.285 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:14.303+0000 7f24da7fc640 1 -- 172.21.15.3:0/153750872 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 7) ==== 90651+0+0 (secure 0 0 0) 0x7f24d002ad10 con 0x7f24dc0b4130 2024-05-09T06:37:14.285 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:14.303+0000 7f24da7fc640 1 --2- 172.21.15.3:0/153750872 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f24c806bfe0 0x7f24c806e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:14.285 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:14.303+0000 7f24da7fc640 1 -- 172.21.15.3:0/153750872 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f24d00abce0 con 0x7f24dc0b4130 2024-05-09T06:37:14.286 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:14.303+0000 7f24e1841640 1 --2- 172.21.15.3:0/153750872 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f24c806bfe0 0x7f24c806e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:14.286 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:14.304+0000 7f24e1841640 1 --2- 172.21.15.3:0/153750872 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f24c806bfe0 0x7f24c806e4a0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f24d4000f60 tx=0x7f24d4007c00 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:14.288 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:14.308+0000 7f24da7fc640 1 -- 172.21.15.3:0/153750872 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f24d007a530 con 0x7f24dc0b4130 2024-05-09T06:37:14.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:14.492+0000 7f24e3aa5640 1 -- 172.21.15.3:0/153750872 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f24dc06d3e0 con 0x7f24dc0b4130 2024-05-09T06:37:14.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:14.492+0000 7f24da7fc640 1 -- 172.21.15.3:0/153750872 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v33) ==== 100+0+11 (secure 0 0 0) 0x7f24d0079e30 con 0x7f24dc0b4130 2024-05-09T06:37:14.474 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 64 2024-05-09T06:37:14.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:14.494+0000 7f24e3aa5640 1 -- 172.21.15.3:0/153750872 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f24c806bfe0 msgr2=0x7f24c806e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:14.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:14.494+0000 7f24e3aa5640 1 --2- 172.21.15.3:0/153750872 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f24c806bfe0 0x7f24c806e4a0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f24d4000f60 tx=0x7f24d4007c00 comp rx=0 tx=0).stop 2024-05-09T06:37:14.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:14.494+0000 7f24e3aa5640 1 -- 172.21.15.3:0/153750872 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f24dc0b4130 msgr2=0x7f24dc136850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:14.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:14.494+0000 7f24e3aa5640 1 --2- 172.21.15.3:0/153750872 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f24dc0b4130 0x7f24dc136850 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f24d0035040 tx=0x7f24d00046c0 comp rx=0 tx=0).stop 2024-05-09T06:37:14.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:14.495+0000 7f24e3aa5640 1 -- 172.21.15.3:0/153750872 shutdown_connections 2024-05-09T06:37:14.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:14.495+0000 7f24e3aa5640 1 --2- 172.21.15.3:0/153750872 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f24c806bfe0 0x7f24c806e4a0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:14.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:14.495+0000 7f24e3aa5640 1 --2- 172.21.15.3:0/153750872 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f24dc144a10 0x7f24dc15dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:14.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:14.495+0000 7f24e3aa5640 1 --2- 172.21.15.3:0/153750872 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f24dc0b4130 0x7f24dc136850 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:14.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:14.495+0000 7f24e3aa5640 1 --2- 172.21.15.3:0/153750872 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f24dc072900 0x7f24dc143e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:14.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:14.495+0000 7f24e3aa5640 1 -- 172.21.15.3:0/153750872 >> 172.21.15.3:0/153750872 conn(0x7f24dc0b1c20 msgr2=0x7f24dc13fec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:14.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:14.495+0000 7f24e3aa5640 1 -- 172.21.15.3:0/153750872 shutdown_connections 2024-05-09T06:37:14.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:14.495+0000 7f24e3aa5640 1 -- 172.21.15.3:0/153750872 wait complete. 2024-05-09T06:37:14.486 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-09T06:37:14.594 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:14.613+0000 7ff7231ce640 1 Processor -- start 2024-05-09T06:37:14.594 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:14.613+0000 7ff7231ce640 1 -- start start 2024-05-09T06:37:14.596 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:14.614+0000 7ff7231ce640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7ff71c0727a0 0x7ff71c1406c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:14.596 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:14.614+0000 7ff7231ce640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff71c0cd820 0x7ff71c140c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:14.596 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:14.614+0000 7ff7231ce640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff71c141340 con 0x7ff71c0cd440 2024-05-09T06:37:14.596 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:14.614+0000 7ff7231ce640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7ff71c13e3b0 con 0x7ff71c0cd820 2024-05-09T06:37:14.596 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:14.614+0000 7ff7231ce640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7ff71c06e510 con 0x7ff71c0727a0 2024-05-09T06:37:14.596 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:14.614+0000 7ff72176b640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff71c0cd820 0x7ff71c140c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:14.596 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:14.614+0000 7ff720f6a640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7ff71c0cd440 0x7ff71c072090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:45914/0 (socket says 172.21.15.3:45914) 2024-05-09T06:37:14.596 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:14.614+0000 7ff720f6a640 1 -- 172.21.15.3:0/2078934195 learned_addr learned my addr 172.21.15.3:0/2078934195 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-05-09T06:37:14.596 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:14.615+0000 7ff71bfff640 1 --2- 172.21.15.3:0/2078934195 >> v2:172.21.15.190:3300/0 conn(0x7ff71c0727a0 0x7ff71c1406c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:14.596 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:14.615+0000 7ff72176b640 1 -- 172.21.15.3:0/2078934195 >> v2:172.21.15.190:3300/0 conn(0x7ff71c0727a0 msgr2=0x7ff71c1406c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:37:14.596 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:14.615+0000 7ff72176b640 1 --2- 172.21.15.3:0/2078934195 >> v2:172.21.15.190:3300/0 conn(0x7ff71c0727a0 0x7ff71c1406c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:14.596 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:14.615+0000 7ff72176b640 1 -- 172.21.15.3:0/2078934195 >> v1:172.21.15.3:6789/0 conn(0x7ff71c0cd440 legacy=0x7ff71c072090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:14.596 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:14.615+0000 7ff72176b640 1 -- 172.21.15.3:0/2078934195 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff71c06e7f0 con 0x7ff71c0cd820 2024-05-09T06:37:14.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:14.615+0000 7ff71b7fe640 1 -- 172.21.15.3:0/2078934195 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3042373237 0 0) 0x7ff71c141340 con 0x7ff71c0cd440 2024-05-09T06:37:14.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:14.615+0000 7ff72176b640 1 --2- 172.21.15.3:0/2078934195 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff71c0cd820 0x7ff71c140c00 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7ff71c0b0b70 tx=0x7ff71402a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6688420d5d31b1e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:14.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:14.616+0000 7ff71b7fe640 1 -- 172.21.15.3:0/2078934195 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff714037070 con 0x7ff71c0cd820 2024-05-09T06:37:14.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:14.616+0000 7ff71b7fe640 1 -- 172.21.15.3:0/2078934195 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff714032440 con 0x7ff71c0cd820 2024-05-09T06:37:14.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:14.616+0000 7ff71b7fe640 1 -- 172.21.15.3:0/2078934195 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff71402ae60 con 0x7ff71c0cd820 2024-05-09T06:37:14.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:14.616+0000 7ff7231ce640 1 -- 172.21.15.3:0/2078934195 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff71c0cd820 msgr2=0x7ff71c140c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:14.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:14.616+0000 7ff7231ce640 1 --2- 172.21.15.3:0/2078934195 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff71c0cd820 0x7ff71c140c00 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7ff71c0b0b70 tx=0x7ff71402a110 comp rx=0 tx=0).stop 2024-05-09T06:37:14.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:14.616+0000 7ff7231ce640 1 -- 172.21.15.3:0/2078934195 shutdown_connections 2024-05-09T06:37:14.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:14.616+0000 7ff7231ce640 1 --2- 172.21.15.3:0/2078934195 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff71c0cd820 0x7ff71c140c00 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:14.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:14.616+0000 7ff7231ce640 1 --2- 172.21.15.3:0/2078934195 >> v2:172.21.15.190:3300/0 conn(0x7ff71c0727a0 0x7ff71c1406c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:14.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:14.616+0000 7ff7231ce640 1 -- 172.21.15.3:0/2078934195 >> 172.21.15.3:0/2078934195 conn(0x7ff71c06ced0 msgr2=0x7ff71c0b2c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:14.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:14.616+0000 7ff7231ce640 1 -- 172.21.15.3:0/2078934195 shutdown_connections 2024-05-09T06:37:14.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:14.616+0000 7ff7231ce640 1 -- 172.21.15.3:0/2078934195 wait complete. 2024-05-09T06:37:14.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:14.617+0000 7ff7231ce640 1 Processor -- start 2024-05-09T06:37:14.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:14.617+0000 7ff7231ce640 1 -- start start 2024-05-09T06:37:14.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:14.618+0000 7ff7231ce640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff71c0727a0 0x7ff71c1f5a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:14.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:14.618+0000 7ff7231ce640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff71c0cd440 0x7ff71c13d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:14.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:14.618+0000 7ff7231ce640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff71c0cd820 0x7ff71c15dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:14.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:14.618+0000 7ff7231ce640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7ff71c06f170 con 0x7ff71c0727a0 2024-05-09T06:37:14.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:14.618+0000 7ff7231ce640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7ff71c06eff0 con 0x7ff71c0cd820 2024-05-09T06:37:14.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:14.618+0000 7ff7231ce640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7ff71c06f2f0 con 0x7ff71c0cd440 2024-05-09T06:37:14.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:14.618+0000 7ff720f6a640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff71c0727a0 0x7ff71c1f5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:14.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:14.618+0000 7ff720f6a640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff71c0727a0 0x7ff71c1f5a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51002/0 (socket says 172.21.15.3:51002) 2024-05-09T06:37:14.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:14.618+0000 7ff720f6a640 1 -- 172.21.15.3:0/3702339960 learned_addr learned my addr 172.21.15.3:0/3702339960 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:37:14.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:14.619+0000 7ff72176b640 1 --2- 172.21.15.3:0/3702339960 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff71c0cd820 0x7ff71c15dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:14.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:14.619+0000 7ff71bfff640 1 --2- 172.21.15.3:0/3702339960 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff71c0cd440 0x7ff71c13d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:14.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:14.619+0000 7ff720f6a640 1 -- 172.21.15.3:0/3702339960 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff71c0cd440 msgr2=0x7ff71c13d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:14.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:14.619+0000 7ff720f6a640 1 --2- 172.21.15.3:0/3702339960 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff71c0cd440 0x7ff71c13d9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:14.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:14.619+0000 7ff720f6a640 1 -- 172.21.15.3:0/3702339960 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff71c0cd820 msgr2=0x7ff71c15dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:14.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:14.619+0000 7ff720f6a640 1 --2- 172.21.15.3:0/3702339960 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff71c0cd820 0x7ff71c15dc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:14.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:14.619+0000 7ff720f6a640 1 -- 172.21.15.3:0/3702339960 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff71c15e140 con 0x7ff71c0727a0 2024-05-09T06:37:14.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:14.619+0000 7ff720f6a640 1 --2- 172.21.15.3:0/3702339960 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff71c0727a0 0x7ff71c1f5a20 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7ff70c00c4d0 tx=0x7ff70c00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:14.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:14.619+0000 7ff7197fa640 1 -- 172.21.15.3:0/3702339960 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff70c018070 con 0x7ff71c0727a0 2024-05-09T06:37:14.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:14.619+0000 7ff7197fa640 1 -- 172.21.15.3:0/3702339960 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff70c007500 con 0x7ff71c0727a0 2024-05-09T06:37:14.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:14.619+0000 7ff7231ce640 1 -- 172.21.15.3:0/3702339960 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff71c15b660 con 0x7ff71c0727a0 2024-05-09T06:37:14.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:14.619+0000 7ff7197fa640 1 -- 172.21.15.3:0/3702339960 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff70c004a20 con 0x7ff71c0727a0 2024-05-09T06:37:14.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:14.619+0000 7ff7231ce640 1 -- 172.21.15.3:0/3702339960 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff71c15bc20 con 0x7ff71c0727a0 2024-05-09T06:37:14.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:14.620+0000 7ff7231ce640 1 -- 172.21.15.3:0/3702339960 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff71c1524d0 con 0x7ff71c0727a0 2024-05-09T06:37:14.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:14.621+0000 7ff7197fa640 1 -- 172.21.15.3:0/3702339960 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 7) ==== 90651+0+0 (secure 0 0 0) 0x7ff70c01b070 con 0x7ff71c0727a0 2024-05-09T06:37:14.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:14.621+0000 7ff7197fa640 1 --2- 172.21.15.3:0/3702339960 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7ff70806bfe0 0x7ff70806e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:14.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:14.621+0000 7ff7197fa640 1 -- 172.21.15.3:0/3702339960 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7ff70c08a430 con 0x7ff71c0727a0 2024-05-09T06:37:14.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:14.622+0000 7ff71bfff640 1 --2- 172.21.15.3:0/3702339960 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7ff70806bfe0 0x7ff70806e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:14.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:14.622+0000 7ff71bfff640 1 --2- 172.21.15.3:0/3702339960 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7ff70806bfe0 0x7ff70806e4a0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7ff71c1576f0 tx=0x7ff7100083a0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:14.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:14.630+0000 7ff7197fa640 1 -- 172.21.15.3:0/3702339960 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7ff71c15e140 con 0x7ff71c0727a0 2024-05-09T06:37:14.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:14.832+0000 7ff7231ce640 1 -- 172.21.15.3:0/3702339960 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7ff71c069550 con 0x7ff71c0727a0 2024-05-09T06:37:14.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:14.832+0000 7ff7197fa640 1 -- 172.21.15.3:0/3702339960 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v33) ==== 80+0+0 (secure 0 0 0) 0x7ff70c058bf0 con 0x7ff71c0727a0 2024-05-09T06:37:14.813 INFO:teuthology.orchestra.run.smithi003.stderr:listed 0 entries 2024-05-09T06:37:14.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:14.834+0000 7ff7231ce640 1 -- 172.21.15.3:0/3702339960 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7ff70806bfe0 msgr2=0x7ff70806e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:14.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:14.834+0000 7ff7231ce640 1 --2- 172.21.15.3:0/3702339960 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7ff70806bfe0 0x7ff70806e4a0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7ff71c1576f0 tx=0x7ff7100083a0 comp rx=0 tx=0).stop 2024-05-09T06:37:14.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:14.834+0000 7ff7231ce640 1 -- 172.21.15.3:0/3702339960 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff71c0727a0 msgr2=0x7ff71c1f5a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:14.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:14.834+0000 7ff7231ce640 1 --2- 172.21.15.3:0/3702339960 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff71c0727a0 0x7ff71c1f5a20 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7ff70c00c4d0 tx=0x7ff70c00c990 comp rx=0 tx=0).stop 2024-05-09T06:37:14.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:14.834+0000 7ff7231ce640 1 -- 172.21.15.3:0/3702339960 shutdown_connections 2024-05-09T06:37:14.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:14.834+0000 7ff7231ce640 1 --2- 172.21.15.3:0/3702339960 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7ff70806bfe0 0x7ff70806e4a0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:14.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:14.834+0000 7ff7231ce640 1 --2- 172.21.15.3:0/3702339960 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff71c0cd820 0x7ff71c15dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:14.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:14.834+0000 7ff7231ce640 1 --2- 172.21.15.3:0/3702339960 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff71c0cd440 0x7ff71c13d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:14.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:14.834+0000 7ff7231ce640 1 --2- 172.21.15.3:0/3702339960 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff71c0727a0 0x7ff71c1f5a20 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:14.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:14.834+0000 7ff7231ce640 1 -- 172.21.15.3:0/3702339960 >> 172.21.15.3:0/3702339960 conn(0x7ff71c06ced0 msgr2=0x7ff71c0b1820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:14.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:14.834+0000 7ff7231ce640 1 -- 172.21.15.3:0/3702339960 shutdown_connections 2024-05-09T06:37:14.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:14.834+0000 7ff7231ce640 1 -- 172.21.15.3:0/3702339960 wait complete. 2024-05-09T06:37:14.825 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-05-09T06:37:14.826 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:37:14.826 DEBUG:teuthology.orchestra.run.smithi190:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-05-09T06:37:14.861 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi190.front.sepia.ceph.com 2024-05-09T06:37:14.861 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:37:14.861 DEBUG:teuthology.orchestra.run.smithi190:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-05-09T06:37:14.920 INFO:tasks.cephfs.mount:Terminating background process 2024-05-09T06:37:14.921 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-05-09T06:37:14.921 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T06:37:14.938 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057103,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13674 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249452k,nr_inodes=812363,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-09T06:37:14.938 INFO:tasks.cephfs.mount:Terminating background process 2024-05-09T06:37:14.938 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-05-09T06:37:14.938 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T06:37:14.979 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056337,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497676k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19681 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497676k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-09T06:37:14.979 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T06:37:15.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.103+0000 7f80cda15640 1 Processor -- start 2024-05-09T06:37:15.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.103+0000 7f80cda15640 1 -- start start 2024-05-09T06:37:15.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.104+0000 7f80cda15640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f80c80cde50 0x7f80c80b4250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:15.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.104+0000 7f80cda15640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f80c80b48e0 con 0x7f80c80ccad0 2024-05-09T06:37:15.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.104+0000 7f80cda15640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f80c80b4bc0 con 0x7f80c80cc6f0 2024-05-09T06:37:15.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.104+0000 7f80cda15640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f80c80b4ea0 con 0x7f80c80cde50 2024-05-09T06:37:15.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.104+0000 7f80c67fc640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f80c80ccad0 0x7f80c806da20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:45930/0 (socket says 172.21.15.3:45930) 2024-05-09T06:37:15.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.104+0000 7f80c67fc640 1 -- 172.21.15.3:0/1556952409 learned_addr learned my addr 172.21.15.3:0/1556952409 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-05-09T06:37:15.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.104+0000 7f80c77fe640 1 --2- 172.21.15.3:0/1556952409 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f80c80cde50 0x7f80c80b4250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:15.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.104+0000 7f80c5ffb640 1 -- 172.21.15.3:0/1556952409 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3035497716 0 0) 0x7f80c80b48e0 con 0x7f80c80ccad0 2024-05-09T06:37:15.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.104+0000 7f80c5ffb640 1 -- 172.21.15.3:0/1556952409 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f80b4003520 con 0x7f80c80ccad0 2024-05-09T06:37:15.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.104+0000 7f80c5ffb640 1 -- 172.21.15.3:0/1556952409 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2730070750 0 0) 0x7f80c80b4bc0 con 0x7f80c80cc6f0 2024-05-09T06:37:15.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.104+0000 7f80c5ffb640 1 -- 172.21.15.3:0/1556952409 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f80c80b48e0 con 0x7f80c80cc6f0 2024-05-09T06:37:15.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.104+0000 7f80c5ffb640 1 -- 172.21.15.3:0/1556952409 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2329767833 0 0) 0x7f80b4003520 con 0x7f80c80ccad0 2024-05-09T06:37:15.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.104+0000 7f80c5ffb640 1 -- 172.21.15.3:0/1556952409 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f80c80cde50 msgr2=0x7f80c80b4250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:37:15.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.104+0000 7f80c5ffb640 1 --2- 172.21.15.3:0/1556952409 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f80c80cde50 0x7f80c80b4250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:15.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.104+0000 7f80c5ffb640 1 -- 172.21.15.3:0/1556952409 >> v1:172.21.15.190:6789/0 conn(0x7f80c80cc6f0 legacy=0x7f80c80cd740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:15.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.105+0000 7f80c5ffb640 1 -- 172.21.15.3:0/1556952409 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80c80b5180 con 0x7f80c80ccad0 2024-05-09T06:37:15.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.105+0000 7f80c5ffb640 1 -- 172.21.15.3:0/1556952409 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 563312031 0 0) 0x7f80b0001670 con 0x7f80c80ccad0 2024-05-09T06:37:15.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.105+0000 7f80c77fe640 1 --2- 172.21.15.3:0/1556952409 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f80c80cde50 0x7f80c80b4250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T06:37:15.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.105+0000 7f80c5ffb640 1 -- 172.21.15.3:0/1556952409 >> v1:172.21.15.3:6789/0 conn(0x7f80c80ccad0 legacy=0x7f80c806da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:15.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.105+0000 7f80c5ffb640 1 --2- 172.21.15.3:0/1556952409 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f80b4003b70 0x7f80b4003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:15.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.105+0000 7f80c5ffb640 1 --2- 172.21.15.3:0/1556952409 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f80b4004800 0x7f80b4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:15.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.105+0000 7f80c5ffb640 1 --2- 172.21.15.3:0/1556952409 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f80c80cde50 0x7f80b40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:15.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.105+0000 7f80c5ffb640 1 -- 172.21.15.3:0/1556952409 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f80c80b4ea0 con 0x7f80b4003b70 2024-05-09T06:37:15.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.105+0000 7f80c5ffb640 1 -- 172.21.15.3:0/1556952409 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f80b400b9f0 con 0x7f80b4004800 2024-05-09T06:37:15.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.105+0000 7f80c5ffb640 1 -- 172.21.15.3:0/1556952409 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f80b400bbf0 con 0x7f80c80cde50 2024-05-09T06:37:15.090 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.105+0000 7f80c6ffd640 1 --2- 172.21.15.3:0/1556952409 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f80b4003b70 0x7f80b4003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:15.090 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.105+0000 7f80c67fc640 1 --2- 172.21.15.3:0/1556952409 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f80b4004800 0x7f80b4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:15.090 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.105+0000 7f80c77fe640 1 --2- 172.21.15.3:0/1556952409 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f80c80cde50 0x7f80b40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:15.090 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.105+0000 7f80c6ffd640 1 -- 172.21.15.3:0/1556952409 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f80c80cde50 msgr2=0x7f80b40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:37:15.090 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.105+0000 7f80c6ffd640 1 --2- 172.21.15.3:0/1556952409 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f80c80cde50 0x7f80b40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:15.090 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.105+0000 7f80c6ffd640 1 -- 172.21.15.3:0/1556952409 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f80b4004800 msgr2=0x7f80b4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:37:15.090 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.105+0000 7f80c6ffd640 1 --2- 172.21.15.3:0/1556952409 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f80b4004800 0x7f80b4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:15.090 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.105+0000 7f80c6ffd640 1 -- 172.21.15.3:0/1556952409 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80b4003520 con 0x7f80b4003b70 2024-05-09T06:37:15.090 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.105+0000 7f80c67fc640 1 --2- 172.21.15.3:0/1556952409 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f80b4004800 0x7f80b4004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:37:15.090 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.106+0000 7f80c6ffd640 1 --2- 172.21.15.3:0/1556952409 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f80b4003b70 0x7f80b4003f70 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f80b8001fa0 tx=0x7f80b4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a095b0eb04f16d3 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:15.090 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.106+0000 7f80c5ffb640 1 -- 172.21.15.3:0/1556952409 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80b8001bf0 con 0x7f80b4003b70 2024-05-09T06:37:15.090 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.106+0000 7f80c5ffb640 1 -- 172.21.15.3:0/1556952409 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f80b8003b70 con 0x7f80b4003b70 2024-05-09T06:37:15.090 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.106+0000 7f80c5ffb640 1 -- 172.21.15.3:0/1556952409 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80b80050d0 con 0x7f80b4003b70 2024-05-09T06:37:15.091 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.106+0000 7f80cda15640 1 -- 172.21.15.3:0/1556952409 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f80b4003b70 msgr2=0x7f80b4003f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:15.091 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.106+0000 7f80cda15640 1 --2- 172.21.15.3:0/1556952409 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f80b4003b70 0x7f80b4003f70 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f80b8001fa0 tx=0x7f80b4005830 comp rx=0 tx=0).stop 2024-05-09T06:37:15.091 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.106+0000 7f80c77fe640 1 -- 172.21.15.3:0/1556952409 reap_dead start 2024-05-09T06:37:15.091 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.106+0000 7f80cda15640 1 -- 172.21.15.3:0/1556952409 shutdown_connections 2024-05-09T06:37:15.091 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.106+0000 7f80cda15640 1 -- 172.21.15.3:0/1556952409 >> 172.21.15.3:0/1556952409 conn(0x7f80c806b430 msgr2=0x7f80c806b810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:15.091 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.107+0000 7f80cda15640 1 -- 172.21.15.3:0/1556952409 shutdown_connections 2024-05-09T06:37:15.091 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.107+0000 7f80cda15640 1 -- 172.21.15.3:0/1556952409 wait complete. 2024-05-09T06:37:15.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.108+0000 7f80cda15640 1 Processor -- start 2024-05-09T06:37:15.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.108+0000 7f80cda15640 1 -- start start 2024-05-09T06:37:15.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.109+0000 7f80cda15640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f80b4003b70 0x7f80c814e500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:15.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.109+0000 7f80cda15640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f80c81f2670 0x7f80c814b860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:15.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.109+0000 7f80cda15640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f80c81503f0 0x7f80c8150ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:15.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.109+0000 7f80cda15640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f80c80b43d0 con 0x7f80b4003b70 2024-05-09T06:37:15.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.109+0000 7f80cda15640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f80c80b4250 con 0x7f80c81f2670 2024-05-09T06:37:15.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.109+0000 7f80cda15640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f80c80b4550 con 0x7f80c81503f0 2024-05-09T06:37:15.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.109+0000 7f80c6ffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f80b4003b70 0x7f80c814e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:15.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.109+0000 7f80c6ffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f80b4003b70 0x7f80c814e500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51022/0 (socket says 172.21.15.3:51022) 2024-05-09T06:37:15.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.109+0000 7f80c6ffd640 1 -- 172.21.15.3:0/2434317694 learned_addr learned my addr 172.21.15.3:0/2434317694 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:37:15.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.109+0000 7f80c67fc640 1 --2- 172.21.15.3:0/2434317694 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f80c81f2670 0x7f80c814b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:15.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.109+0000 7f80c77fe640 1 --2- 172.21.15.3:0/2434317694 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f80c81503f0 0x7f80c8150ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:15.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.109+0000 7f80c6ffd640 1 -- 172.21.15.3:0/2434317694 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f80c81503f0 msgr2=0x7f80c8150ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:15.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.109+0000 7f80c6ffd640 1 --2- 172.21.15.3:0/2434317694 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f80c81503f0 0x7f80c8150ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:15.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.109+0000 7f80c6ffd640 1 -- 172.21.15.3:0/2434317694 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f80c81f2670 msgr2=0x7f80c814b860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:15.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.109+0000 7f80c6ffd640 1 --2- 172.21.15.3:0/2434317694 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f80c81f2670 0x7f80c814b860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:15.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.109+0000 7f80c6ffd640 1 -- 172.21.15.3:0/2434317694 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80c814eb90 con 0x7f80b4003b70 2024-05-09T06:37:15.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.109+0000 7f80c67fc640 1 --2- 172.21.15.3:0/2434317694 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f80c81f2670 0x7f80c814b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:37:15.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.109+0000 7f80c6ffd640 1 --2- 172.21.15.3:0/2434317694 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f80b4003b70 0x7f80c814e500 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f80b8005040 tx=0x7f80b8001650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:15.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.109+0000 7f80bffff640 1 -- 172.21.15.3:0/2434317694 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80b8040070 con 0x7f80b4003b70 2024-05-09T06:37:15.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.109+0000 7f80bffff640 1 -- 172.21.15.3:0/2434317694 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f80b8003970 con 0x7f80b4003b70 2024-05-09T06:37:15.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.110+0000 7f80cda15640 1 -- 172.21.15.3:0/2434317694 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80c814f670 con 0x7f80b4003b70 2024-05-09T06:37:15.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.110+0000 7f80bffff640 1 -- 172.21.15.3:0/2434317694 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80b8001bf0 con 0x7f80b4003b70 2024-05-09T06:37:15.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.110+0000 7f80cda15640 1 -- 172.21.15.3:0/2434317694 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80c814f910 con 0x7f80b4003b70 2024-05-09T06:37:15.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.111+0000 7f80cda15640 1 -- 172.21.15.3:0/2434317694 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80c804f660 con 0x7f80b4003b70 2024-05-09T06:37:15.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.111+0000 7f80bffff640 1 -- 172.21.15.3:0/2434317694 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 7) ==== 90651+0+0 (secure 0 0 0) 0x7f80b8043070 con 0x7f80b4003b70 2024-05-09T06:37:15.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.112+0000 7f80bffff640 1 --2- 172.21.15.3:0/2434317694 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f80a806bfe0 0x7f80a806e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:15.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.112+0000 7f80bffff640 1 -- 172.21.15.3:0/2434317694 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f80b80abf50 con 0x7f80b4003b70 2024-05-09T06:37:15.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.112+0000 7f80c67fc640 1 --2- 172.21.15.3:0/2434317694 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f80a806bfe0 0x7f80a806e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:15.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.112+0000 7f80c67fc640 1 --2- 172.21.15.3:0/2434317694 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f80a806bfe0 0x7f80a806e4a0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f80b0000fd0 tx=0x7f80b0001190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:15.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.116+0000 7f80bffff640 1 -- 172.21.15.3:0/2434317694 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f80b807a790 con 0x7f80b4003b70 2024-05-09T06:37:15.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.337+0000 7f80cda15640 1 -- 172.21.15.3:0/2434317694 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f80c80b3390 con 0x7f80b4003b70 2024-05-09T06:37:15.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.338+0000 7f80bffff640 1 -- 172.21.15.3:0/2434317694 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+5181 (secure 0 0 0) 0x7f80b807a090 con 0x7f80b4003b70 2024-05-09T06:37:15.319 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-09T06:37:15.319 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4912,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6836/2400645940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":2400645940},{"type":"v1","addr":"172.21.15.190:6836","nonce":2400645940}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4915,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6837/3849729576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6835","nonce":3849729576},{"type":"v1","addr":"172.21.15.190:6837","nonce":3849729576}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4918,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6834/2647782276","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":2647782276},{"type":"v1","addr":"172.21.15.3:6834","nonce":2647782276}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-09T06:36:57.929421+0000","modified":"2024-05-09T06:37:00.941894+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5103},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5103":{"gid":5103,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.3:6835/2138710335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6833","nonce":2138710335},{"type":"v1","addr":"172.21.15.3:6835","nonce":2138710335}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5103,"qdb_cluster":[5103]},"id":2}]} 2024-05-09T06:37:15.319 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 18 2024-05-09T06:37:15.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.340+0000 7f80cda15640 1 -- 172.21.15.3:0/2434317694 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f80a806bfe0 msgr2=0x7f80a806e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:15.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.340+0000 7f80cda15640 1 --2- 172.21.15.3:0/2434317694 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f80a806bfe0 0x7f80a806e4a0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f80b0000fd0 tx=0x7f80b0001190 comp rx=0 tx=0).stop 2024-05-09T06:37:15.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.340+0000 7f80cda15640 1 -- 172.21.15.3:0/2434317694 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f80b4003b70 msgr2=0x7f80c814e500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:15.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.340+0000 7f80cda15640 1 --2- 172.21.15.3:0/2434317694 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f80b4003b70 0x7f80c814e500 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f80b8005040 tx=0x7f80b8001650 comp rx=0 tx=0).stop 2024-05-09T06:37:15.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.340+0000 7f80cda15640 1 -- 172.21.15.3:0/2434317694 shutdown_connections 2024-05-09T06:37:15.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.340+0000 7f80cda15640 1 --2- 172.21.15.3:0/2434317694 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f80a806bfe0 0x7f80a806e4a0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:15.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.340+0000 7f80cda15640 1 --2- 172.21.15.3:0/2434317694 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f80c81503f0 0x7f80c8150ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:15.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.340+0000 7f80cda15640 1 --2- 172.21.15.3:0/2434317694 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f80c81f2670 0x7f80c814b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:15.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.340+0000 7f80cda15640 1 --2- 172.21.15.3:0/2434317694 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f80b4003b70 0x7f80c814e500 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:15.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.340+0000 7f80cda15640 1 -- 172.21.15.3:0/2434317694 >> 172.21.15.3:0/2434317694 conn(0x7f80c806b430 msgr2=0x7f80c80cd740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:15.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.340+0000 7f80cda15640 1 -- 172.21.15.3:0/2434317694 shutdown_connections 2024-05-09T06:37:15.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.340+0000 7f80cda15640 1 -- 172.21.15.3:0/2434317694 wait complete. 2024-05-09T06:37:15.332 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T06:37:15.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.454+0000 7fd8bc94f640 1 Processor -- start 2024-05-09T06:37:15.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.454+0000 7fd8bc94f640 1 -- start start 2024-05-09T06:37:15.437 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.455+0000 7fd8bc94f640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd8b80d27c0 0x7fd8b80d2c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:15.437 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.455+0000 7fd8bc94f640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fd8b8166390 0x7fd8b8168780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:15.437 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.455+0000 7fd8bc94f640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd8b80d4e50 con 0x7fd8b815e480 2024-05-09T06:37:15.437 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.455+0000 7fd8bc94f640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fd8b80d3160 con 0x7fd8b8166390 2024-05-09T06:37:15.437 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.455+0000 7fd8bc94f640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fd8b80d0270 con 0x7fd8b80d27c0 2024-05-09T06:37:15.437 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.455+0000 7fd8b6d9d640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fd8b8166390 0x7fd8b8168780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:15.437 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.456+0000 7fd8b6d9d640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fd8b8166390 0x7fd8b8168780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51036/0 (socket says 172.21.15.3:51036) 2024-05-09T06:37:15.437 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.456+0000 7fd8b6d9d640 1 -- 172.21.15.3:0/1962961481 learned_addr learned my addr 172.21.15.3:0/1962961481 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:37:15.437 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.456+0000 7fd8b5d9b640 1 --2- 172.21.15.3:0/1962961481 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd8b80d27c0 0x7fd8b80d2c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:15.437 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.456+0000 7fd8b6d9d640 1 -- 172.21.15.3:0/1962961481 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd8b80d27c0 msgr2=0x7fd8b80d2c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:37:15.437 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.456+0000 7fd8b6d9d640 1 --2- 172.21.15.3:0/1962961481 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd8b80d27c0 0x7fd8b80d2c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:15.437 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.456+0000 7fd8b6d9d640 1 -- 172.21.15.3:0/1962961481 >> v1:172.21.15.190:6789/0 conn(0x7fd8b815e480 legacy=0x7fd8b80d1dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:37:15.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.456+0000 7fd8b6d9d640 1 -- 172.21.15.3:0/1962961481 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8b80cf1d0 con 0x7fd8b8166390 2024-05-09T06:37:15.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.457+0000 7fd8b6d9d640 1 --2- 172.21.15.3:0/1962961481 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd8b8166390 0x7fd8b8168780 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fd8a8009870 tx=0x7fd8a802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=185db0e47c9a6a32 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:15.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.457+0000 7fd8b559a640 1 -- 172.21.15.3:0/1962961481 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8a8037070 con 0x7fd8b8166390 2024-05-09T06:37:15.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.457+0000 7fd8b559a640 1 -- 172.21.15.3:0/1962961481 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd8a8032440 con 0x7fd8b8166390 2024-05-09T06:37:15.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.457+0000 7fd8b559a640 1 -- 172.21.15.3:0/1962961481 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8a8032810 con 0x7fd8b8166390 2024-05-09T06:37:15.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.457+0000 7fd8bc94f640 1 -- 172.21.15.3:0/1962961481 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd8b8166390 msgr2=0x7fd8b8168780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:15.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.457+0000 7fd8bc94f640 1 --2- 172.21.15.3:0/1962961481 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd8b8166390 0x7fd8b8168780 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fd8a8009870 tx=0x7fd8a802a270 comp rx=0 tx=0).stop 2024-05-09T06:37:15.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.457+0000 7fd8bc94f640 1 -- 172.21.15.3:0/1962961481 shutdown_connections 2024-05-09T06:37:15.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.457+0000 7fd8bc94f640 1 --2- 172.21.15.3:0/1962961481 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd8b8166390 0x7fd8b8168780 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:15.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.457+0000 7fd8bc94f640 1 --2- 172.21.15.3:0/1962961481 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd8b80d27c0 0x7fd8b80d2c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:15.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.457+0000 7fd8bc94f640 1 -- 172.21.15.3:0/1962961481 >> 172.21.15.3:0/1962961481 conn(0x7fd8b809d4d0 msgr2=0x7fd8b80d4200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:15.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.457+0000 7fd8bc94f640 1 -- 172.21.15.3:0/1962961481 shutdown_connections 2024-05-09T06:37:15.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.457+0000 7fd8bc94f640 1 -- 172.21.15.3:0/1962961481 wait complete. 2024-05-09T06:37:15.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.459+0000 7fd8bc94f640 1 Processor -- start 2024-05-09T06:37:15.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.459+0000 7fd8bc94f640 1 -- start start 2024-05-09T06:37:15.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.459+0000 7fd8bc94f640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd8b80d27c0 0x7fd8b80b8d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:15.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.459+0000 7fd8bc94f640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd8b815e480 0x7fd8b80ca6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:15.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.459+0000 7fd8bc94f640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd8b8166390 0x7fd8b80cbec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:15.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.460+0000 7fd8bc94f640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fd8b80cfa70 con 0x7fd8b815e480 2024-05-09T06:37:15.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.460+0000 7fd8bc94f640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fd8b80cf790 con 0x7fd8b8166390 2024-05-09T06:37:15.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.460+0000 7fd8bc94f640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fd8b80cfd50 con 0x7fd8b80d27c0 2024-05-09T06:37:15.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.460+0000 7fd8b5d9b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd8b815e480 0x7fd8b80ca6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:15.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.460+0000 7fd8b5d9b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd8b815e480 0x7fd8b80ca6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51048/0 (socket says 172.21.15.3:51048) 2024-05-09T06:37:15.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.460+0000 7fd8b5d9b640 1 -- 172.21.15.3:0/2257430363 learned_addr learned my addr 172.21.15.3:0/2257430363 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:37:15.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.460+0000 7fd8b659c640 1 --2- 172.21.15.3:0/2257430363 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd8b80d27c0 0x7fd8b80b8d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:15.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.460+0000 7fd8b6d9d640 1 --2- 172.21.15.3:0/2257430363 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd8b8166390 0x7fd8b80cbec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:15.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.460+0000 7fd8b5d9b640 1 -- 172.21.15.3:0/2257430363 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd8b80d27c0 msgr2=0x7fd8b80b8d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:15.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.460+0000 7fd8b5d9b640 1 --2- 172.21.15.3:0/2257430363 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd8b80d27c0 0x7fd8b80b8d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:15.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.460+0000 7fd8b5d9b640 1 -- 172.21.15.3:0/2257430363 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd8b8166390 msgr2=0x7fd8b80cbec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:15.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.460+0000 7fd8b5d9b640 1 --2- 172.21.15.3:0/2257430363 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd8b8166390 0x7fd8b80cbec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:15.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.460+0000 7fd8b5d9b640 1 -- 172.21.15.3:0/2257430363 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8b80c7f60 con 0x7fd8b815e480 2024-05-09T06:37:15.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.460+0000 7fd8b6d9d640 1 --2- 172.21.15.3:0/2257430363 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd8b8166390 0x7fd8b80cbec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:37:15.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.460+0000 7fd8b5d9b640 1 --2- 172.21.15.3:0/2257430363 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd8b815e480 0x7fd8b80ca6a0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fd8a000d5d0 tx=0x7fd8a000da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:15.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.460+0000 7fd8aeffd640 1 -- 172.21.15.3:0/2257430363 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8a0004240 con 0x7fd8b815e480 2024-05-09T06:37:15.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.460+0000 7fd8aeffd640 1 -- 172.21.15.3:0/2257430363 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd8a00043e0 con 0x7fd8b815e480 2024-05-09T06:37:15.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.460+0000 7fd8bc94f640 1 -- 172.21.15.3:0/2257430363 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8b80c8240 con 0x7fd8b815e480 2024-05-09T06:37:15.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.460+0000 7fd8aeffd640 1 -- 172.21.15.3:0/2257430363 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8a0003830 con 0x7fd8b815e480 2024-05-09T06:37:15.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.460+0000 7fd8bc94f640 1 -- 172.21.15.3:0/2257430363 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8b80b9290 con 0x7fd8b815e480 2024-05-09T06:37:15.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.461+0000 7fd8bc94f640 1 -- 172.21.15.3:0/2257430363 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8b804f660 con 0x7fd8b815e480 2024-05-09T06:37:15.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.462+0000 7fd8aeffd640 1 -- 172.21.15.3:0/2257430363 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 7) ==== 90651+0+0 (secure 0 0 0) 0x7fd8a00039d0 con 0x7fd8b815e480 2024-05-09T06:37:15.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.462+0000 7fd8aeffd640 1 --2- 172.21.15.3:0/2257430363 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fd89806c010 0x7fd89806e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:15.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.462+0000 7fd8aeffd640 1 -- 172.21.15.3:0/2257430363 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fd8a008b3c0 con 0x7fd8b815e480 2024-05-09T06:37:15.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.463+0000 7fd8b659c640 1 --2- 172.21.15.3:0/2257430363 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fd89806c010 0x7fd89806e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:15.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.463+0000 7fd8b659c640 1 --2- 172.21.15.3:0/2257430363 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fd89806c010 0x7fd89806e4d0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fd8b80ba270 tx=0x7fd8b0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:15.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.467+0000 7fd8aeffd640 1 -- 172.21.15.3:0/2257430363 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fd8a0059ae0 con 0x7fd8b815e480 2024-05-09T06:37:15.632 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.652+0000 7fd8bc94f640 1 -- 172.21.15.3:0/2257430363 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd8b80d18f0 con 0x7fd8b815e480 2024-05-09T06:37:15.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.653+0000 7fd8aeffd640 1 -- 172.21.15.3:0/2257430363 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) ==== 74+0+16585 (secure 0 0 0) 0x7fd8a00593e0 con 0x7fd8b815e480 2024-05-09T06:37:15.634 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-09T06:37:15.634 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":33,"fsid":"16ed737d-2665-431a-8ac6-ef242be3975a","created":"2024-05-09T06:35:42.541368+0000","modified":"2024-05-09T06:36:59.564366+0000","last_up_change":"2024-05-09T06:35:53.778274+0000","last_in_change":"2024-05-09T06:35:44.780586+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T06:35:54.819999+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-05-09T06:36:56.494700+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-05-09T06:36:56.900006+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fae3a8db-4826-4f3c-b33a-12c09bb5df6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":1897849827},{"type":"v1","addr":"172.21.15.3:6817","nonce":1897849827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":1897849827},{"type":"v1","addr":"172.21.15.3:6819","nonce":1897849827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":1897849827},{"type":"v1","addr":"172.21.15.3:6823","nonce":1897849827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":1897849827},{"type":"v1","addr":"172.21.15.3:6821","nonce":1897849827}]},"public_addr":"172.21.15.3:6817/1897849827","cluster_addr":"172.21.15.3:6819/1897849827","heartbeat_back_addr":"172.21.15.3:6823/1897849827","heartbeat_front_addr":"172.21.15.3:6821/1897849827","state":["exists","up"]},{"osd":1,"uuid":"646e882d-4ce4-4720-8c84-f71df563a35e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":926805498},{"type":"v1","addr":"172.21.15.3:6801","nonce":926805498}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":926805498},{"type":"v1","addr":"172.21.15.3:6803","nonce":926805498}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":926805498},{"type":"v1","addr":"172.21.15.3:6807","nonce":926805498}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":926805498},{"type":"v1","addr":"172.21.15.3:6805","nonce":926805498}]},"public_addr":"172.21.15.3:6801/926805498","cluster_addr":"172.21.15.3:6803/926805498","heartbeat_back_addr":"172.21.15.3:6807/926805498","heartbeat_front_addr":"172.21.15.3:6805/926805498","state":["exists","up"]},{"osd":2,"uuid":"1abbadb5-a570-4359-9643-acff3ede35a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":801295451},{"type":"v1","addr":"172.21.15.3:6809","nonce":801295451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":801295451},{"type":"v1","addr":"172.21.15.3:6811","nonce":801295451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":801295451},{"type":"v1","addr":"172.21.15.3:6815","nonce":801295451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":801295451},{"type":"v1","addr":"172.21.15.3:6813","nonce":801295451}]},"public_addr":"172.21.15.3:6809/801295451","cluster_addr":"172.21.15.3:6811/801295451","heartbeat_back_addr":"172.21.15.3:6815/801295451","heartbeat_front_addr":"172.21.15.3:6813/801295451","state":["exists","up"]},{"osd":3,"uuid":"62ac4dc1-2371-4dd2-acd3-e2dbb11aedf5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":1903389857},{"type":"v1","addr":"172.21.15.3:6825","nonce":1903389857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":1903389857},{"type":"v1","addr":"172.21.15.3:6827","nonce":1903389857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":1903389857},{"type":"v1","addr":"172.21.15.3:6831","nonce":1903389857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":1903389857},{"type":"v1","addr":"172.21.15.3:6829","nonce":1903389857}]},"public_addr":"172.21.15.3:6825/1903389857","cluster_addr":"172.21.15.3:6827/1903389857","heartbeat_back_addr":"172.21.15.3:6831/1903389857","heartbeat_front_addr":"172.21.15.3:6829/1903389857","state":["exists","up"]},{"osd":4,"uuid":"ae808398-8326-4d7c-9319-a2a0eafffb59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":2143624576},{"type":"v1","addr":"172.21.15.190:6801","nonce":2143624576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":2143624576},{"type":"v1","addr":"172.21.15.190:6803","nonce":2143624576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":2143624576},{"type":"v1","addr":"172.21.15.190:6807","nonce":2143624576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":2143624576},{"type":"v1","addr":"172.21.15.190:6805","nonce":2143624576}]},"public_addr":"172.21.15.190:6801/2143624576","cluster_addr":"172.21.15.190:6803/2143624576","heartbeat_back_addr":"172.21.15.190:6807/2143624576","heartbeat_front_addr":"172.21.15.190:6805/2143624576","state":["exists","up"]},{"osd":5,"uuid":"918adf50-c440-48ac-9a8d-4837562d7275","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":1766923126},{"type":"v1","addr":"172.21.15.190:6814","nonce":1766923126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":1766923126},{"type":"v1","addr":"172.21.15.190:6818","nonce":1766923126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":1766923126},{"type":"v1","addr":"172.21.15.190:6823","nonce":1766923126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":1766923126},{"type":"v1","addr":"172.21.15.190:6821","nonce":1766923126}]},"public_addr":"172.21.15.190:6814/1766923126","cluster_addr":"172.21.15.190:6818/1766923126","heartbeat_back_addr":"172.21.15.190:6823/1766923126","heartbeat_front_addr":"172.21.15.190:6821/1766923126","state":["exists","up"]},{"osd":6,"uuid":"ac2882bc-2a7c-463c-8f6c-76ff49828adc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":342414248},{"type":"v1","addr":"172.21.15.190:6825","nonce":342414248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":342414248},{"type":"v1","addr":"172.21.15.190:6827","nonce":342414248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":342414248},{"type":"v1","addr":"172.21.15.190:6831","nonce":342414248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":342414248},{"type":"v1","addr":"172.21.15.190:6829","nonce":342414248}]},"public_addr":"172.21.15.190:6825/342414248","cluster_addr":"172.21.15.190:6827/342414248","heartbeat_back_addr":"172.21.15.190:6831/342414248","heartbeat_front_addr":"172.21.15.190:6829/342414248","state":["exists","up"]},{"osd":7,"uuid":"dd4e767a-4730-4c70-8bd5-3f088140a5a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":537849891},{"type":"v1","addr":"172.21.15.190:6809","nonce":537849891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":537849891},{"type":"v1","addr":"172.21.15.190:6811","nonce":537849891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6817","nonce":537849891},{"type":"v1","addr":"172.21.15.190:6819","nonce":537849891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6813","nonce":537849891},{"type":"v1","addr":"172.21.15.190:6815","nonce":537849891}]},"public_addr":"172.21.15.190:6809/537849891","cluster_addr":"172.21.15.190:6811/537849891","heartbeat_back_addr":"172.21.15.190:6819/537849891","heartbeat_front_addr":"172.21.15.190:6815/537849891","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.889784+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.974002+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.973549+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.955396+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.928471+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.902418+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.958696+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.926474+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T06:37:15.636 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.655+0000 7fd8bc94f640 1 -- 172.21.15.3:0/2257430363 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fd89806c010 msgr2=0x7fd89806e4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:15.636 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.655+0000 7fd8bc94f640 1 --2- 172.21.15.3:0/2257430363 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fd89806c010 0x7fd89806e4d0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fd8b80ba270 tx=0x7fd8b0009040 comp rx=0 tx=0).stop 2024-05-09T06:37:15.636 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.655+0000 7fd8bc94f640 1 -- 172.21.15.3:0/2257430363 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd8b815e480 msgr2=0x7fd8b80ca6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:15.636 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.655+0000 7fd8bc94f640 1 --2- 172.21.15.3:0/2257430363 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd8b815e480 0x7fd8b80ca6a0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fd8a000d5d0 tx=0x7fd8a000da90 comp rx=0 tx=0).stop 2024-05-09T06:37:15.636 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.655+0000 7fd8bc94f640 1 -- 172.21.15.3:0/2257430363 shutdown_connections 2024-05-09T06:37:15.636 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.655+0000 7fd8bc94f640 1 --2- 172.21.15.3:0/2257430363 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fd89806c010 0x7fd89806e4d0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:15.636 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.655+0000 7fd8bc94f640 1 --2- 172.21.15.3:0/2257430363 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd8b8166390 0x7fd8b80cbec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:15.636 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.655+0000 7fd8bc94f640 1 --2- 172.21.15.3:0/2257430363 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd8b815e480 0x7fd8b80ca6a0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:15.636 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.655+0000 7fd8bc94f640 1 --2- 172.21.15.3:0/2257430363 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd8b80d27c0 0x7fd8b80b8d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:15.636 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.655+0000 7fd8bc94f640 1 -- 172.21.15.3:0/2257430363 >> 172.21.15.3:0/2257430363 conn(0x7fd8b809d4d0 msgr2=0x7fd8b80b2fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:15.636 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.655+0000 7fd8bc94f640 1 -- 172.21.15.3:0/2257430363 shutdown_connections 2024-05-09T06:37:15.636 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.655+0000 7fd8bc94f640 1 -- 172.21.15.3:0/2257430363 wait complete. 2024-05-09T06:37:15.647 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-09T06:35:54.819999+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-09T06:36:56.494700+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '32', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2024-05-09T06:36:56.900006+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '33', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-09T06:37:15.647 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-09T06:37:15.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.774+0000 7fabeb8fc640 1 Processor -- start 2024-05-09T06:37:15.755 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.774+0000 7fabeb8fc640 1 -- start start 2024-05-09T06:37:15.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.775+0000 7fabeb8fc640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fabe4144a10 0x7fabe40cf8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:15.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.775+0000 7fabeb8fc640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fabe40b4130 0x7fabe40b45e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:15.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.775+0000 7fabeb8fc640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fabe4071860 con 0x7fabe4072900 2024-05-09T06:37:15.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.775+0000 7fabeb8fc640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fabe4072470 con 0x7fabe4144a10 2024-05-09T06:37:15.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.775+0000 7fabeb8fc640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7fabe4071b40 con 0x7fabe40b4130 2024-05-09T06:37:15.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.775+0000 7fabe9698640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fabe4144a10 0x7fabe40cf8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:15.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.775+0000 7fabe9698640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fabe4144a10 0x7fabe40cf8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51064/0 (socket says 172.21.15.3:51064) 2024-05-09T06:37:15.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.775+0000 7fabe9698640 1 -- 172.21.15.3:0/3420207517 learned_addr learned my addr 172.21.15.3:0/3420207517 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:37:15.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.775+0000 7fabe9e99640 1 --2- 172.21.15.3:0/3420207517 >> v2:172.21.15.190:3300/0 conn(0x7fabe40b4130 0x7fabe40b45e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:15.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.775+0000 7fabe9698640 1 -- 172.21.15.3:0/3420207517 >> v2:172.21.15.190:3300/0 conn(0x7fabe40b4130 msgr2=0x7fabe40b45e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:37:15.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.775+0000 7fabe9698640 1 --2- 172.21.15.3:0/3420207517 >> v2:172.21.15.190:3300/0 conn(0x7fabe40b4130 0x7fabe40b45e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:15.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.775+0000 7fabe9698640 1 -- 172.21.15.3:0/3420207517 >> v1:172.21.15.190:6789/0 conn(0x7fabe4072900 legacy=0x7fabe4071080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:37:15.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.775+0000 7fabe9698640 1 -- 172.21.15.3:0/3420207517 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabe4071e20 con 0x7fabe4144a10 2024-05-09T06:37:15.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.775+0000 7fabe9e99640 1 --2- 172.21.15.3:0/3420207517 >> v2:172.21.15.190:3300/0 conn(0x7fabe40b4130 0x7fabe40b45e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:37:15.758 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.776+0000 7fabe9698640 1 --2- 172.21.15.3:0/3420207517 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fabe4144a10 0x7fabe40cf8a0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fabe40b2fb0 tx=0x7fabe002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6232b5945595aac server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:15.758 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.776+0000 7fabdbfff640 1 -- 172.21.15.3:0/3420207517 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabe0037070 con 0x7fabe4144a10 2024-05-09T06:37:15.759 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.776+0000 7fabdbfff640 1 -- 172.21.15.3:0/3420207517 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fabe0032440 con 0x7fabe4144a10 2024-05-09T06:37:15.759 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.776+0000 7fabdbfff640 1 -- 172.21.15.3:0/3420207517 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabe0032810 con 0x7fabe4144a10 2024-05-09T06:37:15.759 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.777+0000 7fabeb8fc640 1 -- 172.21.15.3:0/3420207517 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fabe4144a10 msgr2=0x7fabe40cf8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:15.759 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.777+0000 7fabeb8fc640 1 --2- 172.21.15.3:0/3420207517 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fabe4144a10 0x7fabe40cf8a0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fabe40b2fb0 tx=0x7fabe002a1a0 comp rx=0 tx=0).stop 2024-05-09T06:37:15.759 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.777+0000 7fabeb8fc640 1 -- 172.21.15.3:0/3420207517 shutdown_connections 2024-05-09T06:37:15.759 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.777+0000 7fabeb8fc640 1 --2- 172.21.15.3:0/3420207517 >> v2:172.21.15.190:3300/0 conn(0x7fabe40b4130 0x7fabe40b45e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:15.759 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.777+0000 7fabeb8fc640 1 --2- 172.21.15.3:0/3420207517 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fabe4144a10 0x7fabe40cf8a0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:15.759 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.777+0000 7fabeb8fc640 1 -- 172.21.15.3:0/3420207517 >> 172.21.15.3:0/3420207517 conn(0x7fabe40b1c20 msgr2=0x7fabe40b2030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:15.759 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.777+0000 7fabeb8fc640 1 -- 172.21.15.3:0/3420207517 shutdown_connections 2024-05-09T06:37:15.759 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.777+0000 7fabeb8fc640 1 -- 172.21.15.3:0/3420207517 wait complete. 2024-05-09T06:37:15.759 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.778+0000 7fabeb8fc640 1 Processor -- start 2024-05-09T06:37:15.759 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.778+0000 7fabeb8fc640 1 -- start start 2024-05-09T06:37:15.761 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.779+0000 7fabeb8fc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fabe4072900 0x7fabe40aeaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:15.761 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.779+0000 7fabeb8fc640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fabe40b4130 0x7fabe41f22b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:15.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.779+0000 7fabeb8fc640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fabe4144a10 0x7fabe413b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:15.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.779+0000 7fabeb8fc640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fabe415b750 con 0x7fabe4072900 2024-05-09T06:37:15.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.779+0000 7fabeb8fc640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fabe415b5d0 con 0x7fabe40b4130 2024-05-09T06:37:15.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.779+0000 7fabeb8fc640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fabe415b8d0 con 0x7fabe4144a10 2024-05-09T06:37:15.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.779+0000 7fabe9698640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fabe4072900 0x7fabe40aeaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:15.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.779+0000 7fabe9698640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fabe4072900 0x7fabe40aeaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51068/0 (socket says 172.21.15.3:51068) 2024-05-09T06:37:15.763 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.779+0000 7fabe9698640 1 -- 172.21.15.3:0/3317864475 learned_addr learned my addr 172.21.15.3:0/3317864475 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:37:15.763 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.779+0000 7fabe8e97640 1 --2- 172.21.15.3:0/3317864475 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fabe40b4130 0x7fabe41f22b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:15.763 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.779+0000 7fabe9e99640 1 --2- 172.21.15.3:0/3317864475 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fabe4144a10 0x7fabe413b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:15.763 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.779+0000 7fabe9698640 1 -- 172.21.15.3:0/3317864475 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fabe4144a10 msgr2=0x7fabe413b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:15.763 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.779+0000 7fabe9698640 1 --2- 172.21.15.3:0/3317864475 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fabe4144a10 0x7fabe413b0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:15.763 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.779+0000 7fabe9698640 1 -- 172.21.15.3:0/3317864475 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fabe40b4130 msgr2=0x7fabe41f22b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:15.763 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.779+0000 7fabe9698640 1 --2- 172.21.15.3:0/3317864475 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fabe40b4130 0x7fabe41f22b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:15.763 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.779+0000 7fabe9698640 1 -- 172.21.15.3:0/3317864475 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabe413b5e0 con 0x7fabe4072900 2024-05-09T06:37:15.763 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.779+0000 7fabe9e99640 1 --2- 172.21.15.3:0/3317864475 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fabe4144a10 0x7fabe413b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:37:15.763 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.779+0000 7fabe9698640 1 --2- 172.21.15.3:0/3317864475 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fabe4072900 0x7fabe40aeaf0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fabe002a1e0 tx=0x7fabe00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:15.763 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.780+0000 7fabd9ffb640 1 -- 172.21.15.3:0/3317864475 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabe0041070 con 0x7fabe4072900 2024-05-09T06:37:15.763 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.780+0000 7fabd9ffb640 1 -- 172.21.15.3:0/3317864475 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fabe003c870 con 0x7fabe4072900 2024-05-09T06:37:15.763 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.780+0000 7fabeb8fc640 1 -- 172.21.15.3:0/3317864475 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fabe40af0d0 con 0x7fabe4072900 2024-05-09T06:37:15.763 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.780+0000 7fabd9ffb640 1 -- 172.21.15.3:0/3317864475 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabe0037040 con 0x7fabe4072900 2024-05-09T06:37:15.764 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.780+0000 7fabeb8fc640 1 -- 172.21.15.3:0/3317864475 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fabe413bcb0 con 0x7fabe4072900 2024-05-09T06:37:15.764 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.781+0000 7fabeb8fc640 1 -- 172.21.15.3:0/3317864475 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fabe4069550 con 0x7fabe4072900 2024-05-09T06:37:15.764 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.781+0000 7fabd9ffb640 1 -- 172.21.15.3:0/3317864475 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 7) ==== 90651+0+0 (secure 0 0 0) 0x7fabe0044070 con 0x7fabe4072900 2024-05-09T06:37:15.765 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.782+0000 7fabd9ffb640 1 --2- 172.21.15.3:0/3317864475 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fabc806bfe0 0x7fabc806e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:15.765 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.782+0000 7fabd9ffb640 1 -- 172.21.15.3:0/3317864475 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fabe00abcf0 con 0x7fabe4072900 2024-05-09T06:37:15.765 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.782+0000 7fabe8e97640 1 --2- 172.21.15.3:0/3317864475 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fabc806bfe0 0x7fabc806e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:15.765 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.783+0000 7fabe8e97640 1 --2- 172.21.15.3:0/3317864475 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fabc806bfe0 0x7fabc806e4a0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fabe415dbe0 tx=0x7fabd4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:15.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.786+0000 7fabd9ffb640 1 -- 172.21.15.3:0/3317864475 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fabe007a410 con 0x7fabe4072900 2024-05-09T06:37:15.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.972+0000 7fabeb8fc640 1 -- 172.21.15.3:0/3317864475 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fabe4068380 con 0x7fabe4072900 2024-05-09T06:37:15.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.973+0000 7fabd9ffb640 1 -- 172.21.15.3:0/3317864475 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v33) ==== 93+0+10 (secure 0 0 0) 0x7fabe0079d10 con 0x7fabe4072900 2024-05-09T06:37:15.954 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 1 2024-05-09T06:37:15.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.975+0000 7fabeb8fc640 1 -- 172.21.15.3:0/3317864475 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fabc806bfe0 msgr2=0x7fabc806e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:15.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.975+0000 7fabeb8fc640 1 --2- 172.21.15.3:0/3317864475 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fabc806bfe0 0x7fabc806e4a0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fabe415dbe0 tx=0x7fabd4009040 comp rx=0 tx=0).stop 2024-05-09T06:37:15.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.975+0000 7fabeb8fc640 1 -- 172.21.15.3:0/3317864475 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fabe4072900 msgr2=0x7fabe40aeaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:15.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.975+0000 7fabeb8fc640 1 --2- 172.21.15.3:0/3317864475 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fabe4072900 0x7fabe40aeaf0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fabe002a1e0 tx=0x7fabe00046c0 comp rx=0 tx=0).stop 2024-05-09T06:37:15.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.975+0000 7fabeb8fc640 1 -- 172.21.15.3:0/3317864475 shutdown_connections 2024-05-09T06:37:15.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.975+0000 7fabeb8fc640 1 --2- 172.21.15.3:0/3317864475 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fabc806bfe0 0x7fabc806e4a0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:15.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.975+0000 7fabeb8fc640 1 --2- 172.21.15.3:0/3317864475 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fabe4144a10 0x7fabe413b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:15.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.975+0000 7fabeb8fc640 1 --2- 172.21.15.3:0/3317864475 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fabe40b4130 0x7fabe41f22b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:15.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.975+0000 7fabeb8fc640 1 --2- 172.21.15.3:0/3317864475 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fabe4072900 0x7fabe40aeaf0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:15.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.975+0000 7fabeb8fc640 1 -- 172.21.15.3:0/3317864475 >> 172.21.15.3:0/3317864475 conn(0x7fabe40b1c20 msgr2=0x7fabe4141f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:15.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.975+0000 7fabeb8fc640 1 -- 172.21.15.3:0/3317864475 shutdown_connections 2024-05-09T06:37:15.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:15.975+0000 7fabeb8fc640 1 -- 172.21.15.3:0/3317864475 wait complete. 2024-05-09T06:37:15.966 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-09T06:37:16.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.092+0000 7f2f3dbdc640 1 Processor -- start 2024-05-09T06:37:16.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.092+0000 7f2f3dbdc640 1 -- start start 2024-05-09T06:37:16.076 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.093+0000 7f2f3dbdc640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2f38144a10 0x7f2f380cf8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:16.076 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.093+0000 7f2f3dbdc640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2f38071860 con 0x7f2f380b4130 2024-05-09T06:37:16.076 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.093+0000 7f2f3dbdc640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2f38071b40 con 0x7f2f38072900 2024-05-09T06:37:16.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.093+0000 7f2f3dbdc640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f2f38072470 con 0x7f2f38144a10 2024-05-09T06:37:16.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.093+0000 7f2f37fff640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f2f380b4130 0x7f2f3815b5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:45944/0 (socket says 172.21.15.3:45944) 2024-05-09T06:37:16.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.093+0000 7f2f37fff640 1 -- 172.21.15.3:0/3958977368 learned_addr learned my addr 172.21.15.3:0/3958977368 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-05-09T06:37:16.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.093+0000 7f2f377fe640 1 --2- 172.21.15.3:0/3958977368 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2f38144a10 0x7f2f380cf8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:16.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.093+0000 7f2f367fc640 1 -- 172.21.15.3:0/3958977368 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3364419899 0 0) 0x7f2f38071860 con 0x7f2f380b4130 2024-05-09T06:37:16.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.093+0000 7f2f367fc640 1 -- 172.21.15.3:0/3958977368 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2f20003520 con 0x7f2f380b4130 2024-05-09T06:37:16.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.094+0000 7f2f367fc640 1 -- 172.21.15.3:0/3958977368 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 205085270 0 0) 0x7f2f20003520 con 0x7f2f380b4130 2024-05-09T06:37:16.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.094+0000 7f2f367fc640 1 -- 172.21.15.3:0/3958977368 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2f38144a10 msgr2=0x7f2f380cf8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:37:16.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.094+0000 7f2f367fc640 1 --2- 172.21.15.3:0/3958977368 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2f38144a10 0x7f2f380cf8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:16.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.094+0000 7f2f367fc640 1 -- 172.21.15.3:0/3958977368 >> v1:172.21.15.190:6789/0 conn(0x7f2f38072900 legacy=0x7f2f38071080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:16.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.094+0000 7f2f367fc640 1 -- 172.21.15.3:0/3958977368 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f38071e20 con 0x7f2f380b4130 2024-05-09T06:37:16.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.094+0000 7f2f367fc640 1 -- 172.21.15.3:0/3958977368 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 563312031 0 0) 0x7f2f300019f0 con 0x7f2f380b4130 2024-05-09T06:37:16.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.094+0000 7f2f367fc640 1 -- 172.21.15.3:0/3958977368 >> v1:172.21.15.3:6789/0 conn(0x7f2f380b4130 legacy=0x7f2f3815b5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:16.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.094+0000 7f2f367fc640 1 --2- 172.21.15.3:0/3958977368 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2f38144a10 0x7f2f20003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:16.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.094+0000 7f2f367fc640 1 --2- 172.21.15.3:0/3958977368 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2f20003f80 0x7f2f20004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:16.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.094+0000 7f2f367fc640 1 --2- 172.21.15.3:0/3958977368 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2f20004d40 0x7f2f200051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:16.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.094+0000 7f2f367fc640 1 -- 172.21.15.3:0/3958977368 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f2f38072470 con 0x7f2f20003f80 2024-05-09T06:37:16.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.094+0000 7f2f367fc640 1 -- 172.21.15.3:0/3958977368 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f2f2000bb80 con 0x7f2f20004d40 2024-05-09T06:37:16.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.094+0000 7f2f367fc640 1 -- 172.21.15.3:0/3958977368 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f2f2000bd80 con 0x7f2f38144a10 2024-05-09T06:37:16.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.094+0000 7f2f36ffd640 1 --2- 172.21.15.3:0/3958977368 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2f20003f80 0x7f2f20004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:16.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.094+0000 7f2f377fe640 1 --2- 172.21.15.3:0/3958977368 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2f38144a10 0x7f2f20003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:16.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.094+0000 7f2f37fff640 1 --2- 172.21.15.3:0/3958977368 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2f20004d40 0x7f2f200051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:16.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.094+0000 7f2f36ffd640 1 -- 172.21.15.3:0/3958977368 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2f38144a10 msgr2=0x7f2f20003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:37:16.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.094+0000 7f2f36ffd640 1 --2- 172.21.15.3:0/3958977368 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2f38144a10 0x7f2f20003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:16.079 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.094+0000 7f2f36ffd640 1 -- 172.21.15.3:0/3958977368 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2f20004d40 msgr2=0x7f2f200051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:37:16.079 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.094+0000 7f2f36ffd640 1 --2- 172.21.15.3:0/3958977368 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2f20004d40 0x7f2f200051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:16.079 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.094+0000 7f2f36ffd640 1 -- 172.21.15.3:0/3958977368 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f20003520 con 0x7f2f20003f80 2024-05-09T06:37:16.079 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.095+0000 7f2f36ffd640 1 --2- 172.21.15.3:0/3958977368 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2f20003f80 0x7f2f20004800 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f2f2c001330 tx=0x7f2f20005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfd249ed0ea0b91b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:16.079 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.095+0000 7f2f367fc640 1 -- 172.21.15.3:0/3958977368 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f2c001370 con 0x7f2f20003f80 2024-05-09T06:37:16.079 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.095+0000 7f2f367fc640 1 -- 172.21.15.3:0/3958977368 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f2c02cd90 con 0x7f2f20003f80 2024-05-09T06:37:16.079 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.095+0000 7f2f367fc640 1 -- 172.21.15.3:0/3958977368 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f2c005310 con 0x7f2f20003f80 2024-05-09T06:37:16.079 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.095+0000 7f2f3dbdc640 1 -- 172.21.15.3:0/3958977368 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2f20003f80 msgr2=0x7f2f20004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:16.079 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.095+0000 7f2f3dbdc640 1 --2- 172.21.15.3:0/3958977368 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2f20003f80 0x7f2f20004800 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f2f2c001330 tx=0x7f2f20005830 comp rx=0 tx=0).stop 2024-05-09T06:37:16.079 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.096+0000 7f2f37fff640 1 -- 172.21.15.3:0/3958977368 reap_dead start 2024-05-09T06:37:16.079 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.096+0000 7f2f3dbdc640 1 -- 172.21.15.3:0/3958977368 shutdown_connections 2024-05-09T06:37:16.079 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.096+0000 7f2f3dbdc640 1 -- 172.21.15.3:0/3958977368 >> 172.21.15.3:0/3958977368 conn(0x7f2f380b1c20 msgr2=0x7f2f380b2030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:16.079 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.096+0000 7f2f3dbdc640 1 -- 172.21.15.3:0/3958977368 shutdown_connections 2024-05-09T06:37:16.079 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.096+0000 7f2f3dbdc640 1 -- 172.21.15.3:0/3958977368 wait complete. 2024-05-09T06:37:16.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.097+0000 7f2f3dbdc640 1 Processor -- start 2024-05-09T06:37:16.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.097+0000 7f2f3dbdc640 1 -- start start 2024-05-09T06:37:16.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.098+0000 7f2f3dbdc640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2f20003f80 0x7f2f3815a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:16.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.098+0000 7f2f3dbdc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2f3813bab0 0x7f2f3813bf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:16.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.098+0000 7f2f3dbdc640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2f3815d4b0 0x7f2f3815d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:16.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.098+0000 7f2f3dbdc640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f2f380cfa20 con 0x7f2f3813bab0 2024-05-09T06:37:16.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.098+0000 7f2f3dbdc640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f2f380cf8a0 con 0x7f2f20003f80 2024-05-09T06:37:16.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.098+0000 7f2f3dbdc640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f2f380cfba0 con 0x7f2f3815d4b0 2024-05-09T06:37:16.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.098+0000 7f2f36ffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2f3813bab0 0x7f2f3813bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:16.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.098+0000 7f2f36ffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2f3813bab0 0x7f2f3813bf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51084/0 (socket says 172.21.15.3:51084) 2024-05-09T06:37:16.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.098+0000 7f2f36ffd640 1 -- 172.21.15.3:0/808323157 learned_addr learned my addr 172.21.15.3:0/808323157 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:37:16.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.098+0000 7f2f377fe640 1 --2- 172.21.15.3:0/808323157 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2f20003f80 0x7f2f3815a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:16.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.098+0000 7f2f37fff640 1 --2- 172.21.15.3:0/808323157 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2f3815d4b0 0x7f2f3815d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:16.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.098+0000 7f2f36ffd640 1 -- 172.21.15.3:0/808323157 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2f3815d4b0 msgr2=0x7f2f3815d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:16.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.098+0000 7f2f36ffd640 1 --2- 172.21.15.3:0/808323157 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2f3815d4b0 0x7f2f3815d930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:16.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.098+0000 7f2f36ffd640 1 -- 172.21.15.3:0/808323157 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2f20003f80 msgr2=0x7f2f3815a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:16.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.098+0000 7f2f36ffd640 1 --2- 172.21.15.3:0/808323157 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2f20003f80 0x7f2f3815a830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:16.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.098+0000 7f2f36ffd640 1 -- 172.21.15.3:0/808323157 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f381f2fd0 con 0x7f2f3813bab0 2024-05-09T06:37:16.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.098+0000 7f2f36ffd640 1 --2- 172.21.15.3:0/808323157 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2f3813bab0 0x7f2f3813bf30 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f2f2c001810 tx=0x7f2f2c001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:16.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.099+0000 7f2f1ffff640 1 -- 172.21.15.3:0/808323157 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f2c041030 con 0x7f2f3813bab0 2024-05-09T06:37:16.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.099+0000 7f2f1ffff640 1 -- 172.21.15.3:0/808323157 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f2c02cb90 con 0x7f2f3813bab0 2024-05-09T06:37:16.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.099+0000 7f2f3dbdc640 1 -- 172.21.15.3:0/808323157 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f381f32b0 con 0x7f2f3813bab0 2024-05-09T06:37:16.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.099+0000 7f2f1ffff640 1 -- 172.21.15.3:0/808323157 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f2c039040 con 0x7f2f3813bab0 2024-05-09T06:37:16.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.099+0000 7f2f3dbdc640 1 -- 172.21.15.3:0/808323157 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f381f3870 con 0x7f2f3813bab0 2024-05-09T06:37:16.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.100+0000 7f2f3dbdc640 1 -- 172.21.15.3:0/808323157 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f380b45e0 con 0x7f2f3813bab0 2024-05-09T06:37:16.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.100+0000 7f2f1ffff640 1 -- 172.21.15.3:0/808323157 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 7) ==== 90651+0+0 (secure 0 0 0) 0x7f2f2c044070 con 0x7f2f3813bab0 2024-05-09T06:37:16.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.101+0000 7f2f1ffff640 1 --2- 172.21.15.3:0/808323157 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f2f2406bfe0 0x7f2f2406e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:16.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.101+0000 7f2f1ffff640 1 -- 172.21.15.3:0/808323157 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f2f2c0acc90 con 0x7f2f3813bab0 2024-05-09T06:37:16.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.101+0000 7f2f377fe640 1 --2- 172.21.15.3:0/808323157 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f2f2406bfe0 0x7f2f2406e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:16.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.101+0000 7f2f377fe640 1 --2- 172.21.15.3:0/808323157 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f2f2406bfe0 0x7f2f2406e4a0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f2f28009bc0 tx=0x7f2f28009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:16.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.105+0000 7f2f1ffff640 1 -- 172.21.15.3:0/808323157 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f2f380b45e0 con 0x7f2f3813bab0 2024-05-09T06:37:16.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.286+0000 7f2f3dbdc640 1 -- 172.21.15.3:0/808323157 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f2f38069550 con 0x7f2f3813bab0 2024-05-09T06:37:16.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.286+0000 7f2f1ffff640 1 -- 172.21.15.3:0/808323157 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v33) ==== 104+0+11 (secure 0 0 0) 0x7f2f2c07b480 con 0x7f2f3813bab0 2024-05-09T06:37:16.267 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 32 2024-05-09T06:37:16.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.288+0000 7f2f3dbdc640 1 -- 172.21.15.3:0/808323157 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f2f2406bfe0 msgr2=0x7f2f2406e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:16.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.288+0000 7f2f3dbdc640 1 --2- 172.21.15.3:0/808323157 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f2f2406bfe0 0x7f2f2406e4a0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f2f28009bc0 tx=0x7f2f28009190 comp rx=0 tx=0).stop 2024-05-09T06:37:16.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.288+0000 7f2f3dbdc640 1 -- 172.21.15.3:0/808323157 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2f3813bab0 msgr2=0x7f2f3813bf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:16.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.288+0000 7f2f3dbdc640 1 --2- 172.21.15.3:0/808323157 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2f3813bab0 0x7f2f3813bf30 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f2f2c001810 tx=0x7f2f2c001120 comp rx=0 tx=0).stop 2024-05-09T06:37:16.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.288+0000 7f2f3dbdc640 1 -- 172.21.15.3:0/808323157 shutdown_connections 2024-05-09T06:37:16.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.288+0000 7f2f3dbdc640 1 --2- 172.21.15.3:0/808323157 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f2f2406bfe0 0x7f2f2406e4a0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:16.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.288+0000 7f2f3dbdc640 1 --2- 172.21.15.3:0/808323157 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2f3815d4b0 0x7f2f3815d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:16.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.288+0000 7f2f3dbdc640 1 --2- 172.21.15.3:0/808323157 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2f3813bab0 0x7f2f3813bf30 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:16.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.288+0000 7f2f3dbdc640 1 --2- 172.21.15.3:0/808323157 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2f20003f80 0x7f2f3815a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:16.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.289+0000 7f2f3dbdc640 1 -- 172.21.15.3:0/808323157 >> 172.21.15.3:0/808323157 conn(0x7f2f380b1c20 msgr2=0x7f2f380cef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:16.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.289+0000 7f2f3dbdc640 1 -- 172.21.15.3:0/808323157 shutdown_connections 2024-05-09T06:37:16.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.289+0000 7f2f3dbdc640 1 -- 172.21.15.3:0/808323157 wait complete. 2024-05-09T06:37:16.280 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-09T06:37:16.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.407+0000 7fe4240df640 1 Processor -- start 2024-05-09T06:37:16.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.407+0000 7fe4240df640 1 -- start start 2024-05-09T06:37:16.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.408+0000 7fe4240df640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fe41c06cd70 0x7fe41c162a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:16.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.408+0000 7fe4240df640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe41c0cc750 0x7fe41c0ccbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:16.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.408+0000 7fe4240df640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe41c162fa0 con 0x7fe41c0d02b0 2024-05-09T06:37:16.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.408+0000 7fe4240df640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7fe41c161e40 con 0x7fe41c06cd70 2024-05-09T06:37:16.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.408+0000 7fe4240df640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fe41c0d0870 con 0x7fe41c0cc750 2024-05-09T06:37:16.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.408+0000 7fe421e7b640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fe41c06cd70 0x7fe41c162a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:16.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.408+0000 7fe42267c640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe41c0cc750 0x7fe41c0ccbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:16.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.408+0000 7fe421e7b640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fe41c06cd70 0x7fe41c162a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.3:43454/0 (socket says 172.21.15.3:43454) 2024-05-09T06:37:16.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.408+0000 7fe421e7b640 1 -- 172.21.15.3:0/481409560 learned_addr learned my addr 172.21.15.3:0/481409560 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:37:16.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.408+0000 7fe42267c640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe41c0cc750 0x7fe41c0ccbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.3:38984/0 (socket says 172.21.15.3:38984) 2024-05-09T06:37:16.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.409+0000 7fe42267c640 1 -- 172.21.15.3:0/481409560 >> v2:172.21.15.190:3300/0 conn(0x7fe41c06cd70 msgr2=0x7fe41c162a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:37:16.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.409+0000 7fe42267c640 1 --2- 172.21.15.3:0/481409560 >> v2:172.21.15.190:3300/0 conn(0x7fe41c06cd70 0x7fe41c162a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:16.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.409+0000 7fe42267c640 1 -- 172.21.15.3:0/481409560 >> v1:172.21.15.190:6789/0 conn(0x7fe41c0d02b0 legacy=0x7fe41c06d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:16.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.409+0000 7fe42267c640 1 -- 172.21.15.3:0/481409560 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe41c0d0b50 con 0x7fe41c0cc750 2024-05-09T06:37:16.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.409+0000 7fe421e7b640 1 --2- 172.21.15.3:0/481409560 >> v2:172.21.15.190:3300/0 conn(0x7fe41c06cd70 0x7fe41c162a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T06:37:16.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.409+0000 7fe42267c640 1 --2- 172.21.15.3:0/481409560 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe41c0cc750 0x7fe41c0ccbe0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fe40c009d90 tx=0x7fe40c02a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bf1d0b6c9b40e5bc server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:16.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.410+0000 7fe420e79640 1 -- 172.21.15.3:0/481409560 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe40c007740 con 0x7fe41c0cc750 2024-05-09T06:37:16.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.410+0000 7fe420e79640 1 -- 172.21.15.3:0/481409560 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe40c0078e0 con 0x7fe41c0cc750 2024-05-09T06:37:16.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.410+0000 7fe420e79640 1 -- 172.21.15.3:0/481409560 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe40c007cb0 con 0x7fe41c0cc750 2024-05-09T06:37:16.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.410+0000 7fe4240df640 1 -- 172.21.15.3:0/481409560 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe41c0cc750 msgr2=0x7fe41c0ccbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:16.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.410+0000 7fe4240df640 1 --2- 172.21.15.3:0/481409560 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe41c0cc750 0x7fe41c0ccbe0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fe40c009d90 tx=0x7fe40c02a660 comp rx=0 tx=0).stop 2024-05-09T06:37:16.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.410+0000 7fe4240df640 1 -- 172.21.15.3:0/481409560 shutdown_connections 2024-05-09T06:37:16.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.410+0000 7fe4240df640 1 --2- 172.21.15.3:0/481409560 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe41c0cc750 0x7fe41c0ccbe0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:16.393 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.410+0000 7fe4240df640 1 --2- 172.21.15.3:0/481409560 >> v2:172.21.15.190:3300/0 conn(0x7fe41c06cd70 0x7fe41c162a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:16.393 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.410+0000 7fe4240df640 1 -- 172.21.15.3:0/481409560 >> 172.21.15.3:0/481409560 conn(0x7fe41c0b3af0 msgr2=0x7fe41c0b3f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:16.393 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.410+0000 7fe4240df640 1 -- 172.21.15.3:0/481409560 shutdown_connections 2024-05-09T06:37:16.393 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.410+0000 7fe4240df640 1 -- 172.21.15.3:0/481409560 wait complete. 2024-05-09T06:37:16.393 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.411+0000 7fe4240df640 1 Processor -- start 2024-05-09T06:37:16.393 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.411+0000 7fe4240df640 1 -- start start 2024-05-09T06:37:16.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.412+0000 7fe4240df640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe41c06cd70 0x7fe41c1504f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:16.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.412+0000 7fe4240df640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe41c0cc750 0x7fe41c0c5bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:16.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.412+0000 7fe4240df640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe41c0d02b0 0x7fe41c1604a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:16.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.412+0000 7fe4240df640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fe41c0d13f0 con 0x7fe41c0cc750 2024-05-09T06:37:16.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.412+0000 7fe4240df640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fe41c0d1110 con 0x7fe41c06cd70 2024-05-09T06:37:16.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.412+0000 7fe4240df640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fe41c0cd120 con 0x7fe41c0d02b0 2024-05-09T06:37:16.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.412+0000 7fe42167a640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe41c0cc750 0x7fe41c0c5bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:16.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.412+0000 7fe42167a640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe41c0cc750 0x7fe41c0c5bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51096/0 (socket says 172.21.15.3:51096) 2024-05-09T06:37:16.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.412+0000 7fe42167a640 1 -- 172.21.15.3:0/387317008 learned_addr learned my addr 172.21.15.3:0/387317008 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:37:16.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.412+0000 7fe42267c640 1 --2- 172.21.15.3:0/387317008 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe41c0d02b0 0x7fe41c1604a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:16.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.412+0000 7fe421e7b640 1 --2- 172.21.15.3:0/387317008 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe41c06cd70 0x7fe41c1504f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:16.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.413+0000 7fe42167a640 1 -- 172.21.15.3:0/387317008 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe41c0d02b0 msgr2=0x7fe41c1604a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:16.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.413+0000 7fe42167a640 1 --2- 172.21.15.3:0/387317008 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe41c0d02b0 0x7fe41c1604a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:16.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.413+0000 7fe42167a640 1 -- 172.21.15.3:0/387317008 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe41c06cd70 msgr2=0x7fe41c1504f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:16.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.413+0000 7fe42167a640 1 --2- 172.21.15.3:0/387317008 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe41c06cd70 0x7fe41c1504f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:16.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.413+0000 7fe42167a640 1 -- 172.21.15.3:0/387317008 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe41c0b7470 con 0x7fe41c0cc750 2024-05-09T06:37:16.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.413+0000 7fe421e7b640 1 --2- 172.21.15.3:0/387317008 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe41c06cd70 0x7fe41c1504f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:37:16.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.413+0000 7fe42167a640 1 --2- 172.21.15.3:0/387317008 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe41c0cc750 0x7fe41c0c5bc0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fe41000c400 tx=0x7fe41000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:16.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.413+0000 7fe41a7fc640 1 -- 172.21.15.3:0/387317008 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe410017070 con 0x7fe41c0cc750 2024-05-09T06:37:16.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.413+0000 7fe41a7fc640 1 -- 172.21.15.3:0/387317008 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4100052d0 con 0x7fe41c0cc750 2024-05-09T06:37:16.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.413+0000 7fe4240df640 1 -- 172.21.15.3:0/387317008 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe41c0b7750 con 0x7fe41c0cc750 2024-05-09T06:37:16.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.413+0000 7fe41a7fc640 1 -- 172.21.15.3:0/387317008 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe410003830 con 0x7fe41c0cc750 2024-05-09T06:37:16.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.413+0000 7fe4240df640 1 -- 172.21.15.3:0/387317008 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe41c0c8780 con 0x7fe41c0cc750 2024-05-09T06:37:16.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.414+0000 7fe4240df640 1 -- 172.21.15.3:0/387317008 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe41c069550 con 0x7fe41c0cc750 2024-05-09T06:37:16.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.414+0000 7fe41a7fc640 1 -- 172.21.15.3:0/387317008 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 7) ==== 90651+0+0 (secure 0 0 0) 0x7fe41001a070 con 0x7fe41c0cc750 2024-05-09T06:37:16.398 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.415+0000 7fe41a7fc640 1 --2- 172.21.15.3:0/387317008 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fe40806bfe0 0x7fe40806e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:16.398 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.415+0000 7fe41a7fc640 1 -- 172.21.15.3:0/387317008 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fe41008a4d0 con 0x7fe41c0cc750 2024-05-09T06:37:16.398 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.415+0000 7fe421e7b640 1 --2- 172.21.15.3:0/387317008 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fe40806bfe0 0x7fe40806e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:16.398 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.416+0000 7fe421e7b640 1 --2- 172.21.15.3:0/387317008 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fe40806bfe0 0x7fe40806e4a0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fe414009630 tx=0x7fe414008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:16.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.419+0000 7fe41a7fc640 1 -- 172.21.15.3:0/387317008 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fe410058520 con 0x7fe41c0cc750 2024-05-09T06:37:16.586 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.605+0000 7fe4240df640 1 -- 172.21.15.3:0/387317008 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fe41c06cb50 con 0x7fe41c0cc750 2024-05-09T06:37:16.586 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.605+0000 7fe41a7fc640 1 -- 172.21.15.3:0/387317008 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v33) ==== 100+0+11 (secure 0 0 0) 0x7fe410088e20 con 0x7fe41c0cc750 2024-05-09T06:37:16.586 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 64 2024-05-09T06:37:16.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.607+0000 7fe4240df640 1 -- 172.21.15.3:0/387317008 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fe40806bfe0 msgr2=0x7fe40806e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:16.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.607+0000 7fe4240df640 1 --2- 172.21.15.3:0/387317008 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fe40806bfe0 0x7fe40806e4a0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fe414009630 tx=0x7fe414008040 comp rx=0 tx=0).stop 2024-05-09T06:37:16.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.607+0000 7fe4240df640 1 -- 172.21.15.3:0/387317008 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe41c0cc750 msgr2=0x7fe41c0c5bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:16.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.607+0000 7fe4240df640 1 --2- 172.21.15.3:0/387317008 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe41c0cc750 0x7fe41c0c5bc0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fe41000c400 tx=0x7fe41000c8c0 comp rx=0 tx=0).stop 2024-05-09T06:37:16.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.607+0000 7fe4240df640 1 -- 172.21.15.3:0/387317008 shutdown_connections 2024-05-09T06:37:16.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.607+0000 7fe4240df640 1 --2- 172.21.15.3:0/387317008 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fe40806bfe0 0x7fe40806e4a0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:16.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.607+0000 7fe4240df640 1 --2- 172.21.15.3:0/387317008 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe41c0d02b0 0x7fe41c1604a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:16.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.607+0000 7fe4240df640 1 --2- 172.21.15.3:0/387317008 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe41c0cc750 0x7fe41c0c5bc0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:16.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.607+0000 7fe4240df640 1 --2- 172.21.15.3:0/387317008 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe41c06cd70 0x7fe41c1504f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:16.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.607+0000 7fe4240df640 1 -- 172.21.15.3:0/387317008 >> 172.21.15.3:0/387317008 conn(0x7fe41c0b3af0 msgr2=0x7fe41c1626c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:16.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.608+0000 7fe4240df640 1 -- 172.21.15.3:0/387317008 shutdown_connections 2024-05-09T06:37:16.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.608+0000 7fe4240df640 1 -- 172.21.15.3:0/387317008 wait complete. 2024-05-09T06:37:16.599 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T06:37:16.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.724+0000 7f622a7ee640 1 Processor -- start 2024-05-09T06:37:16.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.724+0000 7f622a7ee640 1 -- start start 2024-05-09T06:37:16.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.725+0000 7f622a7ee640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f622413e0c0 0x7f62240cf720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:16.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.725+0000 7f622a7ee640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6224141000 0x7f62240ccd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:16.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.725+0000 7f622a7ee640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f622413e4a0 con 0x7f62240cc870 2024-05-09T06:37:16.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.725+0000 7f622a7ee640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f6224141430 con 0x7f622413e0c0 2024-05-09T06:37:16.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.725+0000 7f622a7ee640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f62240cd240 con 0x7f6224141000 2024-05-09T06:37:16.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.725+0000 7f62237fe640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f62240cc870 0x7f62241408f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:45950/0 (socket says 172.21.15.3:45950) 2024-05-09T06:37:16.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.725+0000 7f62237fe640 1 -- 172.21.15.3:0/1593874394 learned_addr learned my addr 172.21.15.3:0/1593874394 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-05-09T06:37:16.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.726+0000 7f6223fff640 1 --2- 172.21.15.3:0/1593874394 >> v2:172.21.15.190:3300/0 conn(0x7f622413e0c0 0x7f62240cf720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:16.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.726+0000 7f6228d8b640 1 --2- 172.21.15.3:0/1593874394 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6224141000 0x7f62240ccd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:16.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.726+0000 7f6222ffd640 1 -- 172.21.15.3:0/1593874394 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4088616126 0 0) 0x7f6218001110 con 0x7f62240cc870 2024-05-09T06:37:16.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.726+0000 7f6222ffd640 1 -- 172.21.15.3:0/1593874394 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f620c003520 con 0x7f62240cc870 2024-05-09T06:37:16.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.726+0000 7f6222ffd640 1 -- 172.21.15.3:0/1593874394 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 794889877 0 0) 0x7f620c003520 con 0x7f62240cc870 2024-05-09T06:37:16.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.726+0000 7f6222ffd640 1 -- 172.21.15.3:0/1593874394 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6224141000 msgr2=0x7f62240ccd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:37:16.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.726+0000 7f6222ffd640 1 --2- 172.21.15.3:0/1593874394 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6224141000 0x7f62240ccd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:16.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.726+0000 7f6222ffd640 1 -- 172.21.15.3:0/1593874394 >> v2:172.21.15.190:3300/0 conn(0x7f622413e0c0 msgr2=0x7f62240cf720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:37:16.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.726+0000 7f6222ffd640 1 --2- 172.21.15.3:0/1593874394 >> v2:172.21.15.190:3300/0 conn(0x7f622413e0c0 0x7f62240cf720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:16.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.726+0000 7f6222ffd640 1 -- 172.21.15.3:0/1593874394 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62240ce960 con 0x7f62240cc870 2024-05-09T06:37:16.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.726+0000 7f6222ffd640 1 -- 172.21.15.3:0/1593874394 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 563312031 0 0) 0x7f6218001840 con 0x7f62240cc870 2024-05-09T06:37:16.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.726+0000 7f6222ffd640 1 -- 172.21.15.3:0/1593874394 >> v1:172.21.15.3:6789/0 conn(0x7f62240cc870 legacy=0x7f62241408f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:16.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.726+0000 7f6222ffd640 1 --2- 172.21.15.3:0/1593874394 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f620c0039b0 0x7f620c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:16.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.726+0000 7f6222ffd640 1 --2- 172.21.15.3:0/1593874394 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f620c0043a0 0x7f620c004880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:16.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.726+0000 7f6222ffd640 1 --2- 172.21.15.3:0/1593874394 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f620c004ef0 0x7f620c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:16.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.726+0000 7f6222ffd640 1 -- 172.21.15.3:0/1593874394 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f62240cd240 con 0x7f620c004ef0 2024-05-09T06:37:16.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.726+0000 7f6222ffd640 1 -- 172.21.15.3:0/1593874394 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f620c00bf00 con 0x7f620c0043a0 2024-05-09T06:37:16.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.726+0000 7f6222ffd640 1 -- 172.21.15.3:0/1593874394 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f620c00c080 con 0x7f620c0039b0 2024-05-09T06:37:16.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.726+0000 7f6228d8b640 1 --2- 172.21.15.3:0/1593874394 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6224141000 0x7f62240ccd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T06:37:16.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.726+0000 7f6223fff640 1 --2- 172.21.15.3:0/1593874394 >> v2:172.21.15.190:3300/0 conn(0x7f622413e0c0 0x7f62240cf720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T06:37:16.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.726+0000 7f6228d8b640 1 --2- 172.21.15.3:0/1593874394 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f620c004ef0 0x7f620c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:16.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.726+0000 7f62237fe640 1 --2- 172.21.15.3:0/1593874394 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f620c0043a0 0x7f620c004880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:16.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.726+0000 7f6223fff640 1 --2- 172.21.15.3:0/1593874394 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f620c0039b0 0x7f620c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:16.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.726+0000 7f6228d8b640 1 -- 172.21.15.3:0/1593874394 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f620c0039b0 msgr2=0x7f620c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:37:16.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.726+0000 7f6228d8b640 1 --2- 172.21.15.3:0/1593874394 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f620c0039b0 0x7f620c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:16.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.726+0000 7f6228d8b640 1 -- 172.21.15.3:0/1593874394 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f620c0043a0 msgr2=0x7f620c004880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:37:16.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.726+0000 7f6228d8b640 1 --2- 172.21.15.3:0/1593874394 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f620c0043a0 0x7f620c004880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:16.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.726+0000 7f6228d8b640 1 -- 172.21.15.3:0/1593874394 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f620c003520 con 0x7f620c004ef0 2024-05-09T06:37:16.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.727+0000 7f6228d8b640 1 --2- 172.21.15.3:0/1593874394 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f620c004ef0 0x7f620c00b9c0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f62241413e0 tx=0x7f620c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f5417cf567ed4bd server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:16.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.727+0000 7f6222ffd640 1 -- 172.21.15.3:0/1593874394 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f621c036070 con 0x7f620c004ef0 2024-05-09T06:37:16.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.727+0000 7f6222ffd640 1 -- 172.21.15.3:0/1593874394 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f621c003c50 con 0x7f620c004ef0 2024-05-09T06:37:16.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.727+0000 7f6222ffd640 1 -- 172.21.15.3:0/1593874394 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f621c009d40 con 0x7f620c004ef0 2024-05-09T06:37:16.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.728+0000 7f622a7ee640 1 -- 172.21.15.3:0/1593874394 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f620c004ef0 msgr2=0x7f620c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:16.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.728+0000 7f622a7ee640 1 --2- 172.21.15.3:0/1593874394 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f620c004ef0 0x7f620c00b9c0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f62241413e0 tx=0x7f620c005830 comp rx=0 tx=0).stop 2024-05-09T06:37:16.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.728+0000 7f6228d8b640 1 -- 172.21.15.3:0/1593874394 reap_dead start 2024-05-09T06:37:16.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.728+0000 7f622a7ee640 1 -- 172.21.15.3:0/1593874394 shutdown_connections 2024-05-09T06:37:16.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.728+0000 7f622a7ee640 1 -- 172.21.15.3:0/1593874394 >> 172.21.15.3:0/1593874394 conn(0x7f622406b430 msgr2=0x7f622406b810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:16.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.728+0000 7f622a7ee640 1 -- 172.21.15.3:0/1593874394 shutdown_connections 2024-05-09T06:37:16.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.728+0000 7f622a7ee640 1 -- 172.21.15.3:0/1593874394 wait complete. 2024-05-09T06:37:16.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.729+0000 7f622a7ee640 1 Processor -- start 2024-05-09T06:37:16.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.729+0000 7f622a7ee640 1 -- start start 2024-05-09T06:37:16.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.730+0000 7f622a7ee640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f622413cbe0 0x7f6224154950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:16.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.730+0000 7f622a7ee640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f622413d2d0 0x7f62240b10d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:16.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.730+0000 7f622a7ee640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f62240b1610 0x7f62240afb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:16.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.730+0000 7f622a7ee640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f62240cf200 con 0x7f622413cbe0 2024-05-09T06:37:16.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.730+0000 7f622a7ee640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f62240cef20 con 0x7f622413d2d0 2024-05-09T06:37:16.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.730+0000 7f622a7ee640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f6224141660 con 0x7f62240b1610 2024-05-09T06:37:16.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.730+0000 7f6223fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f622413cbe0 0x7f6224154950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:16.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.730+0000 7f6223fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f622413cbe0 0x7f6224154950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51118/0 (socket says 172.21.15.3:51118) 2024-05-09T06:37:16.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.730+0000 7f6223fff640 1 -- 172.21.15.3:0/1610542673 learned_addr learned my addr 172.21.15.3:0/1610542673 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:37:16.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.730+0000 7f6228d8b640 1 --2- 172.21.15.3:0/1610542673 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f62240b1610 0x7f62240afb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:16.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.730+0000 7f62237fe640 1 --2- 172.21.15.3:0/1610542673 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f622413d2d0 0x7f62240b10d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:16.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.730+0000 7f6223fff640 1 -- 172.21.15.3:0/1610542673 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f62240b1610 msgr2=0x7f62240afb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:16.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.730+0000 7f6223fff640 1 --2- 172.21.15.3:0/1610542673 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f62240b1610 0x7f62240afb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:16.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.730+0000 7f6223fff640 1 -- 172.21.15.3:0/1610542673 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f622413d2d0 msgr2=0x7f62240b10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:16.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.730+0000 7f6223fff640 1 --2- 172.21.15.3:0/1610542673 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f622413d2d0 0x7f62240b10d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:16.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.730+0000 7f6223fff640 1 -- 172.21.15.3:0/1610542673 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62241551d0 con 0x7f622413cbe0 2024-05-09T06:37:16.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.730+0000 7f6228d8b640 1 --2- 172.21.15.3:0/1610542673 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f62240b1610 0x7f62240afb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:37:16.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.730+0000 7f62237fe640 1 --2- 172.21.15.3:0/1610542673 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f622413d2d0 0x7f62240b10d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:37:16.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.731+0000 7f6223fff640 1 --2- 172.21.15.3:0/1610542673 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f622413cbe0 0x7f6224154950 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f6214011d00 tx=0x7f621400f610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:16.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.731+0000 7f6220ff9640 1 -- 172.21.15.3:0/1610542673 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f621400fd20 con 0x7f622413cbe0 2024-05-09T06:37:16.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.731+0000 7f6220ff9640 1 -- 172.21.15.3:0/1610542673 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6214004030 con 0x7f622413cbe0 2024-05-09T06:37:16.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.731+0000 7f6220ff9640 1 -- 172.21.15.3:0/1610542673 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6214004420 con 0x7f622413cbe0 2024-05-09T06:37:16.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.731+0000 7f622a7ee640 1 -- 172.21.15.3:0/1610542673 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62241554b0 con 0x7f622413cbe0 2024-05-09T06:37:16.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.731+0000 7f622a7ee640 1 -- 172.21.15.3:0/1610542673 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6224155a70 con 0x7f622413cbe0 2024-05-09T06:37:16.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.732+0000 7f622a7ee640 1 -- 172.21.15.3:0/1610542673 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f622404f660 con 0x7f622413cbe0 2024-05-09T06:37:16.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.732+0000 7f6220ff9640 1 -- 172.21.15.3:0/1610542673 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 7) ==== 90651+0+0 (secure 0 0 0) 0x7f6214012040 con 0x7f622413cbe0 2024-05-09T06:37:16.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.733+0000 7f6220ff9640 1 --2- 172.21.15.3:0/1610542673 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f621006c010 0x7f621006e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:16.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.733+0000 7f6220ff9640 1 -- 172.21.15.3:0/1610542673 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f621408b710 con 0x7f622413cbe0 2024-05-09T06:37:16.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.733+0000 7f62237fe640 1 --2- 172.21.15.3:0/1610542673 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f621006c010 0x7f621006e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:16.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.733+0000 7f62237fe640 1 --2- 172.21.15.3:0/1610542673 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f621006c010 0x7f621006e4d0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f6218004840 tx=0x7f6218009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:16.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.737+0000 7f6220ff9640 1 -- 172.21.15.3:0/1610542673 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f6214019070 con 0x7f622413cbe0 2024-05-09T06:37:16.938 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.958+0000 7f622a7ee640 1 -- 172.21.15.3:0/1610542673 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f62240b4cf0 con 0x7f622413cbe0 2024-05-09T06:37:16.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.958+0000 7f6220ff9640 1 -- 172.21.15.3:0/1610542673 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+5181 (secure 0 0 0) 0x7f6214059e40 con 0x7f622413cbe0 2024-05-09T06:37:16.939 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-09T06:37:16.939 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4912,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6836/2400645940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":2400645940},{"type":"v1","addr":"172.21.15.190:6836","nonce":2400645940}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4915,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6837/3849729576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6835","nonce":3849729576},{"type":"v1","addr":"172.21.15.190:6837","nonce":3849729576}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4918,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6834/2647782276","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":2647782276},{"type":"v1","addr":"172.21.15.3:6834","nonce":2647782276}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-09T06:36:57.929421+0000","modified":"2024-05-09T06:37:00.941894+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5103},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5103":{"gid":5103,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.3:6835/2138710335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6833","nonce":2138710335},{"type":"v1","addr":"172.21.15.3:6835","nonce":2138710335}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5103,"qdb_cluster":[5103]},"id":2}]} 2024-05-09T06:37:16.940 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 18 2024-05-09T06:37:16.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.960+0000 7f622a7ee640 1 -- 172.21.15.3:0/1610542673 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f621006c010 msgr2=0x7f621006e4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:16.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.960+0000 7f622a7ee640 1 --2- 172.21.15.3:0/1610542673 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f621006c010 0x7f621006e4d0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f6218004840 tx=0x7f6218009040 comp rx=0 tx=0).stop 2024-05-09T06:37:16.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.960+0000 7f622a7ee640 1 -- 172.21.15.3:0/1610542673 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f622413cbe0 msgr2=0x7f6224154950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:16.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.960+0000 7f622a7ee640 1 --2- 172.21.15.3:0/1610542673 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f622413cbe0 0x7f6224154950 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f6214011d00 tx=0x7f621400f610 comp rx=0 tx=0).stop 2024-05-09T06:37:16.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.960+0000 7f622a7ee640 1 -- 172.21.15.3:0/1610542673 shutdown_connections 2024-05-09T06:37:16.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.960+0000 7f622a7ee640 1 --2- 172.21.15.3:0/1610542673 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f621006c010 0x7f621006e4d0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:16.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.960+0000 7f622a7ee640 1 --2- 172.21.15.3:0/1610542673 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f62240b1610 0x7f62240afb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:16.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.960+0000 7f622a7ee640 1 --2- 172.21.15.3:0/1610542673 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f622413d2d0 0x7f62240b10d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:16.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.960+0000 7f622a7ee640 1 --2- 172.21.15.3:0/1610542673 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f622413cbe0 0x7f6224154950 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:16.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.960+0000 7f622a7ee640 1 -- 172.21.15.3:0/1610542673 >> 172.21.15.3:0/1610542673 conn(0x7f622406b430 msgr2=0x7f6224158f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:16.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.960+0000 7f622a7ee640 1 -- 172.21.15.3:0/1610542673 shutdown_connections 2024-05-09T06:37:16.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:16.960+0000 7f622a7ee640 1 -- 172.21.15.3:0/1610542673 wait complete. 2024-05-09T06:37:16.952 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T06:37:17.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:17.075+0000 7f157f46b640 1 Processor -- start 2024-05-09T06:37:17.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:17.076+0000 7f157f46b640 1 -- start start 2024-05-09T06:37:17.057 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:17.076+0000 7f157f46b640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f157815e480 0x7f15780d1dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:17.058 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:17.076+0000 7f157f46b640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f15780d2310 0x7f15780d27c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:17.058 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:17.076+0000 7f157f46b640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f15780d4e50 con 0x7f15780d2d00 2024-05-09T06:37:17.058 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:17.076+0000 7f157f46b640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f15780d0270 con 0x7f157815e480 2024-05-09T06:37:17.058 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:17.076+0000 7f157f46b640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f15780cf1d0 con 0x7f15780d2310 2024-05-09T06:37:17.058 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:17.076+0000 7f157d207640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f157815e480 0x7f15780d1dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:17.058 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:17.077+0000 7f157da08640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f15780d2d00 0x7f1578162070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:45964/0 (socket says 172.21.15.3:45964) 2024-05-09T06:37:17.058 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:17.077+0000 7f157d207640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f157815e480 0x7f15780d1dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51120/0 (socket says 172.21.15.3:51120) 2024-05-09T06:37:17.058 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:17.077+0000 7f157da08640 1 -- 172.21.15.3:0/323253389 learned_addr learned my addr 172.21.15.3:0/323253389 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-05-09T06:37:17.058 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:17.077+0000 7f157ca06640 1 --2- 172.21.15.3:0/323253389 >> v2:172.21.15.190:3300/0 conn(0x7f15780d2310 0x7f15780d27c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:17.058 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:17.077+0000 7f157d207640 1 -- 172.21.15.3:0/323253389 >> v2:172.21.15.190:3300/0 conn(0x7f15780d2310 msgr2=0x7f15780d27c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:37:17.058 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:17.077+0000 7f157d207640 1 --2- 172.21.15.3:0/323253389 >> v2:172.21.15.190:3300/0 conn(0x7f15780d2310 0x7f15780d27c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:17.058 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:17.077+0000 7f157d207640 1 -- 172.21.15.3:0/323253389 >> v1:172.21.15.3:6789/0 conn(0x7f15780d2d00 legacy=0x7f1578162070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:17.058 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:17.077+0000 7f157d207640 1 -- 172.21.15.3:0/323253389 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15780cf4b0 con 0x7f157815e480 2024-05-09T06:37:17.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:17.078+0000 7f157d207640 1 --2- 172.21.15.3:0/323253389 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f157815e480 0x7f15780d1dd0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f15740097a0 tx=0x7f157402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3dad61cb0d2a29e4 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:17.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:17.078+0000 7f156ffff640 1 -- 172.21.15.3:0/323253389 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1574037070 con 0x7f157815e480 2024-05-09T06:37:17.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:17.078+0000 7f156ffff640 1 -- 172.21.15.3:0/323253389 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1574032440 con 0x7f157815e480 2024-05-09T06:37:17.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:17.078+0000 7f156ffff640 1 -- 172.21.15.3:0/323253389 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1574032810 con 0x7f157815e480 2024-05-09T06:37:17.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:17.078+0000 7f157f46b640 1 -- 172.21.15.3:0/323253389 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f157815e480 msgr2=0x7f15780d1dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:17.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:17.078+0000 7f157f46b640 1 --2- 172.21.15.3:0/323253389 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f157815e480 0x7f15780d1dd0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f15740097a0 tx=0x7f157402a1e0 comp rx=0 tx=0).stop 2024-05-09T06:37:17.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:17.078+0000 7f157f46b640 1 -- 172.21.15.3:0/323253389 shutdown_connections 2024-05-09T06:37:17.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:17.078+0000 7f157f46b640 1 --2- 172.21.15.3:0/323253389 >> v2:172.21.15.190:3300/0 conn(0x7f15780d2310 0x7f15780d27c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:17.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:17.078+0000 7f157f46b640 1 --2- 172.21.15.3:0/323253389 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f157815e480 0x7f15780d1dd0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:17.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:17.078+0000 7f157f46b640 1 -- 172.21.15.3:0/323253389 >> 172.21.15.3:0/323253389 conn(0x7f157809d4d0 msgr2=0x7f15780d4200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:17.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:17.078+0000 7f157f46b640 1 -- 172.21.15.3:0/323253389 shutdown_connections 2024-05-09T06:37:17.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:17.078+0000 7f157f46b640 1 -- 172.21.15.3:0/323253389 wait complete. 2024-05-09T06:37:17.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:17.080+0000 7f157f46b640 1 Processor -- start 2024-05-09T06:37:17.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:17.080+0000 7f157f46b640 1 -- start start 2024-05-09T06:37:17.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:17.080+0000 7f157f46b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f15780d2310 0x7f15780cbcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:17.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:17.080+0000 7f157f46b640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f15780d2d00 0x7f15781f3d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:17.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:17.080+0000 7f157f46b640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f157815e480 0x7f15780b8e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:17.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:17.080+0000 7f157f46b640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f15780cfd50 con 0x7f15780d2310 2024-05-09T06:37:17.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:17.080+0000 7f157f46b640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f15780cfa70 con 0x7f157815e480 2024-05-09T06:37:17.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:17.080+0000 7f157f46b640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f15780b5ab0 con 0x7f15780d2d00 2024-05-09T06:37:17.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:17.080+0000 7f157d207640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f15780d2310 0x7f15780cbcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:17.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:17.080+0000 7f157d207640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f15780d2310 0x7f15780cbcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51136/0 (socket says 172.21.15.3:51136) 2024-05-09T06:37:17.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:17.080+0000 7f157d207640 1 -- 172.21.15.3:0/1363159853 learned_addr learned my addr 172.21.15.3:0/1363159853 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:37:17.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:17.081+0000 7f157da08640 1 --2- 172.21.15.3:0/1363159853 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f157815e480 0x7f15780b8e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:17.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:17.081+0000 7f157ca06640 1 --2- 172.21.15.3:0/1363159853 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f15780d2d00 0x7f15781f3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:17.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:17.081+0000 7f157d207640 1 -- 172.21.15.3:0/1363159853 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f15780d2d00 msgr2=0x7f15781f3d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:17.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:17.081+0000 7f157d207640 1 --2- 172.21.15.3:0/1363159853 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f15780d2d00 0x7f15781f3d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:17.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:17.081+0000 7f157d207640 1 -- 172.21.15.3:0/1363159853 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f157815e480 msgr2=0x7f15780b8e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:17.064 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:17.081+0000 7f157d207640 1 --2- 172.21.15.3:0/1363159853 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f157815e480 0x7f15780b8e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:17.064 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:17.081+0000 7f157d207640 1 -- 172.21.15.3:0/1363159853 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15780cc510 con 0x7f15780d2310 2024-05-09T06:37:17.064 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:17.081+0000 7f157ca06640 1 --2- 172.21.15.3:0/1363159853 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f15780d2d00 0x7f15781f3d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:37:17.064 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:17.081+0000 7f157d207640 1 --2- 172.21.15.3:0/1363159853 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f15780d2310 0x7f15780cbcb0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f1574035040 tx=0x7f15740046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:17.064 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:17.081+0000 7f156dffb640 1 -- 172.21.15.3:0/1363159853 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1574037040 con 0x7f15780d2310 2024-05-09T06:37:17.064 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:17.081+0000 7f156dffb640 1 -- 172.21.15.3:0/1363159853 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1574030770 con 0x7f15780d2310 2024-05-09T06:37:17.064 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:17.081+0000 7f157f46b640 1 -- 172.21.15.3:0/1363159853 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15780cc7f0 con 0x7f15780d2310 2024-05-09T06:37:17.064 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:17.081+0000 7f156dffb640 1 -- 172.21.15.3:0/1363159853 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1574030b80 con 0x7f15780d2310 2024-05-09T06:37:17.064 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:17.081+0000 7f157f46b640 1 -- 172.21.15.3:0/1363159853 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15780ccdb0 con 0x7f15780d2310 2024-05-09T06:37:17.065 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:17.082+0000 7f157f46b640 1 -- 172.21.15.3:0/1363159853 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f157804f660 con 0x7f15780d2310 2024-05-09T06:37:17.065 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:17.083+0000 7f156dffb640 1 -- 172.21.15.3:0/1363159853 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 7) ==== 90651+0+0 (secure 0 0 0) 0x7f1574044020 con 0x7f15780d2310 2024-05-09T06:37:17.065 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:17.083+0000 7f156dffb640 1 --2- 172.21.15.3:0/1363159853 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f155c06bfe0 0x7f155c06e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:17.065 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:17.083+0000 7f156dffb640 1 -- 172.21.15.3:0/1363159853 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f15740ab700 con 0x7f15780d2310 2024-05-09T06:37:17.065 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:17.084+0000 7f157ca06640 1 --2- 172.21.15.3:0/1363159853 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f155c06bfe0 0x7f155c06e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:17.065 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:17.084+0000 7f157ca06640 1 --2- 172.21.15.3:0/1363159853 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f155c06bfe0 0x7f155c06e4a0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f1568006fd0 tx=0x7f1568008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:17.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:17.088+0000 7f156dffb640 1 -- 172.21.15.3:0/1363159853 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f1574032e20 con 0x7f15780d2310 2024-05-09T06:37:17.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:17.274+0000 7f157f46b640 1 -- 172.21.15.3:0/1363159853 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f15780c8d80 con 0x7f15780d2310 2024-05-09T06:37:17.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:17.275+0000 7f156dffb640 1 -- 172.21.15.3:0/1363159853 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) ==== 74+0+16585 (secure 0 0 0) 0x7f1574079e30 con 0x7f15780d2310 2024-05-09T06:37:17.256 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-09T06:37:17.256 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":33,"fsid":"16ed737d-2665-431a-8ac6-ef242be3975a","created":"2024-05-09T06:35:42.541368+0000","modified":"2024-05-09T06:36:59.564366+0000","last_up_change":"2024-05-09T06:35:53.778274+0000","last_in_change":"2024-05-09T06:35:44.780586+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T06:35:54.819999+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-05-09T06:36:56.494700+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-05-09T06:36:56.900006+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fae3a8db-4826-4f3c-b33a-12c09bb5df6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":1897849827},{"type":"v1","addr":"172.21.15.3:6817","nonce":1897849827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":1897849827},{"type":"v1","addr":"172.21.15.3:6819","nonce":1897849827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":1897849827},{"type":"v1","addr":"172.21.15.3:6823","nonce":1897849827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":1897849827},{"type":"v1","addr":"172.21.15.3:6821","nonce":1897849827}]},"public_addr":"172.21.15.3:6817/1897849827","cluster_addr":"172.21.15.3:6819/1897849827","heartbeat_back_addr":"172.21.15.3:6823/1897849827","heartbeat_front_addr":"172.21.15.3:6821/1897849827","state":["exists","up"]},{"osd":1,"uuid":"646e882d-4ce4-4720-8c84-f71df563a35e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":926805498},{"type":"v1","addr":"172.21.15.3:6801","nonce":926805498}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":926805498},{"type":"v1","addr":"172.21.15.3:6803","nonce":926805498}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":926805498},{"type":"v1","addr":"172.21.15.3:6807","nonce":926805498}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":926805498},{"type":"v1","addr":"172.21.15.3:6805","nonce":926805498}]},"public_addr":"172.21.15.3:6801/926805498","cluster_addr":"172.21.15.3:6803/926805498","heartbeat_back_addr":"172.21.15.3:6807/926805498","heartbeat_front_addr":"172.21.15.3:6805/926805498","state":["exists","up"]},{"osd":2,"uuid":"1abbadb5-a570-4359-9643-acff3ede35a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":801295451},{"type":"v1","addr":"172.21.15.3:6809","nonce":801295451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":801295451},{"type":"v1","addr":"172.21.15.3:6811","nonce":801295451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":801295451},{"type":"v1","addr":"172.21.15.3:6815","nonce":801295451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":801295451},{"type":"v1","addr":"172.21.15.3:6813","nonce":801295451}]},"public_addr":"172.21.15.3:6809/801295451","cluster_addr":"172.21.15.3:6811/801295451","heartbeat_back_addr":"172.21.15.3:6815/801295451","heartbeat_front_addr":"172.21.15.3:6813/801295451","state":["exists","up"]},{"osd":3,"uuid":"62ac4dc1-2371-4dd2-acd3-e2dbb11aedf5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":1903389857},{"type":"v1","addr":"172.21.15.3:6825","nonce":1903389857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":1903389857},{"type":"v1","addr":"172.21.15.3:6827","nonce":1903389857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":1903389857},{"type":"v1","addr":"172.21.15.3:6831","nonce":1903389857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":1903389857},{"type":"v1","addr":"172.21.15.3:6829","nonce":1903389857}]},"public_addr":"172.21.15.3:6825/1903389857","cluster_addr":"172.21.15.3:6827/1903389857","heartbeat_back_addr":"172.21.15.3:6831/1903389857","heartbeat_front_addr":"172.21.15.3:6829/1903389857","state":["exists","up"]},{"osd":4,"uuid":"ae808398-8326-4d7c-9319-a2a0eafffb59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":2143624576},{"type":"v1","addr":"172.21.15.190:6801","nonce":2143624576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":2143624576},{"type":"v1","addr":"172.21.15.190:6803","nonce":2143624576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":2143624576},{"type":"v1","addr":"172.21.15.190:6807","nonce":2143624576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":2143624576},{"type":"v1","addr":"172.21.15.190:6805","nonce":2143624576}]},"public_addr":"172.21.15.190:6801/2143624576","cluster_addr":"172.21.15.190:6803/2143624576","heartbeat_back_addr":"172.21.15.190:6807/2143624576","heartbeat_front_addr":"172.21.15.190:6805/2143624576","state":["exists","up"]},{"osd":5,"uuid":"918adf50-c440-48ac-9a8d-4837562d7275","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":1766923126},{"type":"v1","addr":"172.21.15.190:6814","nonce":1766923126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":1766923126},{"type":"v1","addr":"172.21.15.190:6818","nonce":1766923126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":1766923126},{"type":"v1","addr":"172.21.15.190:6823","nonce":1766923126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":1766923126},{"type":"v1","addr":"172.21.15.190:6821","nonce":1766923126}]},"public_addr":"172.21.15.190:6814/1766923126","cluster_addr":"172.21.15.190:6818/1766923126","heartbeat_back_addr":"172.21.15.190:6823/1766923126","heartbeat_front_addr":"172.21.15.190:6821/1766923126","state":["exists","up"]},{"osd":6,"uuid":"ac2882bc-2a7c-463c-8f6c-76ff49828adc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":342414248},{"type":"v1","addr":"172.21.15.190:6825","nonce":342414248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":342414248},{"type":"v1","addr":"172.21.15.190:6827","nonce":342414248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":342414248},{"type":"v1","addr":"172.21.15.190:6831","nonce":342414248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":342414248},{"type":"v1","addr":"172.21.15.190:6829","nonce":342414248}]},"public_addr":"172.21.15.190:6825/342414248","cluster_addr":"172.21.15.190:6827/342414248","heartbeat_back_addr":"172.21.15.190:6831/342414248","heartbeat_front_addr":"172.21.15.190:6829/342414248","state":["exists","up"]},{"osd":7,"uuid":"dd4e767a-4730-4c70-8bd5-3f088140a5a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":537849891},{"type":"v1","addr":"172.21.15.190:6809","nonce":537849891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":537849891},{"type":"v1","addr":"172.21.15.190:6811","nonce":537849891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6817","nonce":537849891},{"type":"v1","addr":"172.21.15.190:6819","nonce":537849891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6813","nonce":537849891},{"type":"v1","addr":"172.21.15.190:6815","nonce":537849891}]},"public_addr":"172.21.15.190:6809/537849891","cluster_addr":"172.21.15.190:6811/537849891","heartbeat_back_addr":"172.21.15.190:6819/537849891","heartbeat_front_addr":"172.21.15.190:6815/537849891","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.889784+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.974002+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.973549+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.955396+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.928471+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.902418+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.958696+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.926474+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T06:37:17.258 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:17.277+0000 7f157f46b640 1 -- 172.21.15.3:0/1363159853 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f155c06bfe0 msgr2=0x7f155c06e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:17.258 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:17.277+0000 7f157f46b640 1 --2- 172.21.15.3:0/1363159853 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f155c06bfe0 0x7f155c06e4a0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f1568006fd0 tx=0x7f1568008040 comp rx=0 tx=0).stop 2024-05-09T06:37:17.258 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:17.277+0000 7f157f46b640 1 -- 172.21.15.3:0/1363159853 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f15780d2310 msgr2=0x7f15780cbcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:17.258 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:17.277+0000 7f157f46b640 1 --2- 172.21.15.3:0/1363159853 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f15780d2310 0x7f15780cbcb0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f1574035040 tx=0x7f15740046c0 comp rx=0 tx=0).stop 2024-05-09T06:37:17.258 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:17.277+0000 7f157f46b640 1 -- 172.21.15.3:0/1363159853 shutdown_connections 2024-05-09T06:37:17.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:17.277+0000 7f157f46b640 1 --2- 172.21.15.3:0/1363159853 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f155c06bfe0 0x7f155c06e4a0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:17.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:17.277+0000 7f157f46b640 1 --2- 172.21.15.3:0/1363159853 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f157815e480 0x7f15780b8e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:17.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:17.277+0000 7f157f46b640 1 --2- 172.21.15.3:0/1363159853 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f15780d2d00 0x7f15781f3d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:17.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:17.277+0000 7f157f46b640 1 --2- 172.21.15.3:0/1363159853 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f15780d2310 0x7f15780cbcb0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:17.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:17.277+0000 7f157f46b640 1 -- 172.21.15.3:0/1363159853 >> 172.21.15.3:0/1363159853 conn(0x7f157809d4d0 msgr2=0x7f157806bf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:17.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:17.277+0000 7f157f46b640 1 -- 172.21.15.3:0/1363159853 shutdown_connections 2024-05-09T06:37:17.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:17.277+0000 7f157f46b640 1 -- 172.21.15.3:0/1363159853 wait complete. 2024-05-09T06:37:17.269 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-05-09T06:37:17.269 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T06:37:17.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:17.392+0000 7f0aebf4f640 1 Processor -- start 2024-05-09T06:37:17.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:17.393+0000 7f0aebf4f640 1 -- start start 2024-05-09T06:37:17.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:17.393+0000 7f0aebf4f640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f0ae40d27c0 0x7f0ae40d2c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:17.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:17.393+0000 7f0aebf4f640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0ae4166390 0x7f0ae4168780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:17.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:17.394+0000 7f0aebf4f640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0ae40d4e50 con 0x7f0ae415e480 2024-05-09T06:37:17.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:17.394+0000 7f0aebf4f640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f0ae40d3160 con 0x7f0ae40d27c0 2024-05-09T06:37:17.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:17.394+0000 7f0aebf4f640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f0ae40d0270 con 0x7f0ae4166390 2024-05-09T06:37:17.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:17.394+0000 7f0ae9ceb640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f0ae415e480 0x7f0ae40d1dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:45976/0 (socket says 172.21.15.3:45976) 2024-05-09T06:37:17.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:17.394+0000 7f0ae9ceb640 1 -- 172.21.15.3:0/4011556663 learned_addr learned my addr 172.21.15.3:0/4011556663 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-05-09T06:37:17.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:17.394+0000 7f0aea4ec640 1 --2- 172.21.15.3:0/4011556663 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0ae4166390 0x7f0ae4168780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:17.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:17.394+0000 7f0ae94ea640 1 --2- 172.21.15.3:0/4011556663 >> v2:172.21.15.190:3300/0 conn(0x7f0ae40d27c0 0x7f0ae40d2c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:17.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:17.394+0000 7f0ae8ce9640 1 -- 172.21.15.3:0/4011556663 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3608820688 0 0) 0x7f0ae0001110 con 0x7f0ae415e480 2024-05-09T06:37:17.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:17.394+0000 7f0ae8ce9640 1 -- 172.21.15.3:0/4011556663 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0ad0003520 con 0x7f0ae415e480 2024-05-09T06:37:17.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:17.394+0000 7f0ae8ce9640 1 -- 172.21.15.3:0/4011556663 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 689388269 0 0) 0x7f0ad0003520 con 0x7f0ae415e480 2024-05-09T06:37:17.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:17.394+0000 7f0aea4ec640 1 -- 172.21.15.3:0/4011556663 >> v2:172.21.15.190:3300/0 conn(0x7f0ae40d27c0 msgr2=0x7f0ae40d2c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:37:17.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:17.394+0000 7f0aea4ec640 1 --2- 172.21.15.3:0/4011556663 >> v2:172.21.15.190:3300/0 conn(0x7f0ae40d27c0 0x7f0ae40d2c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:17.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:17.394+0000 7f0aea4ec640 1 -- 172.21.15.3:0/4011556663 >> v1:172.21.15.3:6789/0 conn(0x7f0ae415e480 legacy=0x7f0ae40d1dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:17.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:17.394+0000 7f0aea4ec640 1 -- 172.21.15.3:0/4011556663 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ae40cf1d0 con 0x7f0ae4166390 2024-05-09T06:37:17.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:17.394+0000 7f0ae94ea640 1 --2- 172.21.15.3:0/4011556663 >> v2:172.21.15.190:3300/0 conn(0x7f0ae40d27c0 0x7f0ae40d2c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T06:37:17.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:17.395+0000 7f0aea4ec640 1 --2- 172.21.15.3:0/4011556663 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0ae4166390 0x7f0ae4168780 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f0adc009d90 tx=0x7f0adc02a5f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=877f4db0cf901d0c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:17.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:17.395+0000 7f0ae8ce9640 1 -- 172.21.15.3:0/4011556663 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ae0001840 con 0x7f0ae4166390 2024-05-09T06:37:17.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:17.395+0000 7f0ae8ce9640 1 -- 172.21.15.3:0/4011556663 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0adc007540 con 0x7f0ae4166390 2024-05-09T06:37:17.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:17.395+0000 7f0ae8ce9640 1 -- 172.21.15.3:0/4011556663 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0adc007910 con 0x7f0ae4166390 2024-05-09T06:37:17.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:17.396+0000 7f0aebf4f640 1 -- 172.21.15.3:0/4011556663 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0ae4166390 msgr2=0x7f0ae4168780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:17.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:17.396+0000 7f0aebf4f640 1 --2- 172.21.15.3:0/4011556663 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0ae4166390 0x7f0ae4168780 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f0adc009d90 tx=0x7f0adc02a5f0 comp rx=0 tx=0).stop 2024-05-09T06:37:17.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:17.396+0000 7f0aebf4f640 1 -- 172.21.15.3:0/4011556663 shutdown_connections 2024-05-09T06:37:17.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:17.396+0000 7f0aebf4f640 1 --2- 172.21.15.3:0/4011556663 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0ae4166390 0x7f0ae4168780 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:17.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:17.396+0000 7f0aebf4f640 1 --2- 172.21.15.3:0/4011556663 >> v2:172.21.15.190:3300/0 conn(0x7f0ae40d27c0 0x7f0ae40d2c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:17.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:17.396+0000 7f0aebf4f640 1 -- 172.21.15.3:0/4011556663 >> 172.21.15.3:0/4011556663 conn(0x7f0ae409d4d0 msgr2=0x7f0ae40d4200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:17.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:17.396+0000 7f0aebf4f640 1 -- 172.21.15.3:0/4011556663 shutdown_connections 2024-05-09T06:37:17.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:17.396+0000 7f0aebf4f640 1 -- 172.21.15.3:0/4011556663 wait complete. 2024-05-09T06:37:17.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:17.397+0000 7f0aebf4f640 1 Processor -- start 2024-05-09T06:37:17.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:17.397+0000 7f0aebf4f640 1 -- start start 2024-05-09T06:37:17.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:17.398+0000 7f0aebf4f640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0ae40d27c0 0x7f0ae41592a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:17.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:17.398+0000 7f0aebf4f640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0ae415e480 0x7f0ae41f7e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:17.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:17.398+0000 7f0aebf4f640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0ae4166390 0x7f0ae41f8340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:17.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:17.398+0000 7f0aebf4f640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f0ae40cfa70 con 0x7f0ae415e480 2024-05-09T06:37:17.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:17.398+0000 7f0aebf4f640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f0ae40cf790 con 0x7f0ae40d27c0 2024-05-09T06:37:17.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:17.398+0000 7f0aebf4f640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f0ae40cfd50 con 0x7f0ae4166390 2024-05-09T06:37:17.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:17.398+0000 7f0ae94ea640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0ae415e480 0x7f0ae41f7e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:17.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:17.398+0000 7f0ae94ea640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0ae415e480 0x7f0ae41f7e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51150/0 (socket says 172.21.15.3:51150) 2024-05-09T06:37:17.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:17.398+0000 7f0ae94ea640 1 -- 172.21.15.3:0/3924152794 learned_addr learned my addr 172.21.15.3:0/3924152794 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:37:17.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:17.398+0000 7f0ae9ceb640 1 --2- 172.21.15.3:0/3924152794 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0ae40d27c0 0x7f0ae41592a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:17.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:17.398+0000 7f0aea4ec640 1 --2- 172.21.15.3:0/3924152794 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0ae4166390 0x7f0ae41f8340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:17.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:17.398+0000 7f0ae94ea640 1 -- 172.21.15.3:0/3924152794 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0ae4166390 msgr2=0x7f0ae41f8340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:17.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:17.398+0000 7f0ae94ea640 1 --2- 172.21.15.3:0/3924152794 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0ae4166390 0x7f0ae41f8340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:17.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:17.398+0000 7f0ae94ea640 1 -- 172.21.15.3:0/3924152794 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0ae40d27c0 msgr2=0x7f0ae41592a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:17.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:17.398+0000 7f0ae94ea640 1 --2- 172.21.15.3:0/3924152794 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0ae40d27c0 0x7f0ae41592a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:17.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:17.398+0000 7f0ae94ea640 1 -- 172.21.15.3:0/3924152794 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ae40c87c0 con 0x7f0ae415e480 2024-05-09T06:37:17.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:17.398+0000 7f0ae94ea640 1 --2- 172.21.15.3:0/3924152794 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0ae415e480 0x7f0ae41f7e00 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f0ad400e910 tx=0x7f0ad400edd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:17.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:17.399+0000 7f0ada7fc640 1 -- 172.21.15.3:0/3924152794 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ad4017070 con 0x7f0ae415e480 2024-05-09T06:37:17.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:17.399+0000 7f0ada7fc640 1 -- 172.21.15.3:0/3924152794 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ad400ca00 con 0x7f0ae415e480 2024-05-09T06:37:17.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:17.399+0000 7f0aebf4f640 1 -- 172.21.15.3:0/3924152794 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ae40c8aa0 con 0x7f0ae415e480 2024-05-09T06:37:17.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:17.399+0000 7f0ada7fc640 1 -- 172.21.15.3:0/3924152794 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ad400cdb0 con 0x7f0ae415e480 2024-05-09T06:37:17.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:17.399+0000 7f0aebf4f640 1 -- 172.21.15.3:0/3924152794 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ae40c9060 con 0x7f0ae415e480 2024-05-09T06:37:17.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:17.400+0000 7f0aebf4f640 1 -- 172.21.15.3:0/3924152794 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ae404f660 con 0x7f0ae415e480 2024-05-09T06:37:17.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:17.400+0000 7f0ada7fc640 1 -- 172.21.15.3:0/3924152794 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 7) ==== 90651+0+0 (secure 0 0 0) 0x7f0ad401a070 con 0x7f0ae415e480 2024-05-09T06:37:17.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:17.401+0000 7f0ada7fc640 1 --2- 172.21.15.3:0/3924152794 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f0ac806bfe0 0x7f0ac806e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:17.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:17.401+0000 7f0ada7fc640 1 -- 172.21.15.3:0/3924152794 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f0ad408aeb0 con 0x7f0ae415e480 2024-05-09T06:37:17.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:17.401+0000 7f0ae9ceb640 1 --2- 172.21.15.3:0/3924152794 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f0ac806bfe0 0x7f0ac806e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:17.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:17.401+0000 7f0ae9ceb640 1 --2- 172.21.15.3:0/3924152794 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f0ac806bfe0 0x7f0ac806e4a0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f0ae0005880 tx=0x7f0ae00012d0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:17.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:17.405+0000 7f0ada7fc640 1 -- 172.21.15.3:0/3924152794 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f0ad4059040 con 0x7f0ae415e480 2024-05-09T06:37:17.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:17.627+0000 7f0aebf4f640 1 -- 172.21.15.3:0/3924152794 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0ae40d18f0 con 0x7f0ae415e480 2024-05-09T06:37:17.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:17.629+0000 7f0ada7fc640 1 -- 172.21.15.3:0/3924152794 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+5181 (secure 0 0 0) 0x7f0ad405d340 con 0x7f0ae415e480 2024-05-09T06:37:17.610 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-09T06:37:17.610 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4912,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6836/2400645940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":2400645940},{"type":"v1","addr":"172.21.15.190:6836","nonce":2400645940}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4915,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6837/3849729576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6835","nonce":3849729576},{"type":"v1","addr":"172.21.15.190:6837","nonce":3849729576}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4918,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6834/2647782276","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":2647782276},{"type":"v1","addr":"172.21.15.3:6834","nonce":2647782276}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-09T06:36:57.929421+0000","modified":"2024-05-09T06:37:00.941894+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5103},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5103":{"gid":5103,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.3:6835/2138710335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6833","nonce":2138710335},{"type":"v1","addr":"172.21.15.3:6835","nonce":2138710335}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5103,"qdb_cluster":[5103]},"id":2}]} 2024-05-09T06:37:17.611 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 18 2024-05-09T06:37:17.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:17.632+0000 7f0aebf4f640 1 -- 172.21.15.3:0/3924152794 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f0ac806bfe0 msgr2=0x7f0ac806e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:17.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:17.632+0000 7f0aebf4f640 1 --2- 172.21.15.3:0/3924152794 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f0ac806bfe0 0x7f0ac806e4a0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f0ae0005880 tx=0x7f0ae00012d0 comp rx=0 tx=0).stop 2024-05-09T06:37:17.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:17.632+0000 7f0aebf4f640 1 -- 172.21.15.3:0/3924152794 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0ae415e480 msgr2=0x7f0ae41f7e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:17.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:17.632+0000 7f0aebf4f640 1 --2- 172.21.15.3:0/3924152794 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0ae415e480 0x7f0ae41f7e00 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f0ad400e910 tx=0x7f0ad400edd0 comp rx=0 tx=0).stop 2024-05-09T06:37:17.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:17.632+0000 7f0aebf4f640 1 -- 172.21.15.3:0/3924152794 shutdown_connections 2024-05-09T06:37:17.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:17.632+0000 7f0aebf4f640 1 --2- 172.21.15.3:0/3924152794 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f0ac806bfe0 0x7f0ac806e4a0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:17.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:17.632+0000 7f0aebf4f640 1 --2- 172.21.15.3:0/3924152794 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0ae4166390 0x7f0ae41f8340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:17.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:17.632+0000 7f0aebf4f640 1 --2- 172.21.15.3:0/3924152794 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0ae415e480 0x7f0ae41f7e00 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:17.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:17.632+0000 7f0aebf4f640 1 --2- 172.21.15.3:0/3924152794 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0ae40d27c0 0x7f0ae41592a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:17.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:17.632+0000 7f0aebf4f640 1 -- 172.21.15.3:0/3924152794 >> 172.21.15.3:0/3924152794 conn(0x7f0ae409d4d0 msgr2=0x7f0ae40b4d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:17.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:17.632+0000 7f0aebf4f640 1 -- 172.21.15.3:0/3924152794 shutdown_connections 2024-05-09T06:37:17.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:17.632+0000 7f0aebf4f640 1 -- 172.21.15.3:0/3924152794 wait complete. 2024-05-09T06:37:17.623 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T06:37:17.727 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:17.747+0000 7f144dfa2640 1 Processor -- start 2024-05-09T06:37:17.727 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:17.747+0000 7f144dfa2640 1 -- start start 2024-05-09T06:37:17.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:17.747+0000 7f144dfa2640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f144815e480 0x7f14480d1dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:17.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:17.747+0000 7f144dfa2640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f14480d2310 0x7f14480d27c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:17.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:17.747+0000 7f144dfa2640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f14480d2d00 0x7f1448162070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:17.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:17.747+0000 7f144dfa2640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f14480d4e50 con 0x7f14480d2d00 2024-05-09T06:37:17.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:17.747+0000 7f144dfa2640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f14480d0270 con 0x7f14480d2310 2024-05-09T06:37:17.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:17.747+0000 7f144dfa2640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f14480cf1d0 con 0x7f144815e480 2024-05-09T06:37:17.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:17.748+0000 7f1447fff640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f14480d2d00 0x7f1448162070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:17.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:17.748+0000 7f1447fff640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f14480d2d00 0x7f1448162070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51162/0 (socket says 172.21.15.3:51162) 2024-05-09T06:37:17.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:17.748+0000 7f1447fff640 1 -- 172.21.15.3:0/315410042 learned_addr learned my addr 172.21.15.3:0/315410042 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:37:17.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:17.748+0000 7f1446ffd640 1 --2- 172.21.15.3:0/315410042 >> v2:172.21.15.190:3300/0 conn(0x7f14480d2310 0x7f14480d27c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:17.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:17.748+0000 7f14477fe640 1 --2- 172.21.15.3:0/315410042 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f144815e480 0x7f14480d1dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:17.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:17.748+0000 7f1447fff640 1 -- 172.21.15.3:0/315410042 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f144815e480 msgr2=0x7f14480d1dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:37:17.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:17.748+0000 7f1447fff640 1 --2- 172.21.15.3:0/315410042 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f144815e480 0x7f14480d1dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:17.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:17.748+0000 7f1447fff640 1 -- 172.21.15.3:0/315410042 >> v2:172.21.15.190:3300/0 conn(0x7f14480d2310 msgr2=0x7f14480d27c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:37:17.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:17.748+0000 7f1447fff640 1 --2- 172.21.15.3:0/315410042 >> v2:172.21.15.190:3300/0 conn(0x7f14480d2310 0x7f14480d27c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:17.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:17.748+0000 7f1447fff640 1 -- 172.21.15.3:0/315410042 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14480cf4b0 con 0x7f14480d2d00 2024-05-09T06:37:17.731 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:17.749+0000 7f1447fff640 1 --2- 172.21.15.3:0/315410042 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f14480d2d00 0x7f1448162070 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f144808c8e0 tx=0x7f144002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6bed875bbf260e0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:17.731 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:17.749+0000 7f14467fc640 1 -- 172.21.15.3:0/315410042 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1440036070 con 0x7f14480d2d00 2024-05-09T06:37:17.731 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:17.749+0000 7f14467fc640 1 -- 172.21.15.3:0/315410042 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1440031600 con 0x7f14480d2d00 2024-05-09T06:37:17.731 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:17.749+0000 7f14467fc640 1 -- 172.21.15.3:0/315410042 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14400319d0 con 0x7f14480d2d00 2024-05-09T06:37:17.731 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:17.749+0000 7f144dfa2640 1 -- 172.21.15.3:0/315410042 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f14480d2d00 msgr2=0x7f1448162070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:17.731 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:17.749+0000 7f144dfa2640 1 --2- 172.21.15.3:0/315410042 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f14480d2d00 0x7f1448162070 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f144808c8e0 tx=0x7f144002a6a0 comp rx=0 tx=0).stop 2024-05-09T06:37:17.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:17.749+0000 7f144dfa2640 1 -- 172.21.15.3:0/315410042 shutdown_connections 2024-05-09T06:37:17.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:17.749+0000 7f144dfa2640 1 --2- 172.21.15.3:0/315410042 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f14480d2d00 0x7f1448162070 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:17.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:17.749+0000 7f144dfa2640 1 --2- 172.21.15.3:0/315410042 >> v2:172.21.15.190:3300/0 conn(0x7f14480d2310 0x7f14480d27c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:17.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:17.749+0000 7f144dfa2640 1 --2- 172.21.15.3:0/315410042 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f144815e480 0x7f14480d1dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:17.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:17.749+0000 7f144dfa2640 1 -- 172.21.15.3:0/315410042 >> 172.21.15.3:0/315410042 conn(0x7f144809d4d0 msgr2=0x7f14480d4200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:17.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:17.749+0000 7f144dfa2640 1 -- 172.21.15.3:0/315410042 shutdown_connections 2024-05-09T06:37:17.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:17.749+0000 7f144dfa2640 1 -- 172.21.15.3:0/315410042 wait complete. 2024-05-09T06:37:17.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:17.751+0000 7f144dfa2640 1 Processor -- start 2024-05-09T06:37:17.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:17.751+0000 7f144dfa2640 1 -- start start 2024-05-09T06:37:17.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:17.751+0000 7f144dfa2640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f14480d2310 0x7f14480cb2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:17.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:17.751+0000 7f144dfa2640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f14480d2d00 0x7f144815d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:17.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:17.751+0000 7f144dfa2640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f144815e480 0x7f14481504c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:17.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:17.751+0000 7f144dfa2640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f14480cfd50 con 0x7f14480d2310 2024-05-09T06:37:17.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:17.751+0000 7f144dfa2640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f14480cfa70 con 0x7f144815e480 2024-05-09T06:37:17.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:17.751+0000 7f144dfa2640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f14480b5ab0 con 0x7f14480d2d00 2024-05-09T06:37:17.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:17.752+0000 7f14477fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f14480d2310 0x7f14480cb2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:17.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:17.752+0000 7f14477fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f14480d2310 0x7f14480cb2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51172/0 (socket says 172.21.15.3:51172) 2024-05-09T06:37:17.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:17.752+0000 7f14477fe640 1 -- 172.21.15.3:0/3406101859 learned_addr learned my addr 172.21.15.3:0/3406101859 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:37:17.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:17.752+0000 7f1446ffd640 1 --2- 172.21.15.3:0/3406101859 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f14480d2d00 0x7f144815d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:17.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:17.752+0000 7f1447fff640 1 --2- 172.21.15.3:0/3406101859 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f144815e480 0x7f14481504c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:17.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:17.752+0000 7f14477fe640 1 -- 172.21.15.3:0/3406101859 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f14480d2d00 msgr2=0x7f144815d1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:17.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:17.752+0000 7f14477fe640 1 --2- 172.21.15.3:0/3406101859 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f14480d2d00 0x7f144815d1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:17.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:17.752+0000 7f14477fe640 1 -- 172.21.15.3:0/3406101859 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f144815e480 msgr2=0x7f14481504c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:17.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:17.752+0000 7f14477fe640 1 --2- 172.21.15.3:0/3406101859 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f144815e480 0x7f14481504c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:17.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:17.752+0000 7f14477fe640 1 -- 172.21.15.3:0/3406101859 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14480cd860 con 0x7f14480d2310 2024-05-09T06:37:17.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:17.752+0000 7f14477fe640 1 --2- 172.21.15.3:0/3406101859 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f14480d2310 0x7f14480cb2d0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f143800e9b0 tx=0x7f143800ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:17.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:17.752+0000 7f143ffff640 1 -- 172.21.15.3:0/3406101859 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1438017070 con 0x7f14480d2310 2024-05-09T06:37:17.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:17.752+0000 7f143ffff640 1 -- 172.21.15.3:0/3406101859 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f143800c9a0 con 0x7f14480d2310 2024-05-09T06:37:17.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:17.752+0000 7f144dfa2640 1 -- 172.21.15.3:0/3406101859 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14480cdb40 con 0x7f14480d2310 2024-05-09T06:37:17.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:17.752+0000 7f143ffff640 1 -- 172.21.15.3:0/3406101859 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f143800cd90 con 0x7f14480d2310 2024-05-09T06:37:17.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:17.752+0000 7f144dfa2640 1 -- 172.21.15.3:0/3406101859 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14480ce100 con 0x7f14480d2310 2024-05-09T06:37:17.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:17.753+0000 7f144dfa2640 1 -- 172.21.15.3:0/3406101859 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f144804f660 con 0x7f14480d2310 2024-05-09T06:37:17.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:17.754+0000 7f143ffff640 1 -- 172.21.15.3:0/3406101859 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 7) ==== 90651+0+0 (secure 0 0 0) 0x7f143801a070 con 0x7f14480d2310 2024-05-09T06:37:17.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:17.754+0000 7f143ffff640 1 --2- 172.21.15.3:0/3406101859 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f142806c010 0x7f142806e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:17.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:17.755+0000 7f143ffff640 1 -- 172.21.15.3:0/3406101859 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f143808b390 con 0x7f14480d2310 2024-05-09T06:37:17.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:17.755+0000 7f1446ffd640 1 --2- 172.21.15.3:0/3406101859 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f142806c010 0x7f142806e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:17.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:17.755+0000 7f1446ffd640 1 --2- 172.21.15.3:0/3406101859 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f142806c010 0x7f142806e4d0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f14480d2770 tx=0x7f1430009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:17.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:17.759+0000 7f143ffff640 1 -- 172.21.15.3:0/3406101859 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f1438059520 con 0x7f14480d2310 2024-05-09T06:37:17.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:17.982+0000 7f144dfa2640 1 -- 172.21.15.3:0/3406101859 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f14480d18f0 con 0x7f14480d2310 2024-05-09T06:37:17.963 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:17.982+0000 7f143ffff640 1 -- 172.21.15.3:0/3406101859 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+5181 (secure 0 0 0) 0x7f143805d820 con 0x7f14480d2310 2024-05-09T06:37:17.963 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-09T06:37:17.963 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4912,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6836/2400645940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":2400645940},{"type":"v1","addr":"172.21.15.190:6836","nonce":2400645940}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4915,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6837/3849729576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6835","nonce":3849729576},{"type":"v1","addr":"172.21.15.190:6837","nonce":3849729576}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4918,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6834/2647782276","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":2647782276},{"type":"v1","addr":"172.21.15.3:6834","nonce":2647782276}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-09T06:36:57.929421+0000","modified":"2024-05-09T06:37:00.941894+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5103},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5103":{"gid":5103,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.3:6835/2138710335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6833","nonce":2138710335},{"type":"v1","addr":"172.21.15.3:6835","nonce":2138710335}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5103,"qdb_cluster":[5103]},"id":2}]} 2024-05-09T06:37:17.963 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 18 2024-05-09T06:37:17.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:17.984+0000 7f144dfa2640 1 -- 172.21.15.3:0/3406101859 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f142806c010 msgr2=0x7f142806e4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:17.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:17.984+0000 7f144dfa2640 1 --2- 172.21.15.3:0/3406101859 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f142806c010 0x7f142806e4d0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f14480d2770 tx=0x7f1430009190 comp rx=0 tx=0).stop 2024-05-09T06:37:17.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:17.984+0000 7f144dfa2640 1 -- 172.21.15.3:0/3406101859 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f14480d2310 msgr2=0x7f14480cb2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:17.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:17.984+0000 7f144dfa2640 1 --2- 172.21.15.3:0/3406101859 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f14480d2310 0x7f14480cb2d0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f143800e9b0 tx=0x7f143800ee70 comp rx=0 tx=0).stop 2024-05-09T06:37:17.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:17.984+0000 7f144dfa2640 1 -- 172.21.15.3:0/3406101859 shutdown_connections 2024-05-09T06:37:17.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:17.984+0000 7f144dfa2640 1 --2- 172.21.15.3:0/3406101859 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f142806c010 0x7f142806e4d0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:17.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:17.984+0000 7f144dfa2640 1 --2- 172.21.15.3:0/3406101859 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f144815e480 0x7f14481504c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:17.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:17.984+0000 7f144dfa2640 1 --2- 172.21.15.3:0/3406101859 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f14480d2d00 0x7f144815d1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:17.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:17.984+0000 7f144dfa2640 1 --2- 172.21.15.3:0/3406101859 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f14480d2310 0x7f14480cb2d0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:17.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:17.984+0000 7f144dfa2640 1 -- 172.21.15.3:0/3406101859 >> 172.21.15.3:0/3406101859 conn(0x7f144809d4d0 msgr2=0x7f14480b2350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:17.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:17.985+0000 7f144dfa2640 1 -- 172.21.15.3:0/3406101859 shutdown_connections 2024-05-09T06:37:17.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:17.985+0000 7f144dfa2640 1 -- 172.21.15.3:0/3406101859 wait complete. 2024-05-09T06:37:17.976 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T06:37:18.080 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.100+0000 7fbfa235a640 1 Processor -- start 2024-05-09T06:37:18.080 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.100+0000 7fbfa235a640 1 -- start start 2024-05-09T06:37:18.081 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.100+0000 7fbfa235a640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbf9c15e480 0x7fbf9c0d1dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:18.081 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.100+0000 7fbfa235a640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fbf9c0d2310 0x7fbf9c0d27c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:18.081 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.100+0000 7fbfa235a640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbf9c0d4e50 con 0x7fbf9c0d2d00 2024-05-09T06:37:18.081 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.100+0000 7fbfa235a640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7fbf9c0d0270 con 0x7fbf9c0d2310 2024-05-09T06:37:18.081 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.100+0000 7fbfa235a640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fbf9c0cf1d0 con 0x7fbf9c15e480 2024-05-09T06:37:18.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.101+0000 7fbf9b7fe640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fbf9c0d2310 0x7fbf9c0d27c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:18.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.101+0000 7fbf9bfff640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbf9c15e480 0x7fbf9c0d1dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:18.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.101+0000 7fbf9b7fe640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fbf9c0d2310 0x7fbf9c0d27c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.3:43548/0 (socket says 172.21.15.3:43548) 2024-05-09T06:37:18.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.101+0000 7fbf9b7fe640 1 -- 172.21.15.3:0/294428093 learned_addr learned my addr 172.21.15.3:0/294428093 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:37:18.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.101+0000 7fbfa08f7640 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7fbf9c0d2d00 0x7fbf9c162070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.3:42506/0 (socket says 172.21.15.3:42506) 2024-05-09T06:37:18.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.101+0000 7fbf9b7fe640 1 -- 172.21.15.3:0/294428093 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbf9c15e480 msgr2=0x7fbf9c0d1dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:37:18.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.101+0000 7fbf9b7fe640 1 --2- 172.21.15.3:0/294428093 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbf9c15e480 0x7fbf9c0d1dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:18.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.101+0000 7fbf9b7fe640 1 -- 172.21.15.3:0/294428093 >> v1:172.21.15.190:6789/0 conn(0x7fbf9c0d2d00 legacy=0x7fbf9c162070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:18.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.101+0000 7fbf9b7fe640 1 -- 172.21.15.3:0/294428093 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf9c0cf4b0 con 0x7fbf9c0d2310 2024-05-09T06:37:18.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.102+0000 7fbf9b7fe640 1 --2- 172.21.15.3:0/294428093 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbf9c0d2310 0x7fbf9c0d27c0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fbf90009870 tx=0x7fbf9002a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b048b18027da9103 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:18.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.102+0000 7fbf9affd640 1 -- 172.21.15.3:0/294428093 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf90037070 con 0x7fbf9c0d2310 2024-05-09T06:37:18.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.102+0000 7fbf9affd640 1 -- 172.21.15.3:0/294428093 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf90032440 con 0x7fbf9c0d2310 2024-05-09T06:37:18.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.102+0000 7fbf9affd640 1 -- 172.21.15.3:0/294428093 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf90032810 con 0x7fbf9c0d2310 2024-05-09T06:37:18.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.103+0000 7fbfa235a640 1 -- 172.21.15.3:0/294428093 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbf9c0d2310 msgr2=0x7fbf9c0d27c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:18.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.103+0000 7fbfa235a640 1 --2- 172.21.15.3:0/294428093 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbf9c0d2310 0x7fbf9c0d27c0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fbf90009870 tx=0x7fbf9002a270 comp rx=0 tx=0).stop 2024-05-09T06:37:18.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.103+0000 7fbfa235a640 1 -- 172.21.15.3:0/294428093 shutdown_connections 2024-05-09T06:37:18.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.103+0000 7fbfa235a640 1 --2- 172.21.15.3:0/294428093 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbf9c0d2310 0x7fbf9c0d27c0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:18.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.103+0000 7fbfa235a640 1 --2- 172.21.15.3:0/294428093 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbf9c15e480 0x7fbf9c0d1dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:18.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.103+0000 7fbfa235a640 1 -- 172.21.15.3:0/294428093 >> 172.21.15.3:0/294428093 conn(0x7fbf9c09d4d0 msgr2=0x7fbf9c0d4200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:18.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.103+0000 7fbfa235a640 1 -- 172.21.15.3:0/294428093 shutdown_connections 2024-05-09T06:37:18.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.103+0000 7fbfa235a640 1 -- 172.21.15.3:0/294428093 wait complete. 2024-05-09T06:37:18.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.104+0000 7fbfa235a640 1 Processor -- start 2024-05-09T06:37:18.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.104+0000 7fbfa235a640 1 -- start start 2024-05-09T06:37:18.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.105+0000 7fbfa235a640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbf9c0d2310 0x7fbf9c157eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:18.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.105+0000 7fbfa235a640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbf9c0d2d00 0x7fbf9c0b4ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:18.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.105+0000 7fbfa235a640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbf9c15e480 0x7fbf9c0b8ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:18.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.105+0000 7fbfa235a640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fbf9c0cfd50 con 0x7fbf9c0d2310 2024-05-09T06:37:18.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.105+0000 7fbfa235a640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fbf9c0cfa70 con 0x7fbf9c0d2d00 2024-05-09T06:37:18.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.105+0000 7fbfa235a640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fbf9c0b5ab0 con 0x7fbf9c15e480 2024-05-09T06:37:18.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.105+0000 7fbf9bfff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbf9c0d2310 0x7fbf9c157eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:18.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.105+0000 7fbf9bfff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbf9c0d2310 0x7fbf9c157eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51184/0 (socket says 172.21.15.3:51184) 2024-05-09T06:37:18.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.105+0000 7fbf9bfff640 1 -- 172.21.15.3:0/2736079514 learned_addr learned my addr 172.21.15.3:0/2736079514 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:37:18.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.106+0000 7fbfa08f7640 1 --2- 172.21.15.3:0/2736079514 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbf9c15e480 0x7fbf9c0b8ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:18.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.106+0000 7fbf9b7fe640 1 --2- 172.21.15.3:0/2736079514 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbf9c0d2d00 0x7fbf9c0b4ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:18.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.106+0000 7fbf9bfff640 1 -- 172.21.15.3:0/2736079514 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbf9c15e480 msgr2=0x7fbf9c0b8ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:18.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.106+0000 7fbf9bfff640 1 --2- 172.21.15.3:0/2736079514 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbf9c15e480 0x7fbf9c0b8ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:18.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.106+0000 7fbf9bfff640 1 -- 172.21.15.3:0/2736079514 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbf9c0d2d00 msgr2=0x7fbf9c0b4ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:18.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.106+0000 7fbf9bfff640 1 --2- 172.21.15.3:0/2736079514 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbf9c0d2d00 0x7fbf9c0b4ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:18.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.106+0000 7fbf9bfff640 1 -- 172.21.15.3:0/2736079514 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf9c0c7fa0 con 0x7fbf9c0d2310 2024-05-09T06:37:18.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.106+0000 7fbf9bfff640 1 --2- 172.21.15.3:0/2736079514 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbf9c0d2310 0x7fbf9c157eb0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fbf8c00e9b0 tx=0x7fbf8c00ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:18.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.106+0000 7fbf98ff9640 1 -- 172.21.15.3:0/2736079514 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf8c017070 con 0x7fbf9c0d2310 2024-05-09T06:37:18.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.106+0000 7fbf98ff9640 1 -- 172.21.15.3:0/2736079514 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf8c00c9a0 con 0x7fbf9c0d2310 2024-05-09T06:37:18.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.106+0000 7fbfa235a640 1 -- 172.21.15.3:0/2736079514 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf9c15aba0 con 0x7fbf9c0d2310 2024-05-09T06:37:18.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.106+0000 7fbf98ff9640 1 -- 172.21.15.3:0/2736079514 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf8c00cd90 con 0x7fbf9c0d2310 2024-05-09T06:37:18.090 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.106+0000 7fbfa235a640 1 -- 172.21.15.3:0/2736079514 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf9c15b160 con 0x7fbf9c0d2310 2024-05-09T06:37:18.090 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.107+0000 7fbfa235a640 1 -- 172.21.15.3:0/2736079514 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf9c04f660 con 0x7fbf9c0d2310 2024-05-09T06:37:18.090 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.108+0000 7fbf98ff9640 1 -- 172.21.15.3:0/2736079514 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 7) ==== 90651+0+0 (secure 0 0 0) 0x7fbf8c01a070 con 0x7fbf9c0d2310 2024-05-09T06:37:18.090 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.108+0000 7fbf98ff9640 1 --2- 172.21.15.3:0/2736079514 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fbf8806bfe0 0x7fbf8806e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:18.091 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.108+0000 7fbf98ff9640 1 -- 172.21.15.3:0/2736079514 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fbf8c08b310 con 0x7fbf9c0d2310 2024-05-09T06:37:18.091 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.109+0000 7fbf9b7fe640 1 --2- 172.21.15.3:0/2736079514 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fbf8806bfe0 0x7fbf8806e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:18.091 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.109+0000 7fbf9b7fe640 1 --2- 172.21.15.3:0/2736079514 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fbf8806bfe0 0x7fbf8806e4a0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fbf90034000 tx=0x7fbf9002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:18.093 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.113+0000 7fbf98ff9640 1 -- 172.21.15.3:0/2736079514 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fbf9c0c7fa0 con 0x7fbf9c0d2310 2024-05-09T06:37:18.283 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.302+0000 7fbfa235a640 1 -- 172.21.15.3:0/2736079514 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbf9c0d18f0 con 0x7fbf9c0d2310 2024-05-09T06:37:18.284 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.303+0000 7fbf98ff9640 1 -- 172.21.15.3:0/2736079514 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) ==== 74+0+16585 (secure 0 0 0) 0x7fbf8c059930 con 0x7fbf9c0d2310 2024-05-09T06:37:18.284 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-09T06:37:18.284 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":33,"fsid":"16ed737d-2665-431a-8ac6-ef242be3975a","created":"2024-05-09T06:35:42.541368+0000","modified":"2024-05-09T06:36:59.564366+0000","last_up_change":"2024-05-09T06:35:53.778274+0000","last_in_change":"2024-05-09T06:35:44.780586+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T06:35:54.819999+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-05-09T06:36:56.494700+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-05-09T06:36:56.900006+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fae3a8db-4826-4f3c-b33a-12c09bb5df6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":1897849827},{"type":"v1","addr":"172.21.15.3:6817","nonce":1897849827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":1897849827},{"type":"v1","addr":"172.21.15.3:6819","nonce":1897849827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":1897849827},{"type":"v1","addr":"172.21.15.3:6823","nonce":1897849827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":1897849827},{"type":"v1","addr":"172.21.15.3:6821","nonce":1897849827}]},"public_addr":"172.21.15.3:6817/1897849827","cluster_addr":"172.21.15.3:6819/1897849827","heartbeat_back_addr":"172.21.15.3:6823/1897849827","heartbeat_front_addr":"172.21.15.3:6821/1897849827","state":["exists","up"]},{"osd":1,"uuid":"646e882d-4ce4-4720-8c84-f71df563a35e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":926805498},{"type":"v1","addr":"172.21.15.3:6801","nonce":926805498}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":926805498},{"type":"v1","addr":"172.21.15.3:6803","nonce":926805498}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":926805498},{"type":"v1","addr":"172.21.15.3:6807","nonce":926805498}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":926805498},{"type":"v1","addr":"172.21.15.3:6805","nonce":926805498}]},"public_addr":"172.21.15.3:6801/926805498","cluster_addr":"172.21.15.3:6803/926805498","heartbeat_back_addr":"172.21.15.3:6807/926805498","heartbeat_front_addr":"172.21.15.3:6805/926805498","state":["exists","up"]},{"osd":2,"uuid":"1abbadb5-a570-4359-9643-acff3ede35a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":801295451},{"type":"v1","addr":"172.21.15.3:6809","nonce":801295451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":801295451},{"type":"v1","addr":"172.21.15.3:6811","nonce":801295451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":801295451},{"type":"v1","addr":"172.21.15.3:6815","nonce":801295451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":801295451},{"type":"v1","addr":"172.21.15.3:6813","nonce":801295451}]},"public_addr":"172.21.15.3:6809/801295451","cluster_addr":"172.21.15.3:6811/801295451","heartbeat_back_addr":"172.21.15.3:6815/801295451","heartbeat_front_addr":"172.21.15.3:6813/801295451","state":["exists","up"]},{"osd":3,"uuid":"62ac4dc1-2371-4dd2-acd3-e2dbb11aedf5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":1903389857},{"type":"v1","addr":"172.21.15.3:6825","nonce":1903389857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":1903389857},{"type":"v1","addr":"172.21.15.3:6827","nonce":1903389857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":1903389857},{"type":"v1","addr":"172.21.15.3:6831","nonce":1903389857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":1903389857},{"type":"v1","addr":"172.21.15.3:6829","nonce":1903389857}]},"public_addr":"172.21.15.3:6825/1903389857","cluster_addr":"172.21.15.3:6827/1903389857","heartbeat_back_addr":"172.21.15.3:6831/1903389857","heartbeat_front_addr":"172.21.15.3:6829/1903389857","state":["exists","up"]},{"osd":4,"uuid":"ae808398-8326-4d7c-9319-a2a0eafffb59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":2143624576},{"type":"v1","addr":"172.21.15.190:6801","nonce":2143624576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":2143624576},{"type":"v1","addr":"172.21.15.190:6803","nonce":2143624576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":2143624576},{"type":"v1","addr":"172.21.15.190:6807","nonce":2143624576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":2143624576},{"type":"v1","addr":"172.21.15.190:6805","nonce":2143624576}]},"public_addr":"172.21.15.190:6801/2143624576","cluster_addr":"172.21.15.190:6803/2143624576","heartbeat_back_addr":"172.21.15.190:6807/2143624576","heartbeat_front_addr":"172.21.15.190:6805/2143624576","state":["exists","up"]},{"osd":5,"uuid":"918adf50-c440-48ac-9a8d-4837562d7275","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":1766923126},{"type":"v1","addr":"172.21.15.190:6814","nonce":1766923126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":1766923126},{"type":"v1","addr":"172.21.15.190:6818","nonce":1766923126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":1766923126},{"type":"v1","addr":"172.21.15.190:6823","nonce":1766923126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":1766923126},{"type":"v1","addr":"172.21.15.190:6821","nonce":1766923126}]},"public_addr":"172.21.15.190:6814/1766923126","cluster_addr":"172.21.15.190:6818/1766923126","heartbeat_back_addr":"172.21.15.190:6823/1766923126","heartbeat_front_addr":"172.21.15.190:6821/1766923126","state":["exists","up"]},{"osd":6,"uuid":"ac2882bc-2a7c-463c-8f6c-76ff49828adc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":342414248},{"type":"v1","addr":"172.21.15.190:6825","nonce":342414248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":342414248},{"type":"v1","addr":"172.21.15.190:6827","nonce":342414248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":342414248},{"type":"v1","addr":"172.21.15.190:6831","nonce":342414248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":342414248},{"type":"v1","addr":"172.21.15.190:6829","nonce":342414248}]},"public_addr":"172.21.15.190:6825/342414248","cluster_addr":"172.21.15.190:6827/342414248","heartbeat_back_addr":"172.21.15.190:6831/342414248","heartbeat_front_addr":"172.21.15.190:6829/342414248","state":["exists","up"]},{"osd":7,"uuid":"dd4e767a-4730-4c70-8bd5-3f088140a5a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":537849891},{"type":"v1","addr":"172.21.15.190:6809","nonce":537849891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":537849891},{"type":"v1","addr":"172.21.15.190:6811","nonce":537849891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6817","nonce":537849891},{"type":"v1","addr":"172.21.15.190:6819","nonce":537849891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6813","nonce":537849891},{"type":"v1","addr":"172.21.15.190:6815","nonce":537849891}]},"public_addr":"172.21.15.190:6809/537849891","cluster_addr":"172.21.15.190:6811/537849891","heartbeat_back_addr":"172.21.15.190:6819/537849891","heartbeat_front_addr":"172.21.15.190:6815/537849891","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.889784+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.974002+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.973549+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.955396+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.928471+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.902418+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.958696+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.926474+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T06:37:18.287 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.305+0000 7fbfa235a640 1 -- 172.21.15.3:0/2736079514 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fbf8806bfe0 msgr2=0x7fbf8806e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:18.287 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.305+0000 7fbfa235a640 1 --2- 172.21.15.3:0/2736079514 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fbf8806bfe0 0x7fbf8806e4a0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fbf90034000 tx=0x7fbf9002e000 comp rx=0 tx=0).stop 2024-05-09T06:37:18.287 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.305+0000 7fbfa235a640 1 -- 172.21.15.3:0/2736079514 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbf9c0d2310 msgr2=0x7fbf9c157eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:18.287 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.305+0000 7fbfa235a640 1 --2- 172.21.15.3:0/2736079514 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbf9c0d2310 0x7fbf9c157eb0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fbf8c00e9b0 tx=0x7fbf8c00ee70 comp rx=0 tx=0).stop 2024-05-09T06:37:18.287 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.305+0000 7fbfa235a640 1 -- 172.21.15.3:0/2736079514 shutdown_connections 2024-05-09T06:37:18.287 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.305+0000 7fbfa235a640 1 --2- 172.21.15.3:0/2736079514 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fbf8806bfe0 0x7fbf8806e4a0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:18.287 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.305+0000 7fbfa235a640 1 --2- 172.21.15.3:0/2736079514 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbf9c15e480 0x7fbf9c0b8ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:18.287 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.305+0000 7fbfa235a640 1 --2- 172.21.15.3:0/2736079514 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbf9c0d2d00 0x7fbf9c0b4ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:18.287 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.305+0000 7fbfa235a640 1 --2- 172.21.15.3:0/2736079514 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbf9c0d2310 0x7fbf9c157eb0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:18.287 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.305+0000 7fbfa235a640 1 -- 172.21.15.3:0/2736079514 >> 172.21.15.3:0/2736079514 conn(0x7fbf9c09d4d0 msgr2=0x7fbf9c0b3af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:18.288 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.306+0000 7fbfa235a640 1 -- 172.21.15.3:0/2736079514 shutdown_connections 2024-05-09T06:37:18.288 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.306+0000 7fbfa235a640 1 -- 172.21.15.3:0/2736079514 wait complete. 2024-05-09T06:37:18.298 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-05-09T06:37:18.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.422+0000 7fb32ebfb640 1 Processor -- start 2024-05-09T06:37:18.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.422+0000 7fb32ebfb640 1 -- start start 2024-05-09T06:37:18.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.423+0000 7fb32ebfb640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fb32806cd70 0x7fb328162a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:18.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.423+0000 7fb32ebfb640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb3280cc750 0x7fb3280ccbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:18.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.423+0000 7fb32ebfb640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb328162fa0 con 0x7fb3280d02b0 2024-05-09T06:37:18.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.423+0000 7fb32ebfb640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7fb328161e40 con 0x7fb32806cd70 2024-05-09T06:37:18.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.423+0000 7fb32ebfb640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fb3280d0870 con 0x7fb3280cc750 2024-05-09T06:37:18.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.423+0000 7fb31ffff640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fb3280d02b0 0x7fb32806d150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:45982/0 (socket says 172.21.15.3:45982) 2024-05-09T06:37:18.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.423+0000 7fb31ffff640 1 -- 172.21.15.3:0/4240667759 learned_addr learned my addr 172.21.15.3:0/4240667759 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-05-09T06:37:18.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.423+0000 7fb32c997640 1 --2- 172.21.15.3:0/4240667759 >> v2:172.21.15.190:3300/0 conn(0x7fb32806cd70 0x7fb328162a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:18.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.423+0000 7fb31f7fe640 1 -- 172.21.15.3:0/4240667759 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2219430288 0 0) 0x7fb328162fa0 con 0x7fb3280d02b0 2024-05-09T06:37:18.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.423+0000 7fb32d198640 1 --2- 172.21.15.3:0/4240667759 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb3280cc750 0x7fb3280ccbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:18.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.423+0000 7fb31f7fe640 1 -- 172.21.15.3:0/4240667759 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb310003520 con 0x7fb3280d02b0 2024-05-09T06:37:18.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.424+0000 7fb31f7fe640 1 -- 172.21.15.3:0/4240667759 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 868136499 0 0) 0x7fb310003520 con 0x7fb3280d02b0 2024-05-09T06:37:18.407 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.424+0000 7fb31f7fe640 1 -- 172.21.15.3:0/4240667759 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb3280cc750 msgr2=0x7fb3280ccbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:37:18.407 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.424+0000 7fb31f7fe640 1 --2- 172.21.15.3:0/4240667759 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb3280cc750 0x7fb3280ccbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:18.407 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.424+0000 7fb31f7fe640 1 -- 172.21.15.3:0/4240667759 >> v2:172.21.15.190:3300/0 conn(0x7fb32806cd70 msgr2=0x7fb328162a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:37:18.407 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.424+0000 7fb31f7fe640 1 --2- 172.21.15.3:0/4240667759 >> v2:172.21.15.190:3300/0 conn(0x7fb32806cd70 0x7fb328162a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:18.407 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.424+0000 7fb31f7fe640 1 -- 172.21.15.3:0/4240667759 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3280d0b50 con 0x7fb3280d02b0 2024-05-09T06:37:18.407 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.424+0000 7fb31f7fe640 1 -- 172.21.15.3:0/4240667759 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 563312031 0 0) 0x7fb318001670 con 0x7fb3280d02b0 2024-05-09T06:37:18.407 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.424+0000 7fb31f7fe640 1 -- 172.21.15.3:0/4240667759 >> v1:172.21.15.3:6789/0 conn(0x7fb3280d02b0 legacy=0x7fb32806d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:18.407 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.424+0000 7fb31f7fe640 1 --2- 172.21.15.3:0/4240667759 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb3100039b0 0x7fb310003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:18.407 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.424+0000 7fb31f7fe640 1 --2- 172.21.15.3:0/4240667759 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb310004800 0x7fb310004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:18.407 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.424+0000 7fb31f7fe640 1 --2- 172.21.15.3:0/4240667759 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb3280cc750 0x7fb3100051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:18.407 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.424+0000 7fb31f7fe640 1 -- 172.21.15.3:0/4240667759 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fb328161e40 con 0x7fb3100039b0 2024-05-09T06:37:18.407 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.424+0000 7fb31f7fe640 1 -- 172.21.15.3:0/4240667759 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fb3280d0870 con 0x7fb310004800 2024-05-09T06:37:18.407 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.424+0000 7fb31f7fe640 1 -- 172.21.15.3:0/4240667759 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fb31000b9c0 con 0x7fb3280cc750 2024-05-09T06:37:18.407 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.424+0000 7fb32c997640 1 --2- 172.21.15.3:0/4240667759 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb3100039b0 0x7fb310003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:18.407 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.424+0000 7fb32d198640 1 --2- 172.21.15.3:0/4240667759 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb3280cc750 0x7fb3100051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:18.408 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.424+0000 7fb31ffff640 1 --2- 172.21.15.3:0/4240667759 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb310004800 0x7fb310004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:18.408 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.424+0000 7fb32c997640 1 -- 172.21.15.3:0/4240667759 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb3280cc750 msgr2=0x7fb3100051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:37:18.408 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.424+0000 7fb32c997640 1 --2- 172.21.15.3:0/4240667759 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb3280cc750 0x7fb3100051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:18.408 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.424+0000 7fb32c997640 1 -- 172.21.15.3:0/4240667759 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb310004800 msgr2=0x7fb310004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:37:18.408 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.424+0000 7fb32c997640 1 --2- 172.21.15.3:0/4240667759 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb310004800 0x7fb310004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:18.408 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.424+0000 7fb32c997640 1 -- 172.21.15.3:0/4240667759 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb310003520 con 0x7fb3100039b0 2024-05-09T06:37:18.408 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.425+0000 7fb32c997640 1 --2- 172.21.15.3:0/4240667759 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb3100039b0 0x7fb310003dd0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fb324009ac0 tx=0x7fb310005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f456a935f9a75860 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:18.408 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.425+0000 7fb31f7fe640 1 -- 172.21.15.3:0/4240667759 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb324036070 con 0x7fb3100039b0 2024-05-09T06:37:18.408 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.425+0000 7fb31f7fe640 1 -- 172.21.15.3:0/4240667759 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb324031400 con 0x7fb3100039b0 2024-05-09T06:37:18.408 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.425+0000 7fb31f7fe640 1 -- 172.21.15.3:0/4240667759 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb324009690 con 0x7fb3100039b0 2024-05-09T06:37:18.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.426+0000 7fb32ebfb640 1 -- 172.21.15.3:0/4240667759 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb3100039b0 msgr2=0x7fb310003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:18.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.426+0000 7fb32ebfb640 1 --2- 172.21.15.3:0/4240667759 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb3100039b0 0x7fb310003dd0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fb324009ac0 tx=0x7fb310005830 comp rx=0 tx=0).stop 2024-05-09T06:37:18.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.426+0000 7fb32d198640 1 -- 172.21.15.3:0/4240667759 reap_dead start 2024-05-09T06:37:18.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.426+0000 7fb32ebfb640 1 -- 172.21.15.3:0/4240667759 shutdown_connections 2024-05-09T06:37:18.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.426+0000 7fb32ebfb640 1 -- 172.21.15.3:0/4240667759 >> 172.21.15.3:0/4240667759 conn(0x7fb3280b3af0 msgr2=0x7fb3280b3f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:18.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.426+0000 7fb32ebfb640 1 -- 172.21.15.3:0/4240667759 shutdown_connections 2024-05-09T06:37:18.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.426+0000 7fb32ebfb640 1 -- 172.21.15.3:0/4240667759 wait complete. 2024-05-09T06:37:18.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.427+0000 7fb32ebfb640 1 Processor -- start 2024-05-09T06:37:18.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.427+0000 7fb32ebfb640 1 -- start start 2024-05-09T06:37:18.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.428+0000 7fb32ebfb640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb3100039b0 0x7fb3281f8540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:18.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.428+0000 7fb32ebfb640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb3280cb9c0 0x7fb32806fd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:18.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.428+0000 7fb32ebfb640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb32814ff50 0x7fb3281503b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:18.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.428+0000 7fb32ebfb640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fb3280d13f0 con 0x7fb3100039b0 2024-05-09T06:37:18.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.428+0000 7fb32ebfb640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fb3280d1110 con 0x7fb32814ff50 2024-05-09T06:37:18.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.428+0000 7fb32ebfb640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fb3280ccbe0 con 0x7fb3280cb9c0 2024-05-09T06:37:18.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.428+0000 7fb32c997640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb3100039b0 0x7fb3281f8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:18.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.428+0000 7fb32c997640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb3100039b0 0x7fb3281f8540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51196/0 (socket says 172.21.15.3:51196) 2024-05-09T06:37:18.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.428+0000 7fb32c997640 1 -- 172.21.15.3:0/2903564883 learned_addr learned my addr 172.21.15.3:0/2903564883 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:37:18.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.428+0000 7fb31ffff640 1 --2- 172.21.15.3:0/2903564883 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb3280cb9c0 0x7fb32806fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:18.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.428+0000 7fb32d198640 1 --2- 172.21.15.3:0/2903564883 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb32814ff50 0x7fb3281503b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:18.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.428+0000 7fb32c997640 1 -- 172.21.15.3:0/2903564883 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb3280cb9c0 msgr2=0x7fb32806fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:18.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.428+0000 7fb32c997640 1 --2- 172.21.15.3:0/2903564883 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb3280cb9c0 0x7fb32806fd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:18.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.428+0000 7fb32c997640 1 -- 172.21.15.3:0/2903564883 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb32814ff50 msgr2=0x7fb3281503b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:18.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.428+0000 7fb32c997640 1 --2- 172.21.15.3:0/2903564883 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb32814ff50 0x7fb3281503b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:18.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.428+0000 7fb32c997640 1 -- 172.21.15.3:0/2903564883 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3280ca920 con 0x7fb3100039b0 2024-05-09T06:37:18.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.428+0000 7fb31ffff640 1 --2- 172.21.15.3:0/2903564883 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb3280cb9c0 0x7fb32806fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:37:18.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.428+0000 7fb32c997640 1 --2- 172.21.15.3:0/2903564883 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb3100039b0 0x7fb3281f8540 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fb324034040 tx=0x7fb324004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:18.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.429+0000 7fb31d7fa640 1 -- 172.21.15.3:0/2903564883 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb324041020 con 0x7fb3100039b0 2024-05-09T06:37:18.414 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.429+0000 7fb31d7fa640 1 -- 172.21.15.3:0/2903564883 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb32403c470 con 0x7fb3100039b0 2024-05-09T06:37:18.414 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.429+0000 7fb32ebfb640 1 -- 172.21.15.3:0/2903564883 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3280cac00 con 0x7fb3100039b0 2024-05-09T06:37:18.414 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.429+0000 7fb31d7fa640 1 -- 172.21.15.3:0/2903564883 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb324036040 con 0x7fb3100039b0 2024-05-09T06:37:18.414 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.429+0000 7fb32ebfb640 1 -- 172.21.15.3:0/2903564883 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3280cb1c0 con 0x7fb3100039b0 2024-05-09T06:37:18.414 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.430+0000 7fb32ebfb640 1 -- 172.21.15.3:0/2903564883 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb328069550 con 0x7fb3100039b0 2024-05-09T06:37:18.414 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.430+0000 7fb31d7fa640 1 -- 172.21.15.3:0/2903564883 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 7) ==== 90651+0+0 (secure 0 0 0) 0x7fb324044020 con 0x7fb3100039b0 2024-05-09T06:37:18.414 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.431+0000 7fb31d7fa640 1 --2- 172.21.15.3:0/2903564883 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fb31406bfe0 0x7fb31406e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:18.414 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.431+0000 7fb31d7fa640 1 -- 172.21.15.3:0/2903564883 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fb3240ab810 con 0x7fb3100039b0 2024-05-09T06:37:18.414 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.431+0000 7fb31ffff640 1 --2- 172.21.15.3:0/2903564883 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fb31406bfe0 0x7fb31406e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:18.414 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.431+0000 7fb31ffff640 1 --2- 172.21.15.3:0/2903564883 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fb31406bfe0 0x7fb31406e4a0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fb318007fd0 tx=0x7fb318009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:18.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.435+0000 7fb31d7fa640 1 -- 172.21.15.3:0/2903564883 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fb32404b030 con 0x7fb3100039b0 2024-05-09T06:37:18.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.659+0000 7fb32ebfb640 1 -- 172.21.15.3:0/2903564883 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fb3280734c0 con 0x7fb3100039b0 2024-05-09T06:37:18.640 INFO:tasks.ceph.mon.a.smithi003.stderr:2024-05-09T06:37:18.659+0000 7f5fb0020640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-09T06:37:18.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.661+0000 7fb31d7fa640 1 -- 172.21.15.3:0/2903564883 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v19) ==== 161+0+0 (secure 0 0 0) 0x7fb324079fd0 con 0x7fb3100039b0 2024-05-09T06:37:18.641 INFO:teuthology.orchestra.run.smithi003.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-09T06:37:18.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.663+0000 7fb32ebfb640 1 -- 172.21.15.3:0/2903564883 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fb31406bfe0 msgr2=0x7fb31406e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:18.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.663+0000 7fb32ebfb640 1 --2- 172.21.15.3:0/2903564883 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fb31406bfe0 0x7fb31406e4a0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fb318007fd0 tx=0x7fb318009040 comp rx=0 tx=0).stop 2024-05-09T06:37:18.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.663+0000 7fb32ebfb640 1 -- 172.21.15.3:0/2903564883 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb3100039b0 msgr2=0x7fb3281f8540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:18.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.663+0000 7fb32ebfb640 1 --2- 172.21.15.3:0/2903564883 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb3100039b0 0x7fb3281f8540 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fb324034040 tx=0x7fb324004730 comp rx=0 tx=0).stop 2024-05-09T06:37:18.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.663+0000 7fb32ebfb640 1 -- 172.21.15.3:0/2903564883 shutdown_connections 2024-05-09T06:37:18.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.663+0000 7fb32ebfb640 1 --2- 172.21.15.3:0/2903564883 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fb31406bfe0 0x7fb31406e4a0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:18.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.663+0000 7fb32ebfb640 1 --2- 172.21.15.3:0/2903564883 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb32814ff50 0x7fb3281503b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:18.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.663+0000 7fb32ebfb640 1 --2- 172.21.15.3:0/2903564883 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb3280cb9c0 0x7fb32806fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:18.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.663+0000 7fb32ebfb640 1 --2- 172.21.15.3:0/2903564883 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb3100039b0 0x7fb3281f8540 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:18.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.663+0000 7fb32ebfb640 1 -- 172.21.15.3:0/2903564883 >> 172.21.15.3:0/2903564883 conn(0x7fb3280b3af0 msgr2=0x7fb328161fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:18.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.663+0000 7fb32ebfb640 1 -- 172.21.15.3:0/2903564883 shutdown_connections 2024-05-09T06:37:18.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.663+0000 7fb32ebfb640 1 -- 172.21.15.3:0/2903564883 wait complete. 2024-05-09T06:37:18.657 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-05-09T06:37:18.683 INFO:tasks.ceph.mds.c.smithi003.stdout:starting mds.c at 2024-05-09T06:37:18.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.786+0000 7f07ac74f640 1 Processor -- start 2024-05-09T06:37:18.767 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.786+0000 7f07ac74f640 1 -- start start 2024-05-09T06:37:18.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.787+0000 7f07ac74f640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f07a40b4840 0x7f07a40cf8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:18.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.787+0000 7f07ac74f640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f07a4071860 con 0x7f07a4072900 2024-05-09T06:37:18.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.787+0000 7f07ac74f640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f07a4071b40 con 0x7f07a4144a10 2024-05-09T06:37:18.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.787+0000 7f07ac74f640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f07a4072470 con 0x7f07a40b4840 2024-05-09T06:37:18.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.787+0000 7f07a9cea640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f07a4072900 0x7f07a40b4130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:46012/0 (socket says 172.21.15.3:46012) 2024-05-09T06:37:18.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.787+0000 7f07a9cea640 1 -- 172.21.15.3:0/2063467225 learned_addr learned my addr 172.21.15.3:0/2063467225 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-05-09T06:37:18.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.787+0000 7f07aacec640 1 --2- 172.21.15.3:0/2063467225 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f07a40b4840 0x7f07a40cf8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:18.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.787+0000 7f07a94e9640 1 -- 172.21.15.3:0/2063467225 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3037028917 0 0) 0x7f07a4071860 con 0x7f07a4072900 2024-05-09T06:37:18.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.787+0000 7f07a94e9640 1 -- 172.21.15.3:0/2063467225 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0790003520 con 0x7f07a4072900 2024-05-09T06:37:18.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.787+0000 7f07a94e9640 1 -- 172.21.15.3:0/2063467225 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 742246894 0 0) 0x7f0790003520 con 0x7f07a4072900 2024-05-09T06:37:18.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.787+0000 7f07a94e9640 1 -- 172.21.15.3:0/2063467225 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f07a40b4840 msgr2=0x7f07a40cf8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:37:18.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.787+0000 7f07a94e9640 1 --2- 172.21.15.3:0/2063467225 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f07a40b4840 0x7f07a40cf8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:18.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.787+0000 7f07a94e9640 1 -- 172.21.15.3:0/2063467225 >> v1:172.21.15.190:6789/0 conn(0x7f07a4144a10 legacy=0x7f07a4071080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:18.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.787+0000 7f07a94e9640 1 -- 172.21.15.3:0/2063467225 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07a4071e20 con 0x7f07a4072900 2024-05-09T06:37:18.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.787+0000 7f07a94e9640 1 -- 172.21.15.3:0/2063467225 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 563312031 0 0) 0x7f0794001670 con 0x7f07a4072900 2024-05-09T06:37:18.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.787+0000 7f07a94e9640 1 -- 172.21.15.3:0/2063467225 >> v1:172.21.15.3:6789/0 conn(0x7f07a4072900 legacy=0x7f07a40b4130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:18.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.788+0000 7f07a94e9640 1 --2- 172.21.15.3:0/2063467225 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f07900039e0 0x7f0790003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:18.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.788+0000 7f07a94e9640 1 --2- 172.21.15.3:0/2063467225 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f07900043b0 0x7f0790004880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:18.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.788+0000 7f07a94e9640 1 --2- 172.21.15.3:0/2063467225 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f07a40b4840 0x7f0790005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:18.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.788+0000 7f07a94e9640 1 -- 172.21.15.3:0/2063467225 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f07a4072470 con 0x7f07900043b0 2024-05-09T06:37:18.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.788+0000 7f07a94e9640 1 -- 172.21.15.3:0/2063467225 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f079000bad0 con 0x7f07900039e0 2024-05-09T06:37:18.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.788+0000 7f07a94e9640 1 -- 172.21.15.3:0/2063467225 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f079000bcd0 con 0x7f07a40b4840 2024-05-09T06:37:18.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.788+0000 7f07a9cea640 1 --2- 172.21.15.3:0/2063467225 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f07900043b0 0x7f0790004880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:18.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.788+0000 7f07aa4eb640 1 --2- 172.21.15.3:0/2063467225 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f07900039e0 0x7f0790003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:18.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.788+0000 7f07aacec640 1 --2- 172.21.15.3:0/2063467225 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f07a40b4840 0x7f0790005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:18.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.788+0000 7f07a9cea640 1 -- 172.21.15.3:0/2063467225 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f07a40b4840 msgr2=0x7f0790005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:37:18.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.788+0000 7f07a9cea640 1 --2- 172.21.15.3:0/2063467225 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f07a40b4840 0x7f0790005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:18.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.788+0000 7f07a9cea640 1 -- 172.21.15.3:0/2063467225 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f07900039e0 msgr2=0x7f0790003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:37:18.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.788+0000 7f07a9cea640 1 --2- 172.21.15.3:0/2063467225 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f07900039e0 0x7f0790003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:18.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.788+0000 7f07a9cea640 1 -- 172.21.15.3:0/2063467225 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0790003520 con 0x7f07900043b0 2024-05-09T06:37:18.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.789+0000 7f07a9cea640 1 --2- 172.21.15.3:0/2063467225 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f07900043b0 0x7f0790004880 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f079400c210 tx=0x7f0790005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a1717949bbfd58d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:18.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.789+0000 7f07a94e9640 1 -- 172.21.15.3:0/2063467225 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0794007d10 con 0x7f07900043b0 2024-05-09T06:37:18.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.789+0000 7f07a94e9640 1 -- 172.21.15.3:0/2063467225 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0794005ad0 con 0x7f07900043b0 2024-05-09T06:37:18.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.789+0000 7f07a94e9640 1 -- 172.21.15.3:0/2063467225 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0794007070 con 0x7f07900043b0 2024-05-09T06:37:18.774 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.789+0000 7f07ac74f640 1 -- 172.21.15.3:0/2063467225 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f07900043b0 msgr2=0x7f0790004880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:18.774 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.789+0000 7f07ac74f640 1 --2- 172.21.15.3:0/2063467225 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f07900043b0 0x7f0790004880 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f079400c210 tx=0x7f0790005830 comp rx=0 tx=0).stop 2024-05-09T06:37:18.774 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.789+0000 7f07aacec640 1 -- 172.21.15.3:0/2063467225 reap_dead start 2024-05-09T06:37:18.774 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.790+0000 7f07ac74f640 1 -- 172.21.15.3:0/2063467225 shutdown_connections 2024-05-09T06:37:18.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.790+0000 7f07ac74f640 1 -- 172.21.15.3:0/2063467225 >> 172.21.15.3:0/2063467225 conn(0x7f07a40b1c20 msgr2=0x7f07a40b2030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:18.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.790+0000 7f07ac74f640 1 -- 172.21.15.3:0/2063467225 shutdown_connections 2024-05-09T06:37:18.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.790+0000 7f07ac74f640 1 -- 172.21.15.3:0/2063467225 wait complete. 2024-05-09T06:37:18.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.791+0000 7f07ac74f640 1 Processor -- start 2024-05-09T06:37:18.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.791+0000 7f07ac74f640 1 -- start start 2024-05-09T06:37:18.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.792+0000 7f07ac74f640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f07900043b0 0x7f07a40c4e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:18.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.792+0000 7f07ac74f640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f07a40aede0 0x7f07a4133eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:18.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.792+0000 7f07ac74f640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f07a41f3de0 0x7f07a41f4270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:18.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.792+0000 7f07ac74f640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f07a40cfa20 con 0x7f07a40aede0 2024-05-09T06:37:18.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.792+0000 7f07ac74f640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f07a40cf8a0 con 0x7f07900043b0 2024-05-09T06:37:18.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.792+0000 7f07ac74f640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f07a40cfba0 con 0x7f07a41f3de0 2024-05-09T06:37:18.776 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.792+0000 7f07a9cea640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f07a40aede0 0x7f07a4133eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:18.776 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.792+0000 7f07a9cea640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f07a40aede0 0x7f07a4133eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51248/0 (socket says 172.21.15.3:51248) 2024-05-09T06:37:18.776 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.792+0000 7f07a9cea640 1 -- 172.21.15.3:0/1780184287 learned_addr learned my addr 172.21.15.3:0/1780184287 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:37:18.776 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.792+0000 7f07aacec640 1 --2- 172.21.15.3:0/1780184287 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f07a41f3de0 0x7f07a41f4270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:18.776 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.792+0000 7f07aa4eb640 1 --2- 172.21.15.3:0/1780184287 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f07900043b0 0x7f07a40c4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:18.776 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.792+0000 7f07a9cea640 1 -- 172.21.15.3:0/1780184287 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f07a41f3de0 msgr2=0x7f07a41f4270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:18.776 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.792+0000 7f07a9cea640 1 --2- 172.21.15.3:0/1780184287 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f07a41f3de0 0x7f07a41f4270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:18.776 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.792+0000 7f07a9cea640 1 -- 172.21.15.3:0/1780184287 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f07900043b0 msgr2=0x7f07a40c4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:18.776 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.792+0000 7f07a9cea640 1 --2- 172.21.15.3:0/1780184287 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f07900043b0 0x7f07a40c4e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:18.776 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.792+0000 7f07a9cea640 1 -- 172.21.15.3:0/1780184287 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07a40af2a0 con 0x7f07a40aede0 2024-05-09T06:37:18.776 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.793+0000 7f07a9cea640 1 --2- 172.21.15.3:0/1780184287 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f07a40aede0 0x7f07a4133eb0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f079400c330 tx=0x7f07a40723e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:18.776 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.793+0000 7f079affd640 1 -- 172.21.15.3:0/1780184287 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0794001630 con 0x7f07a40aede0 2024-05-09T06:37:18.776 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.793+0000 7f079affd640 1 -- 172.21.15.3:0/1780184287 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0794042070 con 0x7f07a40aede0 2024-05-09T06:37:18.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.793+0000 7f079affd640 1 -- 172.21.15.3:0/1780184287 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07940049e0 con 0x7f07a40aede0 2024-05-09T06:37:18.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.793+0000 7f07ac74f640 1 -- 172.21.15.3:0/1780184287 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07a41f23d0 con 0x7f07a40aede0 2024-05-09T06:37:18.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.793+0000 7f07ac74f640 1 -- 172.21.15.3:0/1780184287 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07a41f2990 con 0x7f07a40aede0 2024-05-09T06:37:18.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.794+0000 7f07ac74f640 1 -- 172.21.15.3:0/1780184287 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07a4069550 con 0x7f07a40aede0 2024-05-09T06:37:18.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.794+0000 7f079affd640 1 -- 172.21.15.3:0/1780184287 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 7) ==== 90651+0+0 (secure 0 0 0) 0x7f0794004b80 con 0x7f07a40aede0 2024-05-09T06:37:18.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.795+0000 7f079affd640 1 --2- 172.21.15.3:0/1780184287 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f078806c010 0x7f078806e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:18.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.795+0000 7f079affd640 1 -- 172.21.15.3:0/1780184287 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5795+0+0 (secure 0 0 0) 0x7f07940ab050 con 0x7f07a40aede0 2024-05-09T06:37:18.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.795+0000 7f07aa4eb640 1 --2- 172.21.15.3:0/1780184287 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f078806c010 0x7f078806e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:18.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.796+0000 7f07aa4eb640 1 --2- 172.21.15.3:0/1780184287 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f078806c010 0x7f078806e4d0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f07a0001890 tx=0x7f07a0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:18.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:18.799+0000 7f079affd640 1 -- 172.21.15.3:0/1780184287 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f07940792a0 con 0x7f07a40aede0 2024-05-09T06:37:19.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:19.027+0000 7f07ac74f640 1 -- 172.21.15.3:0/1780184287 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f07a415c710 con 0x7f07a40aede0 2024-05-09T06:37:19.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:19.664+0000 7f079affd640 1 -- 172.21.15.3:0/1780184287 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v20) ==== 104+0+0 (secure 0 0 0) 0x7f079407d5a0 con 0x7f07a40aede0 2024-05-09T06:37:19.648 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:19.667+0000 7f07ac74f640 1 -- 172.21.15.3:0/1780184287 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f078806c010 msgr2=0x7f078806e4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:19.648 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:19.667+0000 7f07ac74f640 1 --2- 172.21.15.3:0/1780184287 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f078806c010 0x7f078806e4d0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f07a0001890 tx=0x7f07a0009040 comp rx=0 tx=0).stop 2024-05-09T06:37:19.648 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:19.667+0000 7f07ac74f640 1 -- 172.21.15.3:0/1780184287 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f07a40aede0 msgr2=0x7f07a4133eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:19.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:19.667+0000 7f07ac74f640 1 --2- 172.21.15.3:0/1780184287 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f07a40aede0 0x7f07a4133eb0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f079400c330 tx=0x7f07a40723e0 comp rx=0 tx=0).stop 2024-05-09T06:37:19.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:19.667+0000 7f07ac74f640 1 -- 172.21.15.3:0/1780184287 shutdown_connections 2024-05-09T06:37:19.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:19.667+0000 7f07ac74f640 1 --2- 172.21.15.3:0/1780184287 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f078806c010 0x7f078806e4d0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:19.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:19.667+0000 7f07ac74f640 1 --2- 172.21.15.3:0/1780184287 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f07a41f3de0 0x7f07a41f4270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:19.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:19.667+0000 7f07ac74f640 1 --2- 172.21.15.3:0/1780184287 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f07a40aede0 0x7f07a4133eb0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:19.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:19.667+0000 7f07ac74f640 1 --2- 172.21.15.3:0/1780184287 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f07900043b0 0x7f07a40c4e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:19.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:19.667+0000 7f07ac74f640 1 -- 172.21.15.3:0/1780184287 >> 172.21.15.3:0/1780184287 conn(0x7f07a40b1c20 msgr2=0x7f07a406d3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:19.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:19.668+0000 7f07ac74f640 1 -- 172.21.15.3:0/1780184287 shutdown_connections 2024-05-09T06:37:19.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:19.668+0000 7f07ac74f640 1 -- 172.21.15.3:0/1780184287 wait complete. 2024-05-09T06:37:19.663 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-05-09T06:37:19.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:19.839+0000 7f0f22a37640 1 Processor -- start 2024-05-09T06:37:19.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:19.839+0000 7f0f22a37640 1 -- start start 2024-05-09T06:37:19.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:19.840+0000 7f0f22a37640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f0f1c13e030 0x7f0f1c1407a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:19.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:19.840+0000 7f0f22a37640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0f1c0ceef0 0x7f0f1c071720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:19.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:19.840+0000 7f0f22a37640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0f1c13e410 con 0x7f0f1c140ce0 2024-05-09T06:37:19.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:19.840+0000 7f0f22a37640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f0f1c0cf3c0 con 0x7f0f1c13e030 2024-05-09T06:37:19.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:19.840+0000 7f0f22a37640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f0f1c0cf6a0 con 0x7f0f1c0ceef0 2024-05-09T06:37:19.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:19.840+0000 7f0f1bfff640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f0f1c13e030 0x7f0f1c1407a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:19.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:19.840+0000 7f0f1b7fe640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f0f1c140ce0 0x7f0f1c0ce7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:46024/0 (socket says 172.21.15.3:46024) 2024-05-09T06:37:19.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:19.840+0000 7f0f1bfff640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f0f1c13e030 0x7f0f1c1407a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51260/0 (socket says 172.21.15.3:51260) 2024-05-09T06:37:19.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:19.840+0000 7f0f1b7fe640 1 -- 172.21.15.3:0/915527041 learned_addr learned my addr 172.21.15.3:0/915527041 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-05-09T06:37:19.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:19.841+0000 7f0f20fd4640 1 --2- 172.21.15.3:0/915527041 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0f1c0ceef0 0x7f0f1c071720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:19.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:19.841+0000 7f0f1bfff640 1 -- 172.21.15.3:0/915527041 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0f1c0ceef0 msgr2=0x7f0f1c071720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:37:19.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:19.841+0000 7f0f1bfff640 1 --2- 172.21.15.3:0/915527041 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0f1c0ceef0 0x7f0f1c071720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:19.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:19.841+0000 7f0f1bfff640 1 -- 172.21.15.3:0/915527041 >> v1:172.21.15.3:6789/0 conn(0x7f0f1c140ce0 legacy=0x7f0f1c0ce7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:19.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:19.841+0000 7f0f1bfff640 1 -- 172.21.15.3:0/915527041 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f1c0cf980 con 0x7f0f1c13e030 2024-05-09T06:37:19.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:19.841+0000 7f0f20fd4640 1 --2- 172.21.15.3:0/915527041 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0f1c0ceef0 0x7f0f1c071720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:37:19.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:19.841+0000 7f0f1bfff640 1 --2- 172.21.15.3:0/915527041 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0f1c13e030 0x7f0f1c1407a0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f0f1c13e5e0 tx=0x7f0f0c02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed0027ec167d1375 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:19.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:19.841+0000 7f0f1affd640 1 -- 172.21.15.3:0/915527041 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f0c037070 con 0x7f0f1c13e030 2024-05-09T06:37:19.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:19.842+0000 7f0f1affd640 1 -- 172.21.15.3:0/915527041 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f0c032440 con 0x7f0f1c13e030 2024-05-09T06:37:19.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:19.842+0000 7f0f1affd640 1 -- 172.21.15.3:0/915527041 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f0c032810 con 0x7f0f1c13e030 2024-05-09T06:37:19.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:19.842+0000 7f0f22a37640 1 -- 172.21.15.3:0/915527041 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0f1c13e030 msgr2=0x7f0f1c1407a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:19.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:19.842+0000 7f0f22a37640 1 --2- 172.21.15.3:0/915527041 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0f1c13e030 0x7f0f1c1407a0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f0f1c13e5e0 tx=0x7f0f0c02a180 comp rx=0 tx=0).stop 2024-05-09T06:37:19.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:19.842+0000 7f0f22a37640 1 -- 172.21.15.3:0/915527041 shutdown_connections 2024-05-09T06:37:19.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:19.842+0000 7f0f22a37640 1 --2- 172.21.15.3:0/915527041 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0f1c0ceef0 0x7f0f1c071720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:19.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:19.842+0000 7f0f22a37640 1 --2- 172.21.15.3:0/915527041 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0f1c13e030 0x7f0f1c1407a0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:19.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:19.842+0000 7f0f22a37640 1 -- 172.21.15.3:0/915527041 >> 172.21.15.3:0/915527041 conn(0x7f0f1c06b640 msgr2=0x7f0f1c06ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:19.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:19.842+0000 7f0f22a37640 1 -- 172.21.15.3:0/915527041 shutdown_connections 2024-05-09T06:37:19.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:19.842+0000 7f0f22a37640 1 -- 172.21.15.3:0/915527041 wait complete. 2024-05-09T06:37:19.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:19.843+0000 7f0f22a37640 1 Processor -- start 2024-05-09T06:37:19.828 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:19.844+0000 7f0f22a37640 1 -- start start 2024-05-09T06:37:19.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:19.844+0000 7f0f22a37640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0f1c0ceef0 0x7f0f1c154e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:19.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:19.844+0000 7f0f22a37640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0f1c13e030 0x7f0f1c143e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:19.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:19.844+0000 7f0f22a37640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0f1c140ce0 0x7f0f1c15eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:19.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:19.844+0000 7f0f22a37640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f0f1c071c60 con 0x7f0f1c0ceef0 2024-05-09T06:37:19.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:19.844+0000 7f0f22a37640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f0f1c141510 con 0x7f0f1c140ce0 2024-05-09T06:37:19.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:19.844+0000 7f0f22a37640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f0f1c071de0 con 0x7f0f1c13e030 2024-05-09T06:37:19.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:19.844+0000 7f0f1bfff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0f1c0ceef0 0x7f0f1c154e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:19.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:19.844+0000 7f0f1bfff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0f1c0ceef0 0x7f0f1c154e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51272/0 (socket says 172.21.15.3:51272) 2024-05-09T06:37:19.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:19.844+0000 7f0f1bfff640 1 -- 172.21.15.3:0/32306295 learned_addr learned my addr 172.21.15.3:0/32306295 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:37:19.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:19.845+0000 7f0f1b7fe640 1 --2- 172.21.15.3:0/32306295 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0f1c13e030 0x7f0f1c143e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:19.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:19.845+0000 7f0f20fd4640 1 --2- 172.21.15.3:0/32306295 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0f1c140ce0 0x7f0f1c15eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:19.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:19.845+0000 7f0f1bfff640 1 -- 172.21.15.3:0/32306295 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0f1c13e030 msgr2=0x7f0f1c143e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:19.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:19.845+0000 7f0f1bfff640 1 --2- 172.21.15.3:0/32306295 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0f1c13e030 0x7f0f1c143e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:19.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:19.845+0000 7f0f1bfff640 1 -- 172.21.15.3:0/32306295 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0f1c140ce0 msgr2=0x7f0f1c15eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:19.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:19.845+0000 7f0f1bfff640 1 --2- 172.21.15.3:0/32306295 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0f1c140ce0 0x7f0f1c15eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:19.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:19.845+0000 7f0f1bfff640 1 -- 172.21.15.3:0/32306295 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f1c1553d0 con 0x7f0f1c0ceef0 2024-05-09T06:37:19.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:19.845+0000 7f0f1b7fe640 1 --2- 172.21.15.3:0/32306295 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0f1c13e030 0x7f0f1c143e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:37:19.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:19.845+0000 7f0f20fd4640 1 --2- 172.21.15.3:0/32306295 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0f1c140ce0 0x7f0f1c15eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:37:19.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:19.845+0000 7f0f1bfff640 1 --2- 172.21.15.3:0/32306295 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0f1c0ceef0 0x7f0f1c154e90 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f0f0c000c00 tx=0x7f0f0c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:19.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:19.845+0000 7f0f18ff9640 1 -- 172.21.15.3:0/32306295 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f0c037040 con 0x7f0f1c0ceef0 2024-05-09T06:37:19.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:19.845+0000 7f0f18ff9640 1 -- 172.21.15.3:0/32306295 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f0c042070 con 0x7f0f1c0ceef0 2024-05-09T06:37:19.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:19.845+0000 7f0f22a37640 1 -- 172.21.15.3:0/32306295 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f1c1447e0 con 0x7f0f1c0ceef0 2024-05-09T06:37:19.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:19.845+0000 7f0f18ff9640 1 -- 172.21.15.3:0/32306295 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f0c030a50 con 0x7f0f1c0ceef0 2024-05-09T06:37:19.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:19.845+0000 7f0f22a37640 1 -- 172.21.15.3:0/32306295 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f1c156310 con 0x7f0f1c0ceef0 2024-05-09T06:37:19.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:19.846+0000 7f0f22a37640 1 -- 172.21.15.3:0/32306295 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f1c069550 con 0x7f0f1c0ceef0 2024-05-09T06:37:19.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:19.847+0000 7f0f18ff9640 1 -- 172.21.15.3:0/32306295 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 7) ==== 90651+0+0 (secure 0 0 0) 0x7f0f0c030bf0 con 0x7f0f1c0ceef0 2024-05-09T06:37:19.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:19.847+0000 7f0f18ff9640 1 --2- 172.21.15.3:0/32306295 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f0f0806bfe0 0x7f0f0806e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:19.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:19.847+0000 7f0f18ff9640 1 -- 172.21.15.3:0/32306295 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5795+0+0 (secure 0 0 0) 0x7f0f0c0abd80 con 0x7f0f1c0ceef0 2024-05-09T06:37:19.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:19.848+0000 7f0f1b7fe640 1 --2- 172.21.15.3:0/32306295 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f0f0806bfe0 0x7f0f0806e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:19.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:19.848+0000 7f0f1b7fe640 1 --2- 172.21.15.3:0/32306295 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f0f0806bfe0 0x7f0f0806e4a0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f0f1c0cd180 tx=0x7f0f10009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:19.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:19.852+0000 7f0f18ff9640 1 -- 172.21.15.3:0/32306295 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f0f0c045360 con 0x7f0f1c0ceef0 2024-05-09T06:37:20.019 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:20.038+0000 7f0f22a37640 1 -- 172.21.15.3:0/32306295 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f0f1c0730d0 con 0x7f0f1c0ceef0 2024-05-09T06:37:20.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:20.669+0000 7f0f18ff9640 1 -- 172.21.15.3:0/32306295 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v35) ==== 181+0+0 (secure 0 0 0) 0x7f0f0c07a320 con 0x7f0f1c0ceef0 2024-05-09T06:37:20.651 INFO:teuthology.orchestra.run.smithi003.stderr:pool 'cephfs_metadata' removed 2024-05-09T06:37:20.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:20.673+0000 7f0f22a37640 1 -- 172.21.15.3:0/32306295 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f0f0806bfe0 msgr2=0x7f0f0806e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:20.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:20.673+0000 7f0f22a37640 1 --2- 172.21.15.3:0/32306295 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f0f0806bfe0 0x7f0f0806e4a0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f0f1c0cd180 tx=0x7f0f10009040 comp rx=0 tx=0).stop 2024-05-09T06:37:20.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:20.673+0000 7f0f22a37640 1 -- 172.21.15.3:0/32306295 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0f1c0ceef0 msgr2=0x7f0f1c154e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:20.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:20.673+0000 7f0f22a37640 1 --2- 172.21.15.3:0/32306295 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0f1c0ceef0 0x7f0f1c154e90 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f0f0c000c00 tx=0x7f0f0c0046c0 comp rx=0 tx=0).stop 2024-05-09T06:37:20.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:20.674+0000 7f0f22a37640 1 -- 172.21.15.3:0/32306295 shutdown_connections 2024-05-09T06:37:20.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:20.674+0000 7f0f22a37640 1 --2- 172.21.15.3:0/32306295 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f0f0806bfe0 0x7f0f0806e4a0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:20.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:20.674+0000 7f0f22a37640 1 --2- 172.21.15.3:0/32306295 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0f1c140ce0 0x7f0f1c15eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:20.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:20.674+0000 7f0f22a37640 1 --2- 172.21.15.3:0/32306295 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0f1c13e030 0x7f0f1c143e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:20.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:20.674+0000 7f0f22a37640 1 --2- 172.21.15.3:0/32306295 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0f1c0ceef0 0x7f0f1c154e90 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:20.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:20.674+0000 7f0f22a37640 1 -- 172.21.15.3:0/32306295 >> 172.21.15.3:0/32306295 conn(0x7f0f1c06b640 msgr2=0x7f0f1c15be50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:20.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:20.675+0000 7f0f22a37640 1 -- 172.21.15.3:0/32306295 shutdown_connections 2024-05-09T06:37:20.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:20.675+0000 7f0f22a37640 1 -- 172.21.15.3:0/32306295 wait complete. 2024-05-09T06:37:20.670 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-05-09T06:37:20.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:20.805+0000 7fdd1786d640 1 Processor -- start 2024-05-09T06:37:20.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:20.805+0000 7fdd1786d640 1 -- start start 2024-05-09T06:37:20.789 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:20.806+0000 7fdd1786d640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fdd1006b250 0x7fdd1006b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:20.790 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:20.806+0000 7fdd1786d640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdd100adb40 con 0x7fdd100b56f0 2024-05-09T06:37:20.790 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:20.806+0000 7fdd1786d640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdd100b4650 con 0x7fdd100ae0b0 2024-05-09T06:37:20.790 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:20.806+0000 7fdd1786d640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7fdd1006cc80 con 0x7fdd1006b250 2024-05-09T06:37:20.790 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:20.806+0000 7fdd14e08640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fdd100b56f0 0x7fdd100ae490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:43056/0 (socket says 172.21.15.3:43056) 2024-05-09T06:37:20.790 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:20.806+0000 7fdd14e08640 1 -- 172.21.15.3:0/1055509528 learned_addr learned my addr 172.21.15.3:0/1055509528 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-05-09T06:37:20.790 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:20.806+0000 7fdd15e0a640 1 --2- 172.21.15.3:0/1055509528 >> v2:172.21.15.190:3300/0 conn(0x7fdd1006b250 0x7fdd1006b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:20.790 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:20.807+0000 7fdd07fff640 1 -- 172.21.15.3:0/1055509528 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1780989785 0 0) 0x7fdd100adb40 con 0x7fdd100b56f0 2024-05-09T06:37:20.790 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:20.807+0000 7fdd07fff640 1 -- 172.21.15.3:0/1055509528 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdcfc003520 con 0x7fdd100b56f0 2024-05-09T06:37:20.790 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:20.807+0000 7fdd07fff640 1 -- 172.21.15.3:0/1055509528 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2212435181 0 0) 0x7fdcfc003520 con 0x7fdd100b56f0 2024-05-09T06:37:20.790 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:20.807+0000 7fdd07fff640 1 -- 172.21.15.3:0/1055509528 >> v2:172.21.15.190:3300/0 conn(0x7fdd1006b250 msgr2=0x7fdd1006b6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:37:20.790 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:20.807+0000 7fdd07fff640 1 --2- 172.21.15.3:0/1055509528 >> v2:172.21.15.190:3300/0 conn(0x7fdd1006b250 0x7fdd1006b6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:20.790 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:20.807+0000 7fdd07fff640 1 -- 172.21.15.3:0/1055509528 >> v1:172.21.15.190:6789/0 conn(0x7fdd100ae0b0 legacy=0x7fdd1015f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:20.790 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:20.807+0000 7fdd07fff640 1 -- 172.21.15.3:0/1055509528 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd100b4930 con 0x7fdd100b56f0 2024-05-09T06:37:20.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:20.807+0000 7fdd15e0a640 1 --2- 172.21.15.3:0/1055509528 >> v2:172.21.15.190:3300/0 conn(0x7fdd1006b250 0x7fdd1006b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T06:37:20.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:20.807+0000 7fdd07fff640 1 -- 172.21.15.3:0/1055509528 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 563312031 0 0) 0x7fdd00001670 con 0x7fdd100b56f0 2024-05-09T06:37:20.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:20.807+0000 7fdd07fff640 1 -- 172.21.15.3:0/1055509528 >> v1:172.21.15.3:6789/0 conn(0x7fdd100b56f0 legacy=0x7fdd100ae490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:20.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:20.807+0000 7fdd07fff640 1 --2- 172.21.15.3:0/1055509528 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdcfc0039b0 0x7fdcfc003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:20.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:20.807+0000 7fdd07fff640 1 --2- 172.21.15.3:0/1055509528 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdcfc004800 0x7fdcfc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:20.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:20.807+0000 7fdd07fff640 1 --2- 172.21.15.3:0/1055509528 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdcfc0051c0 0x7fdcfc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:20.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:20.807+0000 7fdd07fff640 1 -- 172.21.15.3:0/1055509528 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fdd1006cc80 con 0x7fdcfc0039b0 2024-05-09T06:37:20.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:20.807+0000 7fdd07fff640 1 -- 172.21.15.3:0/1055509528 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fdcfc00c160 con 0x7fdcfc004800 2024-05-09T06:37:20.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:20.807+0000 7fdd07fff640 1 -- 172.21.15.3:0/1055509528 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fdcfc00c360 con 0x7fdcfc0051c0 2024-05-09T06:37:20.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:20.807+0000 7fdd15609640 1 --2- 172.21.15.3:0/1055509528 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdcfc0039b0 0x7fdcfc003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:20.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:20.807+0000 7fdd14e08640 1 --2- 172.21.15.3:0/1055509528 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdcfc004800 0x7fdcfc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:20.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:20.807+0000 7fdd15e0a640 1 --2- 172.21.15.3:0/1055509528 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdcfc0051c0 0x7fdcfc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:20.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:20.807+0000 7fdd15609640 1 -- 172.21.15.3:0/1055509528 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdcfc0051c0 msgr2=0x7fdcfc00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:37:20.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:20.807+0000 7fdd15609640 1 --2- 172.21.15.3:0/1055509528 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdcfc0051c0 0x7fdcfc00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:20.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:20.807+0000 7fdd15609640 1 -- 172.21.15.3:0/1055509528 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdcfc004800 msgr2=0x7fdcfc004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:37:20.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:20.807+0000 7fdd15609640 1 --2- 172.21.15.3:0/1055509528 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdcfc004800 0x7fdcfc004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:20.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:20.807+0000 7fdd15609640 1 -- 172.21.15.3:0/1055509528 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcfc003520 con 0x7fdcfc0039b0 2024-05-09T06:37:20.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:20.807+0000 7fdd15e0a640 1 --2- 172.21.15.3:0/1055509528 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdcfc0051c0 0x7fdcfc00b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:37:20.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:20.807+0000 7fdd15e0a640 1 -- 172.21.15.3:0/1055509528 reap_dead start 2024-05-09T06:37:20.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:20.808+0000 7fdd15609640 1 --2- 172.21.15.3:0/1055509528 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdcfc0039b0 0x7fdcfc003dd0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fdd0c001750 tx=0x7fdcfc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7a1251c0a95de3c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:20.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:20.808+0000 7fdd07fff640 1 -- 172.21.15.3:0/1055509528 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd0c001370 con 0x7fdcfc0039b0 2024-05-09T06:37:20.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:20.808+0000 7fdd07fff640 1 -- 172.21.15.3:0/1055509528 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd0c0052d0 con 0x7fdcfc0039b0 2024-05-09T06:37:20.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:20.808+0000 7fdd07fff640 1 -- 172.21.15.3:0/1055509528 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd0c02abc0 con 0x7fdcfc0039b0 2024-05-09T06:37:20.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:20.809+0000 7fdd1786d640 1 -- 172.21.15.3:0/1055509528 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdcfc0039b0 msgr2=0x7fdcfc003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:20.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:20.809+0000 7fdd1786d640 1 --2- 172.21.15.3:0/1055509528 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdcfc0039b0 0x7fdcfc003dd0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fdd0c001750 tx=0x7fdcfc005830 comp rx=0 tx=0).stop 2024-05-09T06:37:20.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:20.809+0000 7fdd1786d640 1 -- 172.21.15.3:0/1055509528 shutdown_connections 2024-05-09T06:37:20.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:20.809+0000 7fdd1786d640 1 --2- 172.21.15.3:0/1055509528 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdcfc0039b0 0x7fdcfc003dd0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:20.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:20.809+0000 7fdd1786d640 1 -- 172.21.15.3:0/1055509528 >> 172.21.15.3:0/1055509528 conn(0x7fdd100d05c0 msgr2=0x7fdd100d09d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:20.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:20.809+0000 7fdd1786d640 1 -- 172.21.15.3:0/1055509528 shutdown_connections 2024-05-09T06:37:20.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:20.809+0000 7fdd1786d640 1 -- 172.21.15.3:0/1055509528 wait complete. 2024-05-09T06:37:20.795 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:20.810+0000 7fdd1786d640 1 Processor -- start 2024-05-09T06:37:20.795 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:20.810+0000 7fdd1786d640 1 -- start start 2024-05-09T06:37:20.795 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:20.811+0000 7fdd1786d640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdcfc0039b0 0x7fdd1015f500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:20.795 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:20.811+0000 7fdd1786d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdd10161550 0x7fdd100b1710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:20.795 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:20.811+0000 7fdd1786d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdd1014bb70 0x7fdd1014bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:20.795 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:20.811+0000 7fdd1786d640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fdd100b4ef0 con 0x7fdd1014bb70 2024-05-09T06:37:20.795 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:20.811+0000 7fdd1786d640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fdd100b5260 con 0x7fdd10161550 2024-05-09T06:37:20.795 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:20.811+0000 7fdd1786d640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fdd100b4200 con 0x7fdcfc0039b0 2024-05-09T06:37:20.795 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:20.811+0000 7fdd15e0a640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdd1014bb70 0x7fdd1014bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:20.796 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:20.811+0000 7fdd15e0a640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdd1014bb70 0x7fdd1014bfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40548/0 (socket says 172.21.15.3:40548) 2024-05-09T06:37:20.796 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:20.811+0000 7fdd15e0a640 1 -- 172.21.15.3:0/1540930077 learned_addr learned my addr 172.21.15.3:0/1540930077 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:37:20.796 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:20.811+0000 7fdd14e08640 1 --2- 172.21.15.3:0/1540930077 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdd10161550 0x7fdd100b1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:20.796 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:20.811+0000 7fdd15609640 1 --2- 172.21.15.3:0/1540930077 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdcfc0039b0 0x7fdd1015f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:20.796 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:20.811+0000 7fdd15e0a640 1 -- 172.21.15.3:0/1540930077 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdcfc0039b0 msgr2=0x7fdd1015f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:20.796 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:20.811+0000 7fdd15e0a640 1 --2- 172.21.15.3:0/1540930077 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdcfc0039b0 0x7fdd1015f500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:20.796 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:20.811+0000 7fdd15e0a640 1 -- 172.21.15.3:0/1540930077 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdd10161550 msgr2=0x7fdd100b1710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:20.796 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:20.811+0000 7fdd15e0a640 1 --2- 172.21.15.3:0/1540930077 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdd10161550 0x7fdd100b1710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:20.796 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:20.811+0000 7fdd15e0a640 1 -- 172.21.15.3:0/1540930077 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd1014c510 con 0x7fdd1014bb70 2024-05-09T06:37:20.796 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:20.812+0000 7fdd15e0a640 1 --2- 172.21.15.3:0/1540930077 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdd1014bb70 0x7fdd1014bfd0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fdd10071720 tx=0x7fdd080115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:20.796 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:20.812+0000 7fdd05ffb640 1 -- 172.21.15.3:0/1540930077 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd08011e70 con 0x7fdd1014bb70 2024-05-09T06:37:20.796 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:20.812+0000 7fdd05ffb640 1 -- 172.21.15.3:0/1540930077 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd0801b070 con 0x7fdd1014bb70 2024-05-09T06:37:20.796 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:20.812+0000 7fdd1786d640 1 -- 172.21.15.3:0/1540930077 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd101619d0 con 0x7fdd1014bb70 2024-05-09T06:37:20.796 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:20.812+0000 7fdd05ffb640 1 -- 172.21.15.3:0/1540930077 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd08003990 con 0x7fdd1014bb70 2024-05-09T06:37:20.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:20.812+0000 7fdd1786d640 1 -- 172.21.15.3:0/1540930077 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd1015fee0 con 0x7fdd1014bb70 2024-05-09T06:37:20.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:20.813+0000 7fdd1786d640 1 -- 172.21.15.3:0/1540930077 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd10069550 con 0x7fdd1014bb70 2024-05-09T06:37:20.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:20.814+0000 7fdd05ffb640 1 -- 172.21.15.3:0/1540930077 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 7) ==== 90651+0+0 (secure 0 0 0) 0x7fdd08003b30 con 0x7fdd1014bb70 2024-05-09T06:37:20.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:20.814+0000 7fdd05ffb640 1 --2- 172.21.15.3:0/1540930077 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fdcf406c010 0x7fdcf406e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:20.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:20.814+0000 7fdd05ffb640 1 -- 172.21.15.3:0/1540930077 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5357+0+0 (secure 0 0 0) 0x7fdd0808b890 con 0x7fdd1014bb70 2024-05-09T06:37:20.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:20.814+0000 7fdd15609640 1 --2- 172.21.15.3:0/1540930077 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fdcf406c010 0x7fdcf406e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:20.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:20.815+0000 7fdd15609640 1 --2- 172.21.15.3:0/1540930077 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fdcf406c010 0x7fdcf406e4d0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fdd0c032000 tx=0x7fdd0c02e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:20.799 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:20.818+0000 7fdd05ffb640 1 -- 172.21.15.3:0/1540930077 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fdd08003e20 con 0x7fdd1014bb70 2024-05-09T06:37:20.985 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:21.004+0000 7fdd1786d640 1 -- 172.21.15.3:0/1540930077 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fdd101f4180 con 0x7fdd1014bb70 2024-05-09T06:37:21.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:21.674+0000 7fdd05ffb640 1 -- 172.21.15.3:0/1540930077 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v36) ==== 169+0+0 (secure 0 0 0) 0x7fdd101f4180 con 0x7fdd1014bb70 2024-05-09T06:37:21.656 INFO:teuthology.orchestra.run.smithi003.stderr:pool 'cephfs_data' removed 2024-05-09T06:37:21.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:21.679+0000 7fdd1786d640 1 -- 172.21.15.3:0/1540930077 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fdcf406c010 msgr2=0x7fdcf406e4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:21.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:21.679+0000 7fdd1786d640 1 --2- 172.21.15.3:0/1540930077 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fdcf406c010 0x7fdcf406e4d0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fdd0c032000 tx=0x7fdd0c02e000 comp rx=0 tx=0).stop 2024-05-09T06:37:21.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:21.679+0000 7fdd1786d640 1 -- 172.21.15.3:0/1540930077 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdd1014bb70 msgr2=0x7fdd1014bfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:21.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:21.679+0000 7fdd1786d640 1 --2- 172.21.15.3:0/1540930077 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdd1014bb70 0x7fdd1014bfd0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fdd10071720 tx=0x7fdd080115a0 comp rx=0 tx=0).stop 2024-05-09T06:37:21.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:21.679+0000 7fdd1786d640 1 -- 172.21.15.3:0/1540930077 shutdown_connections 2024-05-09T06:37:21.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:21.679+0000 7fdd1786d640 1 --2- 172.21.15.3:0/1540930077 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fdcf406c010 0x7fdcf406e4d0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:21.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:21.679+0000 7fdd1786d640 1 --2- 172.21.15.3:0/1540930077 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdd1014bb70 0x7fdd1014bfd0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:21.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:21.679+0000 7fdd1786d640 1 --2- 172.21.15.3:0/1540930077 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdd10161550 0x7fdd100b1710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:21.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:21.679+0000 7fdd1786d640 1 --2- 172.21.15.3:0/1540930077 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdcfc0039b0 0x7fdd1015f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:21.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:21.679+0000 7fdd1786d640 1 -- 172.21.15.3:0/1540930077 >> 172.21.15.3:0/1540930077 conn(0x7fdd100d05c0 msgr2=0x7fdd100b00e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:21.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:21.680+0000 7fdd1786d640 1 -- 172.21.15.3:0/1540930077 shutdown_connections 2024-05-09T06:37:21.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:21.680+0000 7fdd1786d640 1 -- 172.21.15.3:0/1540930077 wait complete. 2024-05-09T06:37:21.675 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-05-09T06:37:21.789 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:21.808+0000 7f2dabe7d640 1 Processor -- start 2024-05-09T06:37:21.789 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:21.808+0000 7f2dabe7d640 1 -- start start 2024-05-09T06:37:21.790 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:21.809+0000 7f2dabe7d640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f2da415e490 0x7f2da4162590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:21.790 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:21.809+0000 7f2dabe7d640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f2da406b250 0x7f2da406b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:21.790 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:21.809+0000 7f2dabe7d640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2da406cad0 con 0x7f2da415e0b0 2024-05-09T06:37:21.790 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:21.809+0000 7f2dabe7d640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f2da40b3eb0 con 0x7f2da406b250 2024-05-09T06:37:21.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:21.809+0000 7f2dabe7d640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f2da40b4190 con 0x7f2da415e490 2024-05-09T06:37:21.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:21.809+0000 7f2daa41a640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f2da406b250 0x7f2da406b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:21.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:21.809+0000 7f2daa41a640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f2da406b250 0x7f2da406b6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40562/0 (socket says 172.21.15.3:40562) 2024-05-09T06:37:21.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:21.809+0000 7f2daa41a640 1 -- 172.21.15.3:0/475195037 learned_addr learned my addr 172.21.15.3:0/475195037 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:37:21.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:21.809+0000 7f2da9418640 1 --2- 172.21.15.3:0/475195037 >> v2:172.21.15.190:3300/0 conn(0x7f2da415e490 0x7f2da4162590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:21.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:21.809+0000 7f2daa41a640 1 -- 172.21.15.3:0/475195037 >> v2:172.21.15.190:3300/0 conn(0x7f2da415e490 msgr2=0x7f2da4162590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:37:21.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:21.809+0000 7f2daa41a640 1 --2- 172.21.15.3:0/475195037 >> v2:172.21.15.190:3300/0 conn(0x7f2da415e490 0x7f2da4162590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:21.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:21.809+0000 7f2daa41a640 1 -- 172.21.15.3:0/475195037 >> v1:172.21.15.190:6789/0 conn(0x7f2da415e0b0 legacy=0x7f2da4161e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:21.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:21.809+0000 7f2daa41a640 1 -- 172.21.15.3:0/475195037 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2da40b4470 con 0x7f2da406b250 2024-05-09T06:37:21.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:21.810+0000 7f2daa41a640 1 --2- 172.21.15.3:0/475195037 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2da406b250 0x7f2da406b6c0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f2da406cdb0 tx=0x7f2d9c02a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea705fa9c573261b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:21.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:21.810+0000 7f2da8c17640 1 -- 172.21.15.3:0/475195037 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d9c036070 con 0x7f2da406b250 2024-05-09T06:37:21.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:21.810+0000 7f2da8c17640 1 -- 172.21.15.3:0/475195037 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d9c031600 con 0x7f2da406b250 2024-05-09T06:37:21.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:21.810+0000 7f2da8c17640 1 -- 172.21.15.3:0/475195037 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d9c0319d0 con 0x7f2da406b250 2024-05-09T06:37:21.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:21.811+0000 7f2dabe7d640 1 -- 172.21.15.3:0/475195037 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2da406b250 msgr2=0x7f2da406b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:21.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:21.811+0000 7f2dabe7d640 1 --2- 172.21.15.3:0/475195037 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2da406b250 0x7f2da406b6c0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f2da406cdb0 tx=0x7f2d9c02a730 comp rx=0 tx=0).stop 2024-05-09T06:37:21.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:21.811+0000 7f2dabe7d640 1 -- 172.21.15.3:0/475195037 shutdown_connections 2024-05-09T06:37:21.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:21.811+0000 7f2dabe7d640 1 --2- 172.21.15.3:0/475195037 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2da406b250 0x7f2da406b6c0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:21.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:21.811+0000 7f2dabe7d640 1 --2- 172.21.15.3:0/475195037 >> v2:172.21.15.190:3300/0 conn(0x7f2da415e490 0x7f2da4162590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:21.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:21.811+0000 7f2dabe7d640 1 -- 172.21.15.3:0/475195037 >> 172.21.15.3:0/475195037 conn(0x7f2da40d07b0 msgr2=0x7f2da40d0bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:21.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:21.811+0000 7f2dabe7d640 1 -- 172.21.15.3:0/475195037 shutdown_connections 2024-05-09T06:37:21.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:21.811+0000 7f2dabe7d640 1 -- 172.21.15.3:0/475195037 wait complete. 2024-05-09T06:37:21.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:21.812+0000 7f2dabe7d640 1 Processor -- start 2024-05-09T06:37:21.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:21.812+0000 7f2dabe7d640 1 -- start start 2024-05-09T06:37:21.798 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:21.813+0000 7f2dabe7d640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2da406b250 0x7f2da4156b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:21.799 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:21.813+0000 7f2dabe7d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2da415e0b0 0x7f2da41612e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:21.799 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:21.813+0000 7f2dabe7d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2da415e490 0x7f2da4153800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:21.799 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:21.813+0000 7f2dabe7d640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f2da406da70 con 0x7f2da415e0b0 2024-05-09T06:37:21.799 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:21.813+0000 7f2dabe7d640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f2da406d8f0 con 0x7f2da415e490 2024-05-09T06:37:21.799 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:21.813+0000 7f2dabe7d640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f2da4162ad0 con 0x7f2da406b250 2024-05-09T06:37:21.799 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:21.813+0000 7f2da9418640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2da415e0b0 0x7f2da41612e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:21.799 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:21.813+0000 7f2da9418640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2da415e0b0 0x7f2da41612e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40572/0 (socket says 172.21.15.3:40572) 2024-05-09T06:37:21.799 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:21.813+0000 7f2da9418640 1 -- 172.21.15.3:0/1270941575 learned_addr learned my addr 172.21.15.3:0/1270941575 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:37:21.799 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:21.813+0000 7f2daa41a640 1 --2- 172.21.15.3:0/1270941575 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2da415e490 0x7f2da4153800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:21.800 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:21.813+0000 7f2da9c19640 1 --2- 172.21.15.3:0/1270941575 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2da406b250 0x7f2da4156b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:21.800 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:21.813+0000 7f2da9418640 1 -- 172.21.15.3:0/1270941575 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2da406b250 msgr2=0x7f2da4156b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:21.800 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:21.813+0000 7f2da9418640 1 --2- 172.21.15.3:0/1270941575 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2da406b250 0x7f2da4156b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:21.800 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:21.813+0000 7f2da9418640 1 -- 172.21.15.3:0/1270941575 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2da415e490 msgr2=0x7f2da4153800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:21.800 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:21.813+0000 7f2da9418640 1 --2- 172.21.15.3:0/1270941575 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2da415e490 0x7f2da4153800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:21.800 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:21.814+0000 7f2da9418640 1 -- 172.21.15.3:0/1270941575 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2da4155d90 con 0x7f2da415e0b0 2024-05-09T06:37:21.800 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:21.814+0000 7f2da9c19640 1 --2- 172.21.15.3:0/1270941575 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2da406b250 0x7f2da4156b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:37:21.800 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:21.814+0000 7f2da9418640 1 --2- 172.21.15.3:0/1270941575 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2da415e0b0 0x7f2da41612e0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f2d9400e870 tx=0x7f2d9400ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:21.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:21.814+0000 7f2d9a7fc640 1 -- 172.21.15.3:0/1270941575 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d94017070 con 0x7f2da415e0b0 2024-05-09T06:37:21.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:21.814+0000 7f2d9a7fc640 1 -- 172.21.15.3:0/1270941575 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d9400c940 con 0x7f2da415e0b0 2024-05-09T06:37:21.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:21.814+0000 7f2dabe7d640 1 -- 172.21.15.3:0/1270941575 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2da4156070 con 0x7f2da415e0b0 2024-05-09T06:37:21.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:21.814+0000 7f2d9a7fc640 1 -- 172.21.15.3:0/1270941575 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d9400cd30 con 0x7f2da415e0b0 2024-05-09T06:37:21.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:21.814+0000 7f2dabe7d640 1 -- 172.21.15.3:0/1270941575 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2da415e8a0 con 0x7f2da415e0b0 2024-05-09T06:37:21.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:21.815+0000 7f2dabe7d640 1 -- 172.21.15.3:0/1270941575 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2da4069550 con 0x7f2da415e0b0 2024-05-09T06:37:21.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:21.815+0000 7f2d9a7fc640 1 -- 172.21.15.3:0/1270941575 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 7) ==== 90651+0+0 (secure 0 0 0) 0x7f2d9401a070 con 0x7f2da415e0b0 2024-05-09T06:37:21.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:21.816+0000 7f2d9a7fc640 1 --2- 172.21.15.3:0/1270941575 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f2d8806bfe0 0x7f2d8806e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:21.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:21.816+0000 7f2d9a7fc640 1 -- 172.21.15.3:0/1270941575 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 4943+0+0 (secure 0 0 0) 0x7f2d9408b050 con 0x7f2da415e0b0 2024-05-09T06:37:21.802 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:21.816+0000 7f2da9c19640 1 --2- 172.21.15.3:0/1270941575 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f2d8806bfe0 0x7f2d8806e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:21.802 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:21.817+0000 7f2da9c19640 1 --2- 172.21.15.3:0/1270941575 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f2d8806bfe0 0x7f2d8806e4a0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f2da0001700 tx=0x7f2da0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:21.802 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:21.820+0000 7f2d9a7fc640 1 -- 172.21.15.3:0/1270941575 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f2d94059b10 con 0x7f2da415e0b0 2024-05-09T06:37:21.989 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:22.008+0000 7f2dabe7d640 1 -- 172.21.15.3:0/1270941575 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f2da40b3c90 con 0x7f2da415e0b0 2024-05-09T06:37:21.992 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:22.011+0000 7f2d9a7fc640 1 -- 172.21.15.3:0/1270941575 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v11) ==== 104+0+0 (secure 0 0 0) 0x7f2d94059410 con 0x7f2da415e0b0 2024-05-09T06:37:21.995 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:22.014+0000 7f2dabe7d640 1 -- 172.21.15.3:0/1270941575 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f2d8806bfe0 msgr2=0x7f2d8806e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:21.995 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:22.014+0000 7f2dabe7d640 1 --2- 172.21.15.3:0/1270941575 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f2d8806bfe0 0x7f2d8806e4a0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f2da0001700 tx=0x7f2da0009040 comp rx=0 tx=0).stop 2024-05-09T06:37:21.995 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:22.014+0000 7f2dabe7d640 1 -- 172.21.15.3:0/1270941575 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2da415e0b0 msgr2=0x7f2da41612e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:21.995 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:22.014+0000 7f2dabe7d640 1 --2- 172.21.15.3:0/1270941575 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2da415e0b0 0x7f2da41612e0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f2d9400e870 tx=0x7f2d9400ed30 comp rx=0 tx=0).stop 2024-05-09T06:37:21.995 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:22.014+0000 7f2dabe7d640 1 -- 172.21.15.3:0/1270941575 shutdown_connections 2024-05-09T06:37:21.995 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:22.014+0000 7f2dabe7d640 1 --2- 172.21.15.3:0/1270941575 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f2d8806bfe0 0x7f2d8806e4a0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:21.995 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:22.014+0000 7f2dabe7d640 1 --2- 172.21.15.3:0/1270941575 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2da415e490 0x7f2da4153800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:21.995 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:22.014+0000 7f2dabe7d640 1 --2- 172.21.15.3:0/1270941575 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2da415e0b0 0x7f2da41612e0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:21.995 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:22.014+0000 7f2dabe7d640 1 --2- 172.21.15.3:0/1270941575 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2da406b250 0x7f2da4156b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:21.995 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:22.014+0000 7f2dabe7d640 1 -- 172.21.15.3:0/1270941575 >> 172.21.15.3:0/1270941575 conn(0x7f2da40d07b0 msgr2=0x7f2da406b630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:21.995 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:22.014+0000 7f2dabe7d640 1 -- 172.21.15.3:0/1270941575 shutdown_connections 2024-05-09T06:37:21.995 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:22.014+0000 7f2dabe7d640 1 -- 172.21.15.3:0/1270941575 wait complete. 2024-05-09T06:37:22.006 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestPerModuleFinsherThread.test_volumes_module_finisher_thread' 2024-05-09T06:37:22.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:22.129+0000 7fa13ed61640 1 Processor -- start 2024-05-09T06:37:22.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:22.129+0000 7fa13ed61640 1 -- start start 2024-05-09T06:37:22.114 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:22.130+0000 7fa13ed61640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fa1380cc6f0 0x7fa1380b4250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:22.114 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:22.130+0000 7fa13ed61640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa1380b48e0 con 0x7fa1380ccad0 2024-05-09T06:37:22.114 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:22.130+0000 7fa13ed61640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa1380b4bc0 con 0x7fa1380cde50 2024-05-09T06:37:22.114 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:22.130+0000 7fa13ed61640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7fa1380b4ea0 con 0x7fa1380cc6f0 2024-05-09T06:37:22.114 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:22.130+0000 7fa137fff640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fa1380ccad0 0x7fa1380cd740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:43070/0 (socket says 172.21.15.3:43070) 2024-05-09T06:37:22.114 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:22.130+0000 7fa137fff640 1 -- 172.21.15.3:0/3321206519 learned_addr learned my addr 172.21.15.3:0/3321206519 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-05-09T06:37:22.114 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:22.130+0000 7fa13cafd640 1 --2- 172.21.15.3:0/3321206519 >> v2:172.21.15.190:3300/0 conn(0x7fa1380cc6f0 0x7fa1380b4250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:22.114 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:22.130+0000 7fa1377fe640 1 -- 172.21.15.3:0/3321206519 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1579220006 0 0) 0x7fa1380b48e0 con 0x7fa1380ccad0 2024-05-09T06:37:22.114 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:22.130+0000 7fa1377fe640 1 -- 172.21.15.3:0/3321206519 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa128003520 con 0x7fa1380ccad0 2024-05-09T06:37:22.114 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:22.130+0000 7fa1377fe640 1 -- 172.21.15.3:0/3321206519 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1638798095 0 0) 0x7fa128003520 con 0x7fa1380ccad0 2024-05-09T06:37:22.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:22.130+0000 7fa1377fe640 1 -- 172.21.15.3:0/3321206519 >> v2:172.21.15.190:3300/0 conn(0x7fa1380cc6f0 msgr2=0x7fa1380b4250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:37:22.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:22.130+0000 7fa1377fe640 1 --2- 172.21.15.3:0/3321206519 >> v2:172.21.15.190:3300/0 conn(0x7fa1380cc6f0 0x7fa1380b4250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:22.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:22.130+0000 7fa1377fe640 1 -- 172.21.15.3:0/3321206519 >> v1:172.21.15.190:6789/0 conn(0x7fa1380cde50 legacy=0x7fa13806da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:22.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:22.131+0000 7fa1377fe640 1 -- 172.21.15.3:0/3321206519 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1380b5180 con 0x7fa1380ccad0 2024-05-09T06:37:22.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:22.131+0000 7fa1377fe640 1 -- 172.21.15.3:0/3321206519 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 563312031 0 0) 0x7fa124001670 con 0x7fa1380ccad0 2024-05-09T06:37:22.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:22.131+0000 7fa1377fe640 1 -- 172.21.15.3:0/3321206519 >> v1:172.21.15.3:6789/0 conn(0x7fa1380ccad0 legacy=0x7fa1380cd740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:22.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:22.131+0000 7fa1377fe640 1 --2- 172.21.15.3:0/3321206519 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa1280039b0 0x7fa128003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:22.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:22.131+0000 7fa1377fe640 1 --2- 172.21.15.3:0/3321206519 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa128004800 0x7fa128004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:22.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:22.131+0000 7fa1377fe640 1 --2- 172.21.15.3:0/3321206519 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa1280051c0 0x7fa12800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:22.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:22.131+0000 7fa1377fe640 1 -- 172.21.15.3:0/3321206519 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fa1380b4ea0 con 0x7fa128004800 2024-05-09T06:37:22.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:22.131+0000 7fa1377fe640 1 -- 172.21.15.3:0/3321206519 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fa12800c060 con 0x7fa1280051c0 2024-05-09T06:37:22.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:22.131+0000 7fa1377fe640 1 -- 172.21.15.3:0/3321206519 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fa12800c260 con 0x7fa1280039b0 2024-05-09T06:37:22.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:22.131+0000 7fa137fff640 1 --2- 172.21.15.3:0/3321206519 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa128004800 0x7fa128004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:22.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:22.131+0000 7fa13cafd640 1 --2- 172.21.15.3:0/3321206519 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa1280039b0 0x7fa128003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:22.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:22.131+0000 7fa13d2fe640 1 --2- 172.21.15.3:0/3321206519 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa1280051c0 0x7fa12800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:22.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:22.131+0000 7fa137fff640 1 -- 172.21.15.3:0/3321206519 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa1280039b0 msgr2=0x7fa128003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:37:22.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:22.131+0000 7fa137fff640 1 --2- 172.21.15.3:0/3321206519 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa1280039b0 0x7fa128003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:22.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:22.131+0000 7fa137fff640 1 -- 172.21.15.3:0/3321206519 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa1280051c0 msgr2=0x7fa12800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:37:22.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:22.131+0000 7fa137fff640 1 --2- 172.21.15.3:0/3321206519 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa1280051c0 0x7fa12800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:22.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:22.131+0000 7fa137fff640 1 -- 172.21.15.3:0/3321206519 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa128003520 con 0x7fa128004800 2024-05-09T06:37:22.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:22.131+0000 7fa13d2fe640 1 --2- 172.21.15.3:0/3321206519 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa1280051c0 0x7fa12800b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:37:22.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:22.131+0000 7fa13d2fe640 1 -- 172.21.15.3:0/3321206519 reap_dead start 2024-05-09T06:37:22.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:22.132+0000 7fa137fff640 1 --2- 172.21.15.3:0/3321206519 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa128004800 0x7fa128004c80 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fa12400c210 tx=0x7fa128005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e60140452067463 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:22.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:22.132+0000 7fa1377fe640 1 -- 172.21.15.3:0/3321206519 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa124007cf0 con 0x7fa128004800 2024-05-09T06:37:22.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:22.132+0000 7fa1377fe640 1 -- 172.21.15.3:0/3321206519 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa124005af0 con 0x7fa128004800 2024-05-09T06:37:22.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:22.132+0000 7fa1377fe640 1 -- 172.21.15.3:0/3321206519 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa124007070 con 0x7fa128004800 2024-05-09T06:37:22.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:22.132+0000 7fa13ed61640 1 -- 172.21.15.3:0/3321206519 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa128004800 msgr2=0x7fa128004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:22.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:22.132+0000 7fa13ed61640 1 --2- 172.21.15.3:0/3321206519 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa128004800 0x7fa128004c80 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fa12400c210 tx=0x7fa128005830 comp rx=0 tx=0).stop 2024-05-09T06:37:22.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:22.132+0000 7fa13ed61640 1 -- 172.21.15.3:0/3321206519 shutdown_connections 2024-05-09T06:37:22.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:22.132+0000 7fa13ed61640 1 --2- 172.21.15.3:0/3321206519 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa128004800 0x7fa128004c80 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:22.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:22.132+0000 7fa13ed61640 1 -- 172.21.15.3:0/3321206519 >> 172.21.15.3:0/3321206519 conn(0x7fa13806b430 msgr2=0x7fa13806b810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:22.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:22.133+0000 7fa13ed61640 1 -- 172.21.15.3:0/3321206519 shutdown_connections 2024-05-09T06:37:22.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:22.133+0000 7fa13ed61640 1 -- 172.21.15.3:0/3321206519 wait complete. 2024-05-09T06:37:22.120 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:22.134+0000 7fa13ed61640 1 Processor -- start 2024-05-09T06:37:22.120 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:22.134+0000 7fa13ed61640 1 -- start start 2024-05-09T06:37:22.120 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:22.135+0000 7fa13ed61640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa128004800 0x7fa13806e330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:22.120 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:22.135+0000 7fa13ed61640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa13814f540 0x7fa13814b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:22.120 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:22.135+0000 7fa13ed61640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa1381e7ab0 0x7fa1381e7f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:22.120 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:22.135+0000 7fa13ed61640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fa1380b43d0 con 0x7fa128004800 2024-05-09T06:37:22.120 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:22.135+0000 7fa13ed61640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fa1380b4250 con 0x7fa1381e7ab0 2024-05-09T06:37:22.120 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:22.135+0000 7fa13ed61640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fa1380b4550 con 0x7fa13814f540 2024-05-09T06:37:22.120 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:22.135+0000 7fa13cafd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa128004800 0x7fa13806e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:22.120 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:22.135+0000 7fa13cafd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa128004800 0x7fa13806e330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40586/0 (socket says 172.21.15.3:40586) 2024-05-09T06:37:22.120 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:22.135+0000 7fa13cafd640 1 -- 172.21.15.3:0/3812974451 learned_addr learned my addr 172.21.15.3:0/3812974451 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:37:22.120 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:22.135+0000 7fa137fff640 1 --2- 172.21.15.3:0/3812974451 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa13814f540 0x7fa13814b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:22.120 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:22.135+0000 7fa13d2fe640 1 --2- 172.21.15.3:0/3812974451 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa1381e7ab0 0x7fa1381e7f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:22.121 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:22.135+0000 7fa13cafd640 1 -- 172.21.15.3:0/3812974451 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa13814f540 msgr2=0x7fa13814b170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:22.121 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:22.135+0000 7fa13cafd640 1 --2- 172.21.15.3:0/3812974451 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa13814f540 0x7fa13814b170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:22.121 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:22.135+0000 7fa13cafd640 1 -- 172.21.15.3:0/3812974451 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa1381e7ab0 msgr2=0x7fa1381e7f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:22.121 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:22.135+0000 7fa13cafd640 1 --2- 172.21.15.3:0/3812974451 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa1381e7ab0 0x7fa1381e7f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:22.121 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:22.135+0000 7fa13cafd640 1 -- 172.21.15.3:0/3812974451 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1381520e0 con 0x7fa128004800 2024-05-09T06:37:22.121 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:22.135+0000 7fa13cafd640 1 --2- 172.21.15.3:0/3812974451 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa128004800 0x7fa13806e330 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fa12c011aa0 tx=0x7fa12c011f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:22.121 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:22.135+0000 7fa1357fa640 1 -- 172.21.15.3:0/3812974451 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa12c00fb80 con 0x7fa128004800 2024-05-09T06:37:22.121 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:22.135+0000 7fa1357fa640 1 -- 172.21.15.3:0/3812974451 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa12c00fd20 con 0x7fa128004800 2024-05-09T06:37:22.121 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:22.135+0000 7fa13ed61640 1 -- 172.21.15.3:0/3812974451 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1381523c0 con 0x7fa128004800 2024-05-09T06:37:22.121 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:22.135+0000 7fa1357fa640 1 -- 172.21.15.3:0/3812974451 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa12c014660 con 0x7fa128004800 2024-05-09T06:37:22.121 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:22.136+0000 7fa13ed61640 1 -- 172.21.15.3:0/3812974451 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa13814d260 con 0x7fa128004800 2024-05-09T06:37:22.121 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:22.137+0000 7fa13ed61640 1 -- 172.21.15.3:0/3812974451 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa13804f660 con 0x7fa128004800 2024-05-09T06:37:22.121 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:22.137+0000 7fa1357fa640 1 -- 172.21.15.3:0/3812974451 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 7) ==== 90651+0+0 (secure 0 0 0) 0x7fa12c012040 con 0x7fa128004800 2024-05-09T06:37:22.122 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:22.138+0000 7fa1357fa640 1 --2- 172.21.15.3:0/3812974451 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fa11c06c010 0x7fa11c06e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:22.122 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:22.138+0000 7fa1357fa640 1 -- 172.21.15.3:0/3812974451 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 4943+0+0 (secure 0 0 0) 0x7fa12c08b5c0 con 0x7fa128004800 2024-05-09T06:37:22.122 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:22.138+0000 7fa137fff640 1 --2- 172.21.15.3:0/3812974451 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fa11c06c010 0x7fa11c06e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:22.122 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:22.138+0000 7fa137fff640 1 --2- 172.21.15.3:0/3812974451 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fa11c06c010 0x7fa11c06e4d0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fa124032000 tx=0x7fa124009030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:22.122 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:22.142+0000 7fa1357fa640 1 -- 172.21.15.3:0/3812974451 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fa12c019030 con 0x7fa128004800 2024-05-09T06:37:22.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:22.400+0000 7fa13ed61640 1 -- 172.21.15.3:0/3812974451 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestPerModuleFinsherThread.test_volumes_module_finisher_thread"]} v 0) -- 0x7fa1380b3390 con 0x7fa128004800 2024-05-09T06:37:22.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:22.678+0000 7fa1357fa640 1 -- 172.21.15.3:0/3812974451 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestPerModuleFinsherThread.test_volumes_module_finisher_thread"]}]=0 v99) ==== 167+0+0 (secure 0 0 0) 0x7fa12c059ff0 con 0x7fa128004800 2024-05-09T06:37:22.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:22.681+0000 7fa13ed61640 1 -- 172.21.15.3:0/3812974451 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fa11c06c010 msgr2=0x7fa11c06e4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:22.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:22.681+0000 7fa13ed61640 1 --2- 172.21.15.3:0/3812974451 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fa11c06c010 0x7fa11c06e4d0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fa124032000 tx=0x7fa124009030 comp rx=0 tx=0).stop 2024-05-09T06:37:22.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:22.681+0000 7fa13ed61640 1 -- 172.21.15.3:0/3812974451 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa128004800 msgr2=0x7fa13806e330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:22.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:22.681+0000 7fa13ed61640 1 --2- 172.21.15.3:0/3812974451 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa128004800 0x7fa13806e330 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fa12c011aa0 tx=0x7fa12c011f60 comp rx=0 tx=0).stop 2024-05-09T06:37:22.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:22.681+0000 7fa13ed61640 1 -- 172.21.15.3:0/3812974451 shutdown_connections 2024-05-09T06:37:22.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:22.681+0000 7fa13ed61640 1 --2- 172.21.15.3:0/3812974451 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fa11c06c010 0x7fa11c06e4d0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:22.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:22.681+0000 7fa13ed61640 1 --2- 172.21.15.3:0/3812974451 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa1381e7ab0 0x7fa1381e7f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:22.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:22.681+0000 7fa13ed61640 1 --2- 172.21.15.3:0/3812974451 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa13814f540 0x7fa13814b170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:22.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:22.681+0000 7fa13ed61640 1 --2- 172.21.15.3:0/3812974451 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa128004800 0x7fa13806e330 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:22.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:22.681+0000 7fa13ed61640 1 -- 172.21.15.3:0/3812974451 >> 172.21.15.3:0/3812974451 conn(0x7fa13806b430 msgr2=0x7fa13815e720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:22.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:22.681+0000 7fa13ed61640 1 -- 172.21.15.3:0/3812974451 shutdown_connections 2024-05-09T06:37:22.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:22.681+0000 7fa13ed61640 1 -- 172.21.15.3:0/3812974451 wait complete. 2024-05-09T06:37:22.674 INFO:tasks.cephfs_test_runner:test_volumes_module_finisher_thread (tasks.cephfs.test_volumes.TestPerModuleFinsherThread) ... ok 2024-05-09T06:37:22.674 INFO:tasks.cephfs_test_runner: 2024-05-09T06:37:22.674 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-05-09T06:37:22.674 INFO:tasks.cephfs_test_runner:Ran 1 test in 47.740s 2024-05-09T06:37:22.674 INFO:tasks.cephfs_test_runner: 2024-05-09T06:37:22.674 INFO:tasks.cephfs_test_runner:OK 2024-05-09T06:37:22.675 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2024-05-09T06:37:22.684 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-05-09T06:37:22.708 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T06:37:22.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:22.832+0000 7f13bd529640 1 Processor -- start 2024-05-09T06:37:22.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:22.833+0000 7f13bd529640 1 -- start start 2024-05-09T06:37:22.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:22.833+0000 7f13bd529640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f13b815e290 0x7f13b80ae2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:22.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:22.833+0000 7f13bd529640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f13b80ae7f0 0x7f13b806bc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:22.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:22.833+0000 7f13bd529640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f13b806cc30 con 0x7f13b806c1a0 2024-05-09T06:37:22.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:22.833+0000 7f13bd529640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f13b80aec50 con 0x7f13b80ae7f0 2024-05-09T06:37:22.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:22.833+0000 7f13bd529640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f13b80b4630 con 0x7f13b815e290 2024-05-09T06:37:22.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:22.834+0000 7f13b67fc640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f13b80ae7f0 0x7f13b806bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:22.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:22.834+0000 7f13b67fc640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f13b80ae7f0 0x7f13b806bc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40590/0 (socket says 172.21.15.3:40590) 2024-05-09T06:37:22.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:22.834+0000 7f13b67fc640 1 -- 172.21.15.3:0/2789456950 learned_addr learned my addr 172.21.15.3:0/2789456950 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:37:22.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:22.834+0000 7f13b6ffd640 1 --2- 172.21.15.3:0/2789456950 >> v2:172.21.15.190:3300/0 conn(0x7f13b815e290 0x7f13b80ae2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:22.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:22.834+0000 7f13b67fc640 1 -- 172.21.15.3:0/2789456950 >> v2:172.21.15.190:3300/0 conn(0x7f13b815e290 msgr2=0x7f13b80ae2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:37:22.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:22.834+0000 7f13b67fc640 1 --2- 172.21.15.3:0/2789456950 >> v2:172.21.15.190:3300/0 conn(0x7f13b815e290 0x7f13b80ae2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:22.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:22.834+0000 7f13b67fc640 1 -- 172.21.15.3:0/2789456950 >> v1:172.21.15.190:6789/0 conn(0x7f13b806c1a0 legacy=0x7f13b8161dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:22.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:22.834+0000 7f13b67fc640 1 -- 172.21.15.3:0/2789456950 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13b815e670 con 0x7f13b80ae7f0 2024-05-09T06:37:22.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:22.835+0000 7f13b67fc640 1 --2- 172.21.15.3:0/2789456950 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f13b80ae7f0 0x7f13b806bc60 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f13b80b47b0 tx=0x7f13ac02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f617ae2e40fdf382 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:22.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:22.835+0000 7f13b5ffb640 1 -- 172.21.15.3:0/2789456950 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13ac037070 con 0x7f13b80ae7f0 2024-05-09T06:37:22.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:22.835+0000 7f13b5ffb640 1 -- 172.21.15.3:0/2789456950 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13ac032440 con 0x7f13b80ae7f0 2024-05-09T06:37:22.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:22.835+0000 7f13b5ffb640 1 -- 172.21.15.3:0/2789456950 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13ac032810 con 0x7f13b80ae7f0 2024-05-09T06:37:22.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:22.835+0000 7f13bd529640 1 -- 172.21.15.3:0/2789456950 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f13b80ae7f0 msgr2=0x7f13b806bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:22.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:22.835+0000 7f13bd529640 1 --2- 172.21.15.3:0/2789456950 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f13b80ae7f0 0x7f13b806bc60 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f13b80b47b0 tx=0x7f13ac02a1e0 comp rx=0 tx=0).stop 2024-05-09T06:37:22.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:22.835+0000 7f13bd529640 1 -- 172.21.15.3:0/2789456950 shutdown_connections 2024-05-09T06:37:22.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:22.835+0000 7f13bd529640 1 --2- 172.21.15.3:0/2789456950 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f13b80ae7f0 0x7f13b806bc60 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:22.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:22.835+0000 7f13bd529640 1 --2- 172.21.15.3:0/2789456950 >> v2:172.21.15.190:3300/0 conn(0x7f13b815e290 0x7f13b80ae2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:22.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:22.835+0000 7f13bd529640 1 -- 172.21.15.3:0/2789456950 >> 172.21.15.3:0/2789456950 conn(0x7f13b80d05d0 msgr2=0x7f13b80d09e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:22.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:22.835+0000 7f13bd529640 1 -- 172.21.15.3:0/2789456950 shutdown_connections 2024-05-09T06:37:22.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:22.835+0000 7f13bd529640 1 -- 172.21.15.3:0/2789456950 wait complete. 2024-05-09T06:37:22.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:22.837+0000 7f13bd529640 1 Processor -- start 2024-05-09T06:37:22.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:22.837+0000 7f13bd529640 1 -- start start 2024-05-09T06:37:22.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:22.837+0000 7f13bd529640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f13b806c1a0 0x7f13b80b0ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:22.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:22.837+0000 7f13bd529640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f13b80ae7f0 0x7f13b8157110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:22.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:22.837+0000 7f13bd529640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f13b815e290 0x7f13b814f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:22.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:22.837+0000 7f13bd529640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f13b80b3fe0 con 0x7f13b815e290 2024-05-09T06:37:22.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:22.837+0000 7f13bd529640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f13b80b5ad0 con 0x7f13b80ae7f0 2024-05-09T06:37:22.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:22.837+0000 7f13bd529640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f13b80b42c0 con 0x7f13b806c1a0 2024-05-09T06:37:22.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:22.838+0000 7f13b77fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f13b815e290 0x7f13b814f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:22.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:22.838+0000 7f13b77fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f13b815e290 0x7f13b814f730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40594/0 (socket says 172.21.15.3:40594) 2024-05-09T06:37:22.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:22.838+0000 7f13b77fe640 1 -- 172.21.15.3:0/3074130161 learned_addr learned my addr 172.21.15.3:0/3074130161 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:37:22.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:22.838+0000 7f13b67fc640 1 --2- 172.21.15.3:0/3074130161 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f13b80ae7f0 0x7f13b8157110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:22.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:22.838+0000 7f13b6ffd640 1 --2- 172.21.15.3:0/3074130161 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f13b806c1a0 0x7f13b80b0ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:22.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:22.838+0000 7f13b77fe640 1 -- 172.21.15.3:0/3074130161 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f13b806c1a0 msgr2=0x7f13b80b0ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:22.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:22.838+0000 7f13b77fe640 1 --2- 172.21.15.3:0/3074130161 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f13b806c1a0 0x7f13b80b0ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:22.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:22.838+0000 7f13b77fe640 1 -- 172.21.15.3:0/3074130161 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f13b80ae7f0 msgr2=0x7f13b8157110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:22.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:22.838+0000 7f13b77fe640 1 --2- 172.21.15.3:0/3074130161 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f13b80ae7f0 0x7f13b8157110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:22.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:22.838+0000 7f13b77fe640 1 -- 172.21.15.3:0/3074130161 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13b8151000 con 0x7f13b815e290 2024-05-09T06:37:22.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:22.838+0000 7f13b77fe640 1 --2- 172.21.15.3:0/3074130161 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f13b815e290 0x7f13b814f730 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f13b000c730 tx=0x7f13b000cb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:22.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:22.838+0000 7f139f7fe640 1 -- 172.21.15.3:0/3074130161 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13b0016020 con 0x7f13b815e290 2024-05-09T06:37:22.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:22.838+0000 7f139f7fe640 1 -- 172.21.15.3:0/3074130161 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13b0006d50 con 0x7f13b815e290 2024-05-09T06:37:22.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:22.838+0000 7f13bd529640 1 -- 172.21.15.3:0/3074130161 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13b81515c0 con 0x7f13b815e290 2024-05-09T06:37:22.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:22.838+0000 7f139f7fe640 1 -- 172.21.15.3:0/3074130161 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13b0003bb0 con 0x7f13b815e290 2024-05-09T06:37:22.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:22.838+0000 7f13bd529640 1 -- 172.21.15.3:0/3074130161 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13b8151b80 con 0x7f13b815e290 2024-05-09T06:37:22.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:22.839+0000 7f13bd529640 1 -- 172.21.15.3:0/3074130161 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13b804f660 con 0x7f13b815e290 2024-05-09T06:37:22.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:22.840+0000 7f139f7fe640 1 -- 172.21.15.3:0/3074130161 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 7) ==== 90651+0+0 (secure 0 0 0) 0x7f13b00050d0 con 0x7f13b815e290 2024-05-09T06:37:22.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:22.840+0000 7f139f7fe640 1 --2- 172.21.15.3:0/3074130161 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f13a406bfe0 0x7f13a406e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:22.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:22.840+0000 7f139f7fe640 1 -- 172.21.15.3:0/3074130161 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 4943+0+0 (secure 0 0 0) 0x7f13b008a910 con 0x7f13b815e290 2024-05-09T06:37:22.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:22.841+0000 7f13b6ffd640 1 --2- 172.21.15.3:0/3074130161 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f13a406bfe0 0x7f13a406e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:22.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:22.841+0000 7f13b6ffd640 1 --2- 172.21.15.3:0/3074130161 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f13a406bfe0 0x7f13a406e4a0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f13a8006fd0 tx=0x7f13a8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:22.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:22.845+0000 7f139f7fe640 1 -- 172.21.15.3:0/3074130161 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f13b0020020 con 0x7f13b815e290 2024-05-09T06:37:23.016 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:23.035+0000 7f13bd529640 1 -- 172.21.15.3:0/3074130161 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f13b80b3cb0 con 0x7f13b815e290 2024-05-09T06:37:23.016 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:23.036+0000 7f139f7fe640 1 -- 172.21.15.3:0/3074130161 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v36) ==== 74+0+12934 (secure 0 0 0) 0x7f13b0058cf0 con 0x7f13b815e290 2024-05-09T06:37:23.017 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-09T06:37:23.017 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":36,"fsid":"16ed737d-2665-431a-8ac6-ef242be3975a","created":"2024-05-09T06:35:42.541368+0000","modified":"2024-05-09T06:37:21.671473+0000","last_up_change":"2024-05-09T06:35:53.778274+0000","last_in_change":"2024-05-09T06:35:44.780586+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T06:35:54.819999+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fae3a8db-4826-4f3c-b33a-12c09bb5df6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":1897849827},{"type":"v1","addr":"172.21.15.3:6817","nonce":1897849827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":1897849827},{"type":"v1","addr":"172.21.15.3:6819","nonce":1897849827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":1897849827},{"type":"v1","addr":"172.21.15.3:6823","nonce":1897849827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":1897849827},{"type":"v1","addr":"172.21.15.3:6821","nonce":1897849827}]},"public_addr":"172.21.15.3:6817/1897849827","cluster_addr":"172.21.15.3:6819/1897849827","heartbeat_back_addr":"172.21.15.3:6823/1897849827","heartbeat_front_addr":"172.21.15.3:6821/1897849827","state":["exists","up"]},{"osd":1,"uuid":"646e882d-4ce4-4720-8c84-f71df563a35e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":926805498},{"type":"v1","addr":"172.21.15.3:6801","nonce":926805498}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":926805498},{"type":"v1","addr":"172.21.15.3:6803","nonce":926805498}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":926805498},{"type":"v1","addr":"172.21.15.3:6807","nonce":926805498}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":926805498},{"type":"v1","addr":"172.21.15.3:6805","nonce":926805498}]},"public_addr":"172.21.15.3:6801/926805498","cluster_addr":"172.21.15.3:6803/926805498","heartbeat_back_addr":"172.21.15.3:6807/926805498","heartbeat_front_addr":"172.21.15.3:6805/926805498","state":["exists","up"]},{"osd":2,"uuid":"1abbadb5-a570-4359-9643-acff3ede35a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":801295451},{"type":"v1","addr":"172.21.15.3:6809","nonce":801295451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":801295451},{"type":"v1","addr":"172.21.15.3:6811","nonce":801295451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":801295451},{"type":"v1","addr":"172.21.15.3:6815","nonce":801295451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":801295451},{"type":"v1","addr":"172.21.15.3:6813","nonce":801295451}]},"public_addr":"172.21.15.3:6809/801295451","cluster_addr":"172.21.15.3:6811/801295451","heartbeat_back_addr":"172.21.15.3:6815/801295451","heartbeat_front_addr":"172.21.15.3:6813/801295451","state":["exists","up"]},{"osd":3,"uuid":"62ac4dc1-2371-4dd2-acd3-e2dbb11aedf5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":1903389857},{"type":"v1","addr":"172.21.15.3:6825","nonce":1903389857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":1903389857},{"type":"v1","addr":"172.21.15.3:6827","nonce":1903389857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":1903389857},{"type":"v1","addr":"172.21.15.3:6831","nonce":1903389857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":1903389857},{"type":"v1","addr":"172.21.15.3:6829","nonce":1903389857}]},"public_addr":"172.21.15.3:6825/1903389857","cluster_addr":"172.21.15.3:6827/1903389857","heartbeat_back_addr":"172.21.15.3:6831/1903389857","heartbeat_front_addr":"172.21.15.3:6829/1903389857","state":["exists","up"]},{"osd":4,"uuid":"ae808398-8326-4d7c-9319-a2a0eafffb59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":2143624576},{"type":"v1","addr":"172.21.15.190:6801","nonce":2143624576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":2143624576},{"type":"v1","addr":"172.21.15.190:6803","nonce":2143624576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":2143624576},{"type":"v1","addr":"172.21.15.190:6807","nonce":2143624576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":2143624576},{"type":"v1","addr":"172.21.15.190:6805","nonce":2143624576}]},"public_addr":"172.21.15.190:6801/2143624576","cluster_addr":"172.21.15.190:6803/2143624576","heartbeat_back_addr":"172.21.15.190:6807/2143624576","heartbeat_front_addr":"172.21.15.190:6805/2143624576","state":["exists","up"]},{"osd":5,"uuid":"918adf50-c440-48ac-9a8d-4837562d7275","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":1766923126},{"type":"v1","addr":"172.21.15.190:6814","nonce":1766923126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":1766923126},{"type":"v1","addr":"172.21.15.190:6818","nonce":1766923126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":1766923126},{"type":"v1","addr":"172.21.15.190:6823","nonce":1766923126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":1766923126},{"type":"v1","addr":"172.21.15.190:6821","nonce":1766923126}]},"public_addr":"172.21.15.190:6814/1766923126","cluster_addr":"172.21.15.190:6818/1766923126","heartbeat_back_addr":"172.21.15.190:6823/1766923126","heartbeat_front_addr":"172.21.15.190:6821/1766923126","state":["exists","up"]},{"osd":6,"uuid":"ac2882bc-2a7c-463c-8f6c-76ff49828adc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":342414248},{"type":"v1","addr":"172.21.15.190:6825","nonce":342414248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":342414248},{"type":"v1","addr":"172.21.15.190:6827","nonce":342414248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":342414248},{"type":"v1","addr":"172.21.15.190:6831","nonce":342414248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":342414248},{"type":"v1","addr":"172.21.15.190:6829","nonce":342414248}]},"public_addr":"172.21.15.190:6825/342414248","cluster_addr":"172.21.15.190:6827/342414248","heartbeat_back_addr":"172.21.15.190:6831/342414248","heartbeat_front_addr":"172.21.15.190:6829/342414248","state":["exists","up"]},{"osd":7,"uuid":"dd4e767a-4730-4c70-8bd5-3f088140a5a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":537849891},{"type":"v1","addr":"172.21.15.190:6809","nonce":537849891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":537849891},{"type":"v1","addr":"172.21.15.190:6811","nonce":537849891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6817","nonce":537849891},{"type":"v1","addr":"172.21.15.190:6819","nonce":537849891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6813","nonce":537849891},{"type":"v1","addr":"172.21.15.190:6815","nonce":537849891}]},"public_addr":"172.21.15.190:6809/537849891","cluster_addr":"172.21.15.190:6811/537849891","heartbeat_back_addr":"172.21.15.190:6819/537849891","heartbeat_front_addr":"172.21.15.190:6815/537849891","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.889784+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.974002+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.973549+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.955396+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.928471+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.902418+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.958696+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.926474+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.3:6835/2138710335":"2024-05-10T06:37:18.660898+0000","172.21.15.3:6833/2138710335":"2024-05-10T06:37:18.660898+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T06:37:23.019 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:23.038+0000 7f13bd529640 1 -- 172.21.15.3:0/3074130161 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f13a406bfe0 msgr2=0x7f13a406e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:23.019 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:23.038+0000 7f13bd529640 1 --2- 172.21.15.3:0/3074130161 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f13a406bfe0 0x7f13a406e4a0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f13a8006fd0 tx=0x7f13a8008040 comp rx=0 tx=0).stop 2024-05-09T06:37:23.019 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:23.038+0000 7f13bd529640 1 -- 172.21.15.3:0/3074130161 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f13b815e290 msgr2=0x7f13b814f730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:23.019 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:23.038+0000 7f13bd529640 1 --2- 172.21.15.3:0/3074130161 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f13b815e290 0x7f13b814f730 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f13b000c730 tx=0x7f13b000cb30 comp rx=0 tx=0).stop 2024-05-09T06:37:23.020 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:23.038+0000 7f13bd529640 1 -- 172.21.15.3:0/3074130161 shutdown_connections 2024-05-09T06:37:23.020 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:23.038+0000 7f13bd529640 1 --2- 172.21.15.3:0/3074130161 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f13a406bfe0 0x7f13a406e4a0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:23.020 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:23.038+0000 7f13bd529640 1 --2- 172.21.15.3:0/3074130161 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f13b815e290 0x7f13b814f730 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:23.020 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:23.038+0000 7f13bd529640 1 --2- 172.21.15.3:0/3074130161 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f13b80ae7f0 0x7f13b8157110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:23.020 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:23.038+0000 7f13bd529640 1 --2- 172.21.15.3:0/3074130161 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f13b806c1a0 0x7f13b80b0ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:23.020 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:23.038+0000 7f13bd529640 1 -- 172.21.15.3:0/3074130161 >> 172.21.15.3:0/3074130161 conn(0x7f13b80d05d0 msgr2=0x7f13b806eab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:23.020 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:23.038+0000 7f13bd529640 1 -- 172.21.15.3:0/3074130161 shutdown_connections 2024-05-09T06:37:23.020 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:23.038+0000 7f13bd529640 1 -- 172.21.15.3:0/3074130161 wait complete. 2024-05-09T06:37:23.030 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-09T06:35:54.819999+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-09T06:37:23.030 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-09T06:37:23.138 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:23.157+0000 7f9f325c1640 1 Processor -- start 2024-05-09T06:37:23.138 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:23.157+0000 7f9f325c1640 1 -- start start 2024-05-09T06:37:23.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:23.158+0000 7f9f325c1640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f9f2c144910 0x7f9f2c0cf8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:23.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:23.158+0000 7f9f325c1640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f9f2c0b4130 0x7f9f2c0b4560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:23.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:23.158+0000 7f9f325c1640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9f2c144cf0 con 0x7f9f2c072970 2024-05-09T06:37:23.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:23.158+0000 7f9f325c1640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f9f2c0724e0 con 0x7f9f2c144910 2024-05-09T06:37:23.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:23.158+0000 7f9f325c1640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f9f2c0b4aa0 con 0x7f9f2c0b4130 2024-05-09T06:37:23.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:23.158+0000 7f9f2bfff640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f9f2c144910 0x7f9f2c0cf8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:23.140 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:23.158+0000 7f9f2b7fe640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f9f2c072970 0x7f9f2c071080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:43084/0 (socket says 172.21.15.3:43084) 2024-05-09T06:37:23.140 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:23.158+0000 7f9f2b7fe640 1 -- 172.21.15.3:0/169917190 learned_addr learned my addr 172.21.15.3:0/169917190 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-05-09T06:37:23.140 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:23.158+0000 7f9f30b5e640 1 --2- 172.21.15.3:0/169917190 >> v2:172.21.15.190:3300/0 conn(0x7f9f2c0b4130 0x7f9f2c0b4560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:23.140 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:23.158+0000 7f9f2bfff640 1 -- 172.21.15.3:0/169917190 >> v2:172.21.15.190:3300/0 conn(0x7f9f2c0b4130 msgr2=0x7f9f2c0b4560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:37:23.140 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:23.158+0000 7f9f2bfff640 1 --2- 172.21.15.3:0/169917190 >> v2:172.21.15.190:3300/0 conn(0x7f9f2c0b4130 0x7f9f2c0b4560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:23.140 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:23.158+0000 7f9f2bfff640 1 -- 172.21.15.3:0/169917190 >> v1:172.21.15.3:6789/0 conn(0x7f9f2c072970 legacy=0x7f9f2c071080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:23.140 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:23.158+0000 7f9f2bfff640 1 -- 172.21.15.3:0/169917190 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f2c0718d0 con 0x7f9f2c144910 2024-05-09T06:37:23.140 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:23.158+0000 7f9f2affd640 1 -- 172.21.15.3:0/169917190 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3343591664 0 0) 0x7f9f2c144cf0 con 0x7f9f2c072970 2024-05-09T06:37:23.141 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:23.159+0000 7f9f2bfff640 1 --2- 172.21.15.3:0/169917190 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9f2c144910 0x7f9f2c0cf8c0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f9f2c071010 tx=0x7f9f1c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=604c64a66d97b501 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:23.141 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:23.159+0000 7f9f2affd640 1 -- 172.21.15.3:0/169917190 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f1c037070 con 0x7f9f2c144910 2024-05-09T06:37:23.141 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:23.159+0000 7f9f2affd640 1 -- 172.21.15.3:0/169917190 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f1c032440 con 0x7f9f2c144910 2024-05-09T06:37:23.141 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:23.159+0000 7f9f2affd640 1 -- 172.21.15.3:0/169917190 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f1c032810 con 0x7f9f2c144910 2024-05-09T06:37:23.141 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:23.160+0000 7f9f325c1640 1 -- 172.21.15.3:0/169917190 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9f2c144910 msgr2=0x7f9f2c0cf8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:23.141 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:23.160+0000 7f9f325c1640 1 --2- 172.21.15.3:0/169917190 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9f2c144910 0x7f9f2c0cf8c0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f9f2c071010 tx=0x7f9f1c02a1a0 comp rx=0 tx=0).stop 2024-05-09T06:37:23.142 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:23.160+0000 7f9f325c1640 1 -- 172.21.15.3:0/169917190 shutdown_connections 2024-05-09T06:37:23.142 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:23.160+0000 7f9f325c1640 1 --2- 172.21.15.3:0/169917190 >> v2:172.21.15.190:3300/0 conn(0x7f9f2c0b4130 0x7f9f2c0b4560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:23.142 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:23.160+0000 7f9f325c1640 1 --2- 172.21.15.3:0/169917190 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9f2c144910 0x7f9f2c0cf8c0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:23.142 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:23.160+0000 7f9f325c1640 1 -- 172.21.15.3:0/169917190 >> 172.21.15.3:0/169917190 conn(0x7f9f2c0b1c20 msgr2=0x7f9f2c0b2030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:23.142 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:23.160+0000 7f9f325c1640 1 -- 172.21.15.3:0/169917190 shutdown_connections 2024-05-09T06:37:23.142 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:23.160+0000 7f9f325c1640 1 -- 172.21.15.3:0/169917190 wait complete. 2024-05-09T06:37:23.144 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:23.161+0000 7f9f325c1640 1 Processor -- start 2024-05-09T06:37:23.144 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:23.161+0000 7f9f325c1640 1 -- start start 2024-05-09T06:37:23.144 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:23.162+0000 7f9f325c1640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9f2c072970 0x7f9f2c135b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:23.144 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:23.162+0000 7f9f325c1640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9f2c0b4130 0x7f9f2c1536c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:23.145 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:23.162+0000 7f9f325c1640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9f2c144910 0x7f9f2c156280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:23.145 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:23.162+0000 7f9f325c1640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f9f2c072170 con 0x7f9f2c144910 2024-05-09T06:37:23.145 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:23.162+0000 7f9f325c1640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f9f2c071e90 con 0x7f9f2c072970 2024-05-09T06:37:23.145 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:23.162+0000 7f9f325c1640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f9f2c0b3020 con 0x7f9f2c0b4130 2024-05-09T06:37:23.145 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:23.162+0000 7f9f30b5e640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9f2c144910 0x7f9f2c156280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:23.145 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:23.162+0000 7f9f30b5e640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9f2c144910 0x7f9f2c156280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40612/0 (socket says 172.21.15.3:40612) 2024-05-09T06:37:23.145 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:23.162+0000 7f9f30b5e640 1 -- 172.21.15.3:0/3307922514 learned_addr learned my addr 172.21.15.3:0/3307922514 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:37:23.145 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:23.162+0000 7f9f2b7fe640 1 --2- 172.21.15.3:0/3307922514 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9f2c0b4130 0x7f9f2c1536c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:23.145 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:23.162+0000 7f9f2bfff640 1 --2- 172.21.15.3:0/3307922514 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9f2c072970 0x7f9f2c135b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:23.145 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:23.162+0000 7f9f30b5e640 1 -- 172.21.15.3:0/3307922514 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9f2c0b4130 msgr2=0x7f9f2c1536c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:23.145 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:23.162+0000 7f9f30b5e640 1 --2- 172.21.15.3:0/3307922514 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9f2c0b4130 0x7f9f2c1536c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:23.145 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:23.162+0000 7f9f30b5e640 1 -- 172.21.15.3:0/3307922514 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9f2c072970 msgr2=0x7f9f2c135b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:23.145 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:23.162+0000 7f9f30b5e640 1 --2- 172.21.15.3:0/3307922514 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9f2c072970 0x7f9f2c135b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:23.145 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:23.162+0000 7f9f30b5e640 1 -- 172.21.15.3:0/3307922514 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f2c1360b0 con 0x7f9f2c144910 2024-05-09T06:37:23.145 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:23.162+0000 7f9f2b7fe640 1 --2- 172.21.15.3:0/3307922514 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9f2c0b4130 0x7f9f2c1536c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:37:23.146 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:23.163+0000 7f9f30b5e640 1 --2- 172.21.15.3:0/3307922514 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9f2c144910 0x7f9f2c156280 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f9f24007e70 tx=0x7f9f2400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:23.146 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:23.163+0000 7f9f28ff9640 1 -- 172.21.15.3:0/3307922514 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f24014620 con 0x7f9f2c144910 2024-05-09T06:37:23.146 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:23.163+0000 7f9f28ff9640 1 -- 172.21.15.3:0/3307922514 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f24019070 con 0x7f9f2c144910 2024-05-09T06:37:23.146 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:23.163+0000 7f9f28ff9640 1 -- 172.21.15.3:0/3307922514 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f24014a20 con 0x7f9f2c144910 2024-05-09T06:37:23.146 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:23.163+0000 7f9f325c1640 1 -- 172.21.15.3:0/3307922514 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f2c13d090 con 0x7f9f2c144910 2024-05-09T06:37:23.146 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:23.163+0000 7f9f325c1640 1 -- 172.21.15.3:0/3307922514 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f2c141710 con 0x7f9f2c144910 2024-05-09T06:37:23.146 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:23.164+0000 7f9f325c1640 1 -- 172.21.15.3:0/3307922514 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f2c069550 con 0x7f9f2c144910 2024-05-09T06:37:23.147 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:23.164+0000 7f9f28ff9640 1 -- 172.21.15.3:0/3307922514 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 7) ==== 90651+0+0 (secure 0 0 0) 0x7f9f24014bc0 con 0x7f9f2c144910 2024-05-09T06:37:23.147 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:23.165+0000 7f9f28ff9640 1 --2- 172.21.15.3:0/3307922514 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f9f1806bfe0 0x7f9f1806e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:23.147 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:23.165+0000 7f9f28ff9640 1 -- 172.21.15.3:0/3307922514 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 4943+0+0 (secure 0 0 0) 0x7f9f2408b730 con 0x7f9f2c144910 2024-05-09T06:37:23.147 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:23.165+0000 7f9f2bfff640 1 --2- 172.21.15.3:0/3307922514 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f9f1806bfe0 0x7f9f1806e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:23.147 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:23.166+0000 7f9f2bfff640 1 --2- 172.21.15.3:0/3307922514 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f9f1806bfe0 0x7f9f1806e4a0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f9f1c034000 tx=0x7f9f1c02e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:23.149 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:23.169+0000 7f9f28ff9640 1 -- 172.21.15.3:0/3307922514 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f9f2c1360b0 con 0x7f9f2c144910 2024-05-09T06:37:23.333 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:23.352+0000 7f9f325c1640 1 -- 172.21.15.3:0/3307922514 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f9f2c156060 con 0x7f9f2c144910 2024-05-09T06:37:23.334 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:23.353+0000 7f9f28ff9640 1 -- 172.21.15.3:0/3307922514 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v36) ==== 93+0+10 (secure 0 0 0) 0x7f9f2405a1f0 con 0x7f9f2c144910 2024-05-09T06:37:23.334 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 1 2024-05-09T06:37:23.336 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:23.355+0000 7f9f325c1640 1 -- 172.21.15.3:0/3307922514 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f9f1806bfe0 msgr2=0x7f9f1806e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:23.336 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:23.355+0000 7f9f325c1640 1 --2- 172.21.15.3:0/3307922514 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f9f1806bfe0 0x7f9f1806e4a0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f9f1c034000 tx=0x7f9f1c02e000 comp rx=0 tx=0).stop 2024-05-09T06:37:23.336 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:23.355+0000 7f9f325c1640 1 -- 172.21.15.3:0/3307922514 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9f2c144910 msgr2=0x7f9f2c156280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:23.336 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:23.355+0000 7f9f325c1640 1 --2- 172.21.15.3:0/3307922514 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9f2c144910 0x7f9f2c156280 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f9f24007e70 tx=0x7f9f2400e4b0 comp rx=0 tx=0).stop 2024-05-09T06:37:23.336 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:23.355+0000 7f9f325c1640 1 -- 172.21.15.3:0/3307922514 shutdown_connections 2024-05-09T06:37:23.336 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:23.355+0000 7f9f325c1640 1 --2- 172.21.15.3:0/3307922514 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f9f1806bfe0 0x7f9f1806e4a0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:23.336 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:23.355+0000 7f9f325c1640 1 --2- 172.21.15.3:0/3307922514 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9f2c144910 0x7f9f2c156280 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:23.336 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:23.355+0000 7f9f325c1640 1 --2- 172.21.15.3:0/3307922514 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9f2c0b4130 0x7f9f2c1536c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:23.336 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:23.355+0000 7f9f325c1640 1 --2- 172.21.15.3:0/3307922514 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9f2c072970 0x7f9f2c135b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:23.336 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:23.355+0000 7f9f325c1640 1 -- 172.21.15.3:0/3307922514 >> 172.21.15.3:0/3307922514 conn(0x7f9f2c0b1c20 msgr2=0x7f9f2c06dc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:23.336 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:23.355+0000 7f9f325c1640 1 -- 172.21.15.3:0/3307922514 shutdown_connections 2024-05-09T06:37:23.337 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:23.355+0000 7f9f325c1640 1 -- 172.21.15.3:0/3307922514 wait complete. 2024-05-09T06:37:23.347 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-05-09T06:37:23.452 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:23.471+0000 7f83a228c640 1 Processor -- start 2024-05-09T06:37:23.452 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:23.471+0000 7f83a228c640 1 -- start start 2024-05-09T06:37:23.455 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:23.472+0000 7f83a228c640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f839c0cc6f0 0x7f839c0b4250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:23.455 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:23.472+0000 7f83a228c640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f839c0ccad0 0x7f839c15d160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:23.455 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:23.472+0000 7f83a228c640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f839c0b48e0 con 0x7f839c0cd740 2024-05-09T06:37:23.456 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:23.472+0000 7f83a228c640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f839c0b4bc0 con 0x7f839c0ccad0 2024-05-09T06:37:23.456 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:23.472+0000 7f83a228c640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f839c0b4ea0 con 0x7f839c0cc6f0 2024-05-09T06:37:23.456 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:23.472+0000 7f83a0829640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f839c0cd740 0x7f839c15fc60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:43096/0 (socket says 172.21.15.3:43096) 2024-05-09T06:37:23.456 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:23.472+0000 7f83a0829640 1 -- 172.21.15.3:0/1592437932 learned_addr learned my addr 172.21.15.3:0/1592437932 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-05-09T06:37:23.456 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:23.473+0000 7f839affd640 1 -- 172.21.15.3:0/1592437932 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2857138644 0 0) 0x7f839c0b48e0 con 0x7f839c0cd740 2024-05-09T06:37:23.456 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:23.473+0000 7f839affd640 1 -- 172.21.15.3:0/1592437932 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8384003520 con 0x7f839c0cd740 2024-05-09T06:37:23.456 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:23.473+0000 7f839bfff640 1 --2- 172.21.15.3:0/1592437932 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f839c0cc6f0 0x7f839c0b4250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:23.456 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:23.473+0000 7f839b7fe640 1 --2- 172.21.15.3:0/1592437932 >> v2:172.21.15.190:3300/0 conn(0x7f839c0ccad0 0x7f839c15d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:23.456 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:23.473+0000 7f839affd640 1 -- 172.21.15.3:0/1592437932 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2396855484 0 0) 0x7f8384003520 con 0x7f839c0cd740 2024-05-09T06:37:23.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:23.473+0000 7f839affd640 1 -- 172.21.15.3:0/1592437932 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f839c0cc6f0 msgr2=0x7f839c0b4250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:37:23.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:23.473+0000 7f839affd640 1 --2- 172.21.15.3:0/1592437932 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f839c0cc6f0 0x7f839c0b4250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:23.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:23.473+0000 7f839affd640 1 -- 172.21.15.3:0/1592437932 >> v2:172.21.15.190:3300/0 conn(0x7f839c0ccad0 msgr2=0x7f839c15d160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:37:23.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:23.473+0000 7f839affd640 1 --2- 172.21.15.3:0/1592437932 >> v2:172.21.15.190:3300/0 conn(0x7f839c0ccad0 0x7f839c15d160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:23.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:23.473+0000 7f839affd640 1 -- 172.21.15.3:0/1592437932 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f839c0b5180 con 0x7f839c0cd740 2024-05-09T06:37:23.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:23.473+0000 7f839affd640 1 -- 172.21.15.3:0/1592437932 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 563312031 0 0) 0x7f83940019f0 con 0x7f839c0cd740 2024-05-09T06:37:23.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:23.473+0000 7f839affd640 1 -- 172.21.15.3:0/1592437932 >> v1:172.21.15.3:6789/0 conn(0x7f839c0cd740 legacy=0x7f839c15fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:23.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:23.473+0000 7f839affd640 1 --2- 172.21.15.3:0/1592437932 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f83840039b0 0x7f8384003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:23.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:23.473+0000 7f839affd640 1 --2- 172.21.15.3:0/1592437932 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f839c0cc6f0 0x7f8384004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:23.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:23.473+0000 7f839affd640 1 --2- 172.21.15.3:0/1592437932 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8384004d40 0x7f83840051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:23.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:23.473+0000 7f839affd640 1 -- 172.21.15.3:0/1592437932 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f839c0b4bc0 con 0x7f83840039b0 2024-05-09T06:37:23.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:23.473+0000 7f839affd640 1 -- 172.21.15.3:0/1592437932 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f839c0b4ea0 con 0x7f8384004d40 2024-05-09T06:37:23.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:23.473+0000 7f839affd640 1 -- 172.21.15.3:0/1592437932 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f838400ba70 con 0x7f839c0cc6f0 2024-05-09T06:37:23.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:23.473+0000 7f839bfff640 1 --2- 172.21.15.3:0/1592437932 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f83840039b0 0x7f8384003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:23.458 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:23.473+0000 7f83a0829640 1 --2- 172.21.15.3:0/1592437932 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8384004d40 0x7f83840051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:23.458 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:23.473+0000 7f839b7fe640 1 --2- 172.21.15.3:0/1592437932 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f839c0cc6f0 0x7f8384004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:23.458 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:23.473+0000 7f839bfff640 1 -- 172.21.15.3:0/1592437932 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f839c0cc6f0 msgr2=0x7f8384004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:37:23.458 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:23.473+0000 7f839bfff640 1 --2- 172.21.15.3:0/1592437932 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f839c0cc6f0 0x7f8384004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:23.458 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:23.473+0000 7f839bfff640 1 -- 172.21.15.3:0/1592437932 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8384004d40 msgr2=0x7f83840051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:37:23.458 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:23.473+0000 7f839bfff640 1 --2- 172.21.15.3:0/1592437932 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8384004d40 0x7f83840051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:23.458 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:23.473+0000 7f839bfff640 1 -- 172.21.15.3:0/1592437932 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8384003520 con 0x7f83840039b0 2024-05-09T06:37:23.458 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:23.474+0000 7f839bfff640 1 --2- 172.21.15.3:0/1592437932 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f83840039b0 0x7f8384003dd0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f838c00ba60 tx=0x7f8384005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5598284f9ad34c7 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:23.458 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:23.474+0000 7f839affd640 1 -- 172.21.15.3:0/1592437932 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f838c030070 con 0x7f83840039b0 2024-05-09T06:37:23.458 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:23.474+0000 7f839affd640 1 -- 172.21.15.3:0/1592437932 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f838c02ae10 con 0x7f83840039b0 2024-05-09T06:37:23.458 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:23.474+0000 7f839affd640 1 -- 172.21.15.3:0/1592437932 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f838c00b600 con 0x7f83840039b0 2024-05-09T06:37:23.458 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:23.475+0000 7f83a228c640 1 -- 172.21.15.3:0/1592437932 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f83840039b0 msgr2=0x7f8384003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:23.458 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:23.475+0000 7f83a228c640 1 --2- 172.21.15.3:0/1592437932 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f83840039b0 0x7f8384003dd0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f838c00ba60 tx=0x7f8384005830 comp rx=0 tx=0).stop 2024-05-09T06:37:23.458 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:23.475+0000 7f83a0829640 1 -- 172.21.15.3:0/1592437932 reap_dead start 2024-05-09T06:37:23.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:23.475+0000 7f83a228c640 1 -- 172.21.15.3:0/1592437932 shutdown_connections 2024-05-09T06:37:23.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:23.475+0000 7f83a228c640 1 -- 172.21.15.3:0/1592437932 >> 172.21.15.3:0/1592437932 conn(0x7f839c06b430 msgr2=0x7f839c06b810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:23.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:23.475+0000 7f83a228c640 1 -- 172.21.15.3:0/1592437932 shutdown_connections 2024-05-09T06:37:23.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:23.475+0000 7f83a228c640 1 -- 172.21.15.3:0/1592437932 wait complete. 2024-05-09T06:37:23.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:23.476+0000 7f83a228c640 1 Processor -- start 2024-05-09T06:37:23.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:23.476+0000 7f83a228c640 1 -- start start 2024-05-09T06:37:23.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:23.477+0000 7f83a228c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f83840039b0 0x7f839c13f300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:23.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:23.477+0000 7f83a228c640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f839c13f9b0 0x7f839c140fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:23.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:23.477+0000 7f83a228c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f839c1414e0 0x7f839c142ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:23.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:23.477+0000 7f83a228c640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f839c06ebb0 con 0x7f83840039b0 2024-05-09T06:37:23.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:23.477+0000 7f83a228c640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f839c06ea30 con 0x7f839c1414e0 2024-05-09T06:37:23.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:23.477+0000 7f83a228c640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f839c06ed30 con 0x7f839c13f9b0 2024-05-09T06:37:23.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:23.477+0000 7f839bfff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f83840039b0 0x7f839c13f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:23.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:23.477+0000 7f839bfff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f83840039b0 0x7f839c13f300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40628/0 (socket says 172.21.15.3:40628) 2024-05-09T06:37:23.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:23.477+0000 7f839bfff640 1 -- 172.21.15.3:0/2899903044 learned_addr learned my addr 172.21.15.3:0/2899903044 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:37:23.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:23.477+0000 7f83a0829640 1 --2- 172.21.15.3:0/2899903044 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f839c1414e0 0x7f839c142ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:23.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:23.477+0000 7f839b7fe640 1 --2- 172.21.15.3:0/2899903044 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f839c13f9b0 0x7f839c140fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:23.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:23.477+0000 7f839bfff640 1 -- 172.21.15.3:0/2899903044 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f839c13f9b0 msgr2=0x7f839c140fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:23.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:23.477+0000 7f839bfff640 1 --2- 172.21.15.3:0/2899903044 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f839c13f9b0 0x7f839c140fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:23.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:23.477+0000 7f839bfff640 1 -- 172.21.15.3:0/2899903044 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f839c1414e0 msgr2=0x7f839c142ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:23.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:23.477+0000 7f839bfff640 1 --2- 172.21.15.3:0/2899903044 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f839c1414e0 0x7f839c142ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:23.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:23.477+0000 7f839bfff640 1 -- 172.21.15.3:0/2899903044 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f839c14fd60 con 0x7f83840039b0 2024-05-09T06:37:23.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:23.477+0000 7f839b7fe640 1 --2- 172.21.15.3:0/2899903044 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f839c13f9b0 0x7f839c140fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:37:23.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:23.477+0000 7f83a0829640 1 --2- 172.21.15.3:0/2899903044 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f839c1414e0 0x7f839c142ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:37:23.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:23.478+0000 7f839bfff640 1 --2- 172.21.15.3:0/2899903044 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f83840039b0 0x7f839c13f300 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f838c00bb80 tx=0x7f838c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:23.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:23.478+0000 7f8398ff9640 1 -- 172.21.15.3:0/2899903044 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f838c043020 con 0x7f83840039b0 2024-05-09T06:37:23.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:23.478+0000 7f8398ff9640 1 -- 172.21.15.3:0/2899903044 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f838c03f680 con 0x7f83840039b0 2024-05-09T06:37:23.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:23.478+0000 7f83a228c640 1 -- 172.21.15.3:0/2899903044 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f839c13ff00 con 0x7f83840039b0 2024-05-09T06:37:23.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:23.478+0000 7f8398ff9640 1 -- 172.21.15.3:0/2899903044 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f838c00b600 con 0x7f83840039b0 2024-05-09T06:37:23.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:23.478+0000 7f83a228c640 1 -- 172.21.15.3:0/2899903044 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f839c1404c0 con 0x7f83840039b0 2024-05-09T06:37:23.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:23.479+0000 7f83a228c640 1 -- 172.21.15.3:0/2899903044 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f839c04f660 con 0x7f83840039b0 2024-05-09T06:37:23.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:23.479+0000 7f8398ff9640 1 -- 172.21.15.3:0/2899903044 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 7) ==== 90651+0+0 (secure 0 0 0) 0x7f838c046020 con 0x7f83840039b0 2024-05-09T06:37:23.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:23.480+0000 7f8398ff9640 1 --2- 172.21.15.3:0/2899903044 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f838806bfe0 0x7f838806e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:23.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:23.480+0000 7f8398ff9640 1 -- 172.21.15.3:0/2899903044 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 4943+0+0 (secure 0 0 0) 0x7f838c0ab8f0 con 0x7f83840039b0 2024-05-09T06:37:23.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:23.480+0000 7f839b7fe640 1 --2- 172.21.15.3:0/2899903044 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f838806bfe0 0x7f838806e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:23.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:23.481+0000 7f839b7fe640 1 --2- 172.21.15.3:0/2899903044 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f838806bfe0 0x7f838806e4a0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f839c156220 tx=0x7f8390008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:23.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:23.484+0000 7f8398ff9640 1 -- 172.21.15.3:0/2899903044 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f838c04d070 con 0x7f83840039b0 2024-05-09T06:37:23.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:23.719+0000 7f83a228c640 1 -- 172.21.15.3:0/2899903044 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7f839c0af940 con 0x7f83840039b0 2024-05-09T06:37:23.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:23.723+0000 7f8398ff9640 1 -- 172.21.15.3:0/2899903044 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v7) ==== 81+0+471201 (secure 0 0 0) 0x7f838c03be20 con 0x7f83840039b0 2024-05-09T06:37:23.711 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-09T06:37:23.711 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-05-09T06:37:23.711 INFO:teuthology.orchestra.run.smithi003.stdout: "epoch": 7, 2024-05-09T06:37:23.711 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:23.711 INFO:teuthology.orchestra.run.smithi003.stdout: "active_gid": 4105, 2024-05-09T06:37:23.711 INFO:teuthology.orchestra.run.smithi003.stdout: "active_name": "x", 2024-05-09T06:37:23.711 INFO:teuthology.orchestra.run.smithi003.stdout: "active_addrs": { 2024-05-09T06:37:23.711 INFO:teuthology.orchestra.run.smithi003.stdout: "addrvec": [ 2024-05-09T06:37:23.711 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-09T06:37:23.711 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "v2", 2024-05-09T06:37:23.711 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": "172.21.15.190:6832", 2024-05-09T06:37:23.711 INFO:teuthology.orchestra.run.smithi003.stdout: "nonce": 975893321 2024-05-09T06:37:23.712 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.712 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-09T06:37:23.712 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "v1", 2024-05-09T06:37:23.712 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": "172.21.15.190:6833", 2024-05-09T06:37:23.712 INFO:teuthology.orchestra.run.smithi003.stdout: "nonce": 975893321 2024-05-09T06:37:23.712 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-09T06:37:23.712 INFO:teuthology.orchestra.run.smithi003.stdout: ] 2024-05-09T06:37:23.712 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.712 INFO:teuthology.orchestra.run.smithi003.stdout: "active_addr": "172.21.15.190:6833/975893321", 2024-05-09T06:37:23.712 INFO:teuthology.orchestra.run.smithi003.stdout: "active_change": "2024-05-09T06:35:46.764079+0000", 2024-05-09T06:37:23.712 INFO:teuthology.orchestra.run.smithi003.stdout: "active_mgr_features": 4540701547738038271, 2024-05-09T06:37:23.712 INFO:teuthology.orchestra.run.smithi003.stdout: "available": true, 2024-05-09T06:37:23.712 INFO:teuthology.orchestra.run.smithi003.stdout: "standbys": [ 2024-05-09T06:37:23.712 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-09T06:37:23.713 INFO:teuthology.orchestra.run.smithi003.stdout: "gid": 4107, 2024-05-09T06:37:23.713 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "y", 2024-05-09T06:37:23.713 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr_features": 4540701547738038271, 2024-05-09T06:37:23.713 INFO:teuthology.orchestra.run.smithi003.stdout: "available_modules": [ 2024-05-09T06:37:23.713 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-09T06:37:23.713 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "alerts", 2024-05-09T06:37:23.713 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-05-09T06:37:23.713 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-05-09T06:37:23.713 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-05-09T06:37:23.713 INFO:teuthology.orchestra.run.smithi003.stdout: "interval": { 2024-05-09T06:37:23.713 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "interval", 2024-05-09T06:37:23.713 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-05-09T06:37:23.713 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.713 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:23.713 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "60", 2024-05-09T06:37:23.714 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.714 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.714 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:23.714 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "How frequently to reexamine health status", 2024-05-09T06:37:23.714 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.714 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.714 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.714 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.714 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-05-09T06:37:23.714 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-05-09T06:37:23.714 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:23.714 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.714 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:23.714 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-09T06:37:23.715 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.715 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.715 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-09T06:37:23.715 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-09T06:37:23.715 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-09T06:37:23.715 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-09T06:37:23.715 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-09T06:37:23.715 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-09T06:37:23.715 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-09T06:37:23.715 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-09T06:37:23.715 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:23.715 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.715 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.715 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.715 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.716 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-05-09T06:37:23.716 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-05-09T06:37:23.716 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:23.716 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.716 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:23.716 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-09T06:37:23.716 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.716 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.716 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:23.716 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:23.716 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.716 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.716 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.716 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.717 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-05-09T06:37:23.717 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-05-09T06:37:23.717 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:23.717 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.717 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:23.717 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-05-09T06:37:23.717 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.717 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.717 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-09T06:37:23.717 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-09T06:37:23.717 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-09T06:37:23.717 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-09T06:37:23.717 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-09T06:37:23.717 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-09T06:37:23.717 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-09T06:37:23.718 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-09T06:37:23.718 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:23.718 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.718 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.718 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.718 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.718 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-05-09T06:37:23.718 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-05-09T06:37:23.718 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:23.718 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.718 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:23.718 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-09T06:37:23.718 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.718 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.719 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:23.719 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:23.719 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.719 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.719 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.719 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.719 INFO:teuthology.orchestra.run.smithi003.stdout: "smtp_destination": { 2024-05-09T06:37:23.719 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "smtp_destination", 2024-05-09T06:37:23.719 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:23.719 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.719 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:23.719 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-09T06:37:23.719 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.719 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.720 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:23.720 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Email address to send alerts to", 2024-05-09T06:37:23.720 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.720 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.720 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.720 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.720 INFO:teuthology.orchestra.run.smithi003.stdout: "smtp_from_name": { 2024-05-09T06:37:23.720 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "smtp_from_name", 2024-05-09T06:37:23.720 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:23.720 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.720 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:23.720 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "Ceph", 2024-05-09T06:37:23.720 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.720 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.720 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:23.720 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Email From: name", 2024-05-09T06:37:23.721 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.721 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.721 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.721 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.721 INFO:teuthology.orchestra.run.smithi003.stdout: "smtp_host": { 2024-05-09T06:37:23.721 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "smtp_host", 2024-05-09T06:37:23.721 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:23.721 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.721 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:23.721 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-09T06:37:23.721 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.721 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.721 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:23.721 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "SMTP server", 2024-05-09T06:37:23.722 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.722 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.722 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.722 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.722 INFO:teuthology.orchestra.run.smithi003.stdout: "smtp_password": { 2024-05-09T06:37:23.722 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "smtp_password", 2024-05-09T06:37:23.722 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:23.722 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.722 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:23.722 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-09T06:37:23.722 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.722 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.722 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:23.722 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Password to authenticate with", 2024-05-09T06:37:23.722 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.723 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.723 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.723 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.723 INFO:teuthology.orchestra.run.smithi003.stdout: "smtp_port": { 2024-05-09T06:37:23.723 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "smtp_port", 2024-05-09T06:37:23.723 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-09T06:37:23.723 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.723 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:23.723 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "465", 2024-05-09T06:37:23.723 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.723 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.723 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:23.723 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "SMTP port", 2024-05-09T06:37:23.723 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.724 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.724 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.724 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.724 INFO:teuthology.orchestra.run.smithi003.stdout: "smtp_sender": { 2024-05-09T06:37:23.724 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "smtp_sender", 2024-05-09T06:37:23.724 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:23.724 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.724 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:23.724 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-09T06:37:23.724 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.724 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.724 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:23.724 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "SMTP envelope sender", 2024-05-09T06:37:23.724 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.725 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.725 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.725 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.725 INFO:teuthology.orchestra.run.smithi003.stdout: "smtp_ssl": { 2024-05-09T06:37:23.725 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "smtp_ssl", 2024-05-09T06:37:23.725 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:23.725 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.725 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:23.725 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-05-09T06:37:23.725 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.725 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.725 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:23.725 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Use SSL to connect to SMTP server", 2024-05-09T06:37:23.725 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.726 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.726 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.726 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.726 INFO:teuthology.orchestra.run.smithi003.stdout: "smtp_user": { 2024-05-09T06:37:23.726 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "smtp_user", 2024-05-09T06:37:23.726 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:23.726 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.726 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:23.726 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-09T06:37:23.726 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.726 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.726 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:23.726 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "User to authenticate as", 2024-05-09T06:37:23.726 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.727 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.727 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.727 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-09T06:37:23.727 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-09T06:37:23.727 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.727 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-09T06:37:23.727 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "balancer", 2024-05-09T06:37:23.727 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-05-09T06:37:23.727 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-05-09T06:37:23.727 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-05-09T06:37:23.727 INFO:teuthology.orchestra.run.smithi003.stdout: "active": { 2024-05-09T06:37:23.727 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "active", 2024-05-09T06:37:23.727 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:23.727 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.727 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:23.728 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-05-09T06:37:23.728 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.728 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.728 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:23.728 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "automatically balance PGs across cluster", 2024-05-09T06:37:23.728 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.728 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.728 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.728 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.728 INFO:teuthology.orchestra.run.smithi003.stdout: "begin_time": { 2024-05-09T06:37:23.728 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "begin_time", 2024-05-09T06:37:23.728 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:23.728 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.728 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:23.729 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0000", 2024-05-09T06:37:23.729 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.729 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.729 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:23.729 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "beginning time of day to automatically balance", 2024-05-09T06:37:23.729 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-09T06:37:23.729 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.729 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.729 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.729 INFO:teuthology.orchestra.run.smithi003.stdout: "begin_weekday": { 2024-05-09T06:37:23.729 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "begin_weekday", 2024-05-09T06:37:23.729 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "uint", 2024-05-09T06:37:23.729 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.729 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:23.730 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-05-09T06:37:23.730 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "0", 2024-05-09T06:37:23.730 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "6", 2024-05-09T06:37:23.730 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:23.730 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-05-09T06:37:23.730 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-09T06:37:23.730 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.730 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.730 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.730 INFO:teuthology.orchestra.run.smithi003.stdout: "crush_compat_max_iterations": { 2024-05-09T06:37:23.730 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "crush_compat_max_iterations", 2024-05-09T06:37:23.730 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "uint", 2024-05-09T06:37:23.730 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.730 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:23.731 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "25", 2024-05-09T06:37:23.731 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "1", 2024-05-09T06:37:23.731 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "250", 2024-05-09T06:37:23.731 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:23.731 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-05-09T06:37:23.731 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.731 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.731 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.731 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.731 INFO:teuthology.orchestra.run.smithi003.stdout: "crush_compat_metrics": { 2024-05-09T06:37:23.731 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "crush_compat_metrics", 2024-05-09T06:37:23.731 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:23.731 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.731 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:23.731 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "pgs,objects,bytes", 2024-05-09T06:37:23.732 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.732 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.732 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:23.732 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-05-09T06:37:23.732 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-05-09T06:37:23.732 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.732 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.732 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.732 INFO:teuthology.orchestra.run.smithi003.stdout: "crush_compat_step": { 2024-05-09T06:37:23.732 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "crush_compat_step", 2024-05-09T06:37:23.732 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "float", 2024-05-09T06:37:23.732 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.732 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:23.732 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0.5", 2024-05-09T06:37:23.733 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "0.001", 2024-05-09T06:37:23.733 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "0.999", 2024-05-09T06:37:23.733 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:23.733 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "aggressiveness of optimization", 2024-05-09T06:37:23.733 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-05-09T06:37:23.733 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.733 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.733 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.733 INFO:teuthology.orchestra.run.smithi003.stdout: "end_time": { 2024-05-09T06:37:23.733 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "end_time", 2024-05-09T06:37:23.733 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:23.733 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.733 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:23.733 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "2359", 2024-05-09T06:37:23.734 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.734 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.734 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:23.734 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "ending time of day to automatically balance", 2024-05-09T06:37:23.734 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-09T06:37:23.734 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.734 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.734 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.734 INFO:teuthology.orchestra.run.smithi003.stdout: "end_weekday": { 2024-05-09T06:37:23.734 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "end_weekday", 2024-05-09T06:37:23.734 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "uint", 2024-05-09T06:37:23.734 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.734 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:23.734 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-05-09T06:37:23.734 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "0", 2024-05-09T06:37:23.735 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "6", 2024-05-09T06:37:23.735 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:23.735 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-05-09T06:37:23.735 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-09T06:37:23.735 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.735 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.735 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.735 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-05-09T06:37:23.735 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-05-09T06:37:23.735 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:23.735 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.735 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:23.735 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-09T06:37:23.735 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.736 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.736 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-09T06:37:23.736 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-09T06:37:23.736 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-09T06:37:23.736 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-09T06:37:23.736 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-09T06:37:23.736 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-09T06:37:23.736 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-09T06:37:23.736 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-09T06:37:23.736 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:23.736 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.736 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.736 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.737 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.737 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-05-09T06:37:23.737 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-05-09T06:37:23.737 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:23.737 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.737 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:23.737 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-09T06:37:23.737 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.737 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.737 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:23.737 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:23.737 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.737 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.738 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.738 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.738 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-05-09T06:37:23.738 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-05-09T06:37:23.738 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:23.738 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.738 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:23.738 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-05-09T06:37:23.738 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.738 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.738 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-09T06:37:23.738 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-09T06:37:23.738 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-09T06:37:23.739 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-09T06:37:23.739 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-09T06:37:23.739 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-09T06:37:23.739 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-09T06:37:23.739 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-09T06:37:23.739 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:23.739 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.739 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.739 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.739 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.739 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-05-09T06:37:23.739 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-05-09T06:37:23.739 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:23.740 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.740 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:23.740 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-09T06:37:23.740 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.740 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.740 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:23.740 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:23.740 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.740 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.740 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.740 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.740 INFO:teuthology.orchestra.run.smithi003.stdout: "min_score": { 2024-05-09T06:37:23.741 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "min_score", 2024-05-09T06:37:23.741 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "float", 2024-05-09T06:37:23.741 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.741 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:23.741 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-05-09T06:37:23.741 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.741 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.741 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:23.741 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-05-09T06:37:23.741 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.741 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.741 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.741 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.741 INFO:teuthology.orchestra.run.smithi003.stdout: "mode": { 2024-05-09T06:37:23.742 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "mode", 2024-05-09T06:37:23.742 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:23.742 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.742 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:23.742 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "upmap", 2024-05-09T06:37:23.742 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.742 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.742 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-09T06:37:23.742 INFO:teuthology.orchestra.run.smithi003.stdout: "crush-compat", 2024-05-09T06:37:23.742 INFO:teuthology.orchestra.run.smithi003.stdout: "none", 2024-05-09T06:37:23.742 INFO:teuthology.orchestra.run.smithi003.stdout: "read", 2024-05-09T06:37:23.742 INFO:teuthology.orchestra.run.smithi003.stdout: "upmap", 2024-05-09T06:37:23.742 INFO:teuthology.orchestra.run.smithi003.stdout: "upmap-read" 2024-05-09T06:37:23.743 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-09T06:37:23.743 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Balancer mode", 2024-05-09T06:37:23.743 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.743 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.743 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.743 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.743 INFO:teuthology.orchestra.run.smithi003.stdout: "pool_ids": { 2024-05-09T06:37:23.743 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "pool_ids", 2024-05-09T06:37:23.743 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:23.743 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.743 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:23.743 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-09T06:37:23.743 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.743 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.744 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:23.744 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-05-09T06:37:23.744 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.744 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.744 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.744 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.744 INFO:teuthology.orchestra.run.smithi003.stdout: "sleep_interval": { 2024-05-09T06:37:23.744 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sleep_interval", 2024-05-09T06:37:23.744 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-05-09T06:37:23.744 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.744 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:23.744 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "60", 2024-05-09T06:37:23.744 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.744 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.744 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:23.744 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-05-09T06:37:23.745 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.745 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.745 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.745 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.745 INFO:teuthology.orchestra.run.smithi003.stdout: "upmap_max_deviation": { 2024-05-09T06:37:23.745 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "upmap_max_deviation", 2024-05-09T06:37:23.745 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-09T06:37:23.745 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.745 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:23.745 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "5", 2024-05-09T06:37:23.745 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "1", 2024-05-09T06:37:23.745 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.745 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:23.745 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "deviation below which no optimization is attempted", 2024-05-09T06:37:23.746 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-05-09T06:37:23.746 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.746 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.746 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.746 INFO:teuthology.orchestra.run.smithi003.stdout: "upmap_max_optimizations": { 2024-05-09T06:37:23.746 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "upmap_max_optimizations", 2024-05-09T06:37:23.746 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "uint", 2024-05-09T06:37:23.746 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.746 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:23.746 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "10", 2024-05-09T06:37:23.746 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.746 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.746 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:23.746 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-05-09T06:37:23.747 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.747 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.747 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.747 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-09T06:37:23.747 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-09T06:37:23.747 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.747 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-09T06:37:23.747 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "cephadm", 2024-05-09T06:37:23.747 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-05-09T06:37:23.747 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-05-09T06:37:23.747 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-05-09T06:37:23.747 INFO:teuthology.orchestra.run.smithi003.stdout: "agent_down_multiplier": { 2024-05-09T06:37:23.747 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "agent_down_multiplier", 2024-05-09T06:37:23.747 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "float", 2024-05-09T06:37:23.747 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.748 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:23.748 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "3.0", 2024-05-09T06:37:23.748 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.748 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.748 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:23.748 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-05-09T06:37:23.748 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.748 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.748 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.748 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.748 INFO:teuthology.orchestra.run.smithi003.stdout: "agent_refresh_rate": { 2024-05-09T06:37:23.748 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "agent_refresh_rate", 2024-05-09T06:37:23.748 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-05-09T06:37:23.748 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.748 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:23.749 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "20", 2024-05-09T06:37:23.749 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.749 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.749 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:23.749 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-05-09T06:37:23.749 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.749 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.749 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.749 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.749 INFO:teuthology.orchestra.run.smithi003.stdout: "agent_starting_port": { 2024-05-09T06:37:23.749 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "agent_starting_port", 2024-05-09T06:37:23.749 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-09T06:37:23.750 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.750 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:23.750 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "4721", 2024-05-09T06:37:23.750 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.750 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.750 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:23.750 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-05-09T06:37:23.750 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.750 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.750 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.750 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.750 INFO:teuthology.orchestra.run.smithi003.stdout: "allow_ptrace": { 2024-05-09T06:37:23.750 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "allow_ptrace", 2024-05-09T06:37:23.750 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:23.751 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.751 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:23.751 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-09T06:37:23.751 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.751 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.751 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:23.751 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-05-09T06:37:23.751 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-05-09T06:37:23.751 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.751 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.751 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.751 INFO:teuthology.orchestra.run.smithi003.stdout: "autotune_interval": { 2024-05-09T06:37:23.751 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "autotune_interval", 2024-05-09T06:37:23.751 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-05-09T06:37:23.751 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.752 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:23.752 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "600", 2024-05-09T06:37:23.752 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.752 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.752 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:23.752 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "how frequently to autotune daemon memory", 2024-05-09T06:37:23.752 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.752 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.752 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.752 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.752 INFO:teuthology.orchestra.run.smithi003.stdout: "autotune_memory_target_ratio": { 2024-05-09T06:37:23.752 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "autotune_memory_target_ratio", 2024-05-09T06:37:23.752 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "float", 2024-05-09T06:37:23.753 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.753 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:23.753 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0.7", 2024-05-09T06:37:23.753 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.753 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.753 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:23.753 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-05-09T06:37:23.753 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.753 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.753 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.753 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.753 INFO:teuthology.orchestra.run.smithi003.stdout: "cephadm_log_destination": { 2024-05-09T06:37:23.753 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "cephadm_log_destination", 2024-05-09T06:37:23.753 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:23.753 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.754 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:23.754 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-09T06:37:23.754 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.754 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.754 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-09T06:37:23.754 INFO:teuthology.orchestra.run.smithi003.stdout: "file", 2024-05-09T06:37:23.754 INFO:teuthology.orchestra.run.smithi003.stdout: "file,syslog", 2024-05-09T06:37:23.754 INFO:teuthology.orchestra.run.smithi003.stdout: "syslog" 2024-05-09T06:37:23.754 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-09T06:37:23.754 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-05-09T06:37:23.754 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.754 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.754 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.755 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.755 INFO:teuthology.orchestra.run.smithi003.stdout: "cgroups_split": { 2024-05-09T06:37:23.755 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "cgroups_split", 2024-05-09T06:37:23.755 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:23.755 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.755 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:23.755 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-05-09T06:37:23.755 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.755 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.755 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:23.755 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-05-09T06:37:23.755 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.755 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.755 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.756 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.756 INFO:teuthology.orchestra.run.smithi003.stdout: "config_checks_enabled": { 2024-05-09T06:37:23.756 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "config_checks_enabled", 2024-05-09T06:37:23.756 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:23.756 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.756 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:23.756 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-09T06:37:23.756 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.756 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.756 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:23.756 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-05-09T06:37:23.756 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.756 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.756 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.756 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.757 INFO:teuthology.orchestra.run.smithi003.stdout: "config_dashboard": { 2024-05-09T06:37:23.757 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "config_dashboard", 2024-05-09T06:37:23.757 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:23.757 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.757 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:23.757 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-05-09T06:37:23.757 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.757 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.757 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:23.757 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-05-09T06:37:23.757 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.757 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.757 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.758 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.758 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_alertmanager": { 2024-05-09T06:37:23.758 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_alertmanager", 2024-05-09T06:37:23.758 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:23.758 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.758 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:23.758 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-05-09T06:37:23.758 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.758 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.758 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:23.758 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Prometheus container image", 2024-05-09T06:37:23.758 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.758 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.758 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.759 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.759 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_base": { 2024-05-09T06:37:23.759 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_base", 2024-05-09T06:37:23.759 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:23.759 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.759 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:23.759 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/ceph/ceph", 2024-05-09T06:37:23.759 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.759 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.759 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:23.759 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Container image name, without the tag", 2024-05-09T06:37:23.759 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.759 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.759 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.760 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.760 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_elasticsearch": { 2024-05-09T06:37:23.760 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_elasticsearch", 2024-05-09T06:37:23.760 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:23.760 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.760 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:23.760 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-05-09T06:37:23.760 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.760 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.760 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:23.760 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "elasticsearch container image", 2024-05-09T06:37:23.760 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.760 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.761 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.761 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.761 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_grafana": { 2024-05-09T06:37:23.761 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_grafana", 2024-05-09T06:37:23.761 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:23.761 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.761 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:23.761 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-05-09T06:37:23.761 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.761 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.761 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:23.761 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Prometheus container image", 2024-05-09T06:37:23.761 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.761 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.761 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.761 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.762 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_haproxy": { 2024-05-09T06:37:23.762 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_haproxy", 2024-05-09T06:37:23.762 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:23.762 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.762 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:23.762 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-05-09T06:37:23.762 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.762 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.762 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:23.762 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "HAproxy container image", 2024-05-09T06:37:23.762 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.762 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.762 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.763 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.763 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_jaeger_agent": { 2024-05-09T06:37:23.763 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_jaeger_agent", 2024-05-09T06:37:23.763 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:23.763 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.763 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:23.763 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-05-09T06:37:23.763 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.763 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.763 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:23.763 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Jaeger agent container image", 2024-05-09T06:37:23.763 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.763 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.763 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.763 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.764 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_jaeger_collector": { 2024-05-09T06:37:23.764 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_jaeger_collector", 2024-05-09T06:37:23.764 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:23.764 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.764 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:23.764 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-05-09T06:37:23.764 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.764 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.764 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:23.764 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Jaeger collector container image", 2024-05-09T06:37:23.764 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.764 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.764 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.764 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.765 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_jaeger_query": { 2024-05-09T06:37:23.765 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_jaeger_query", 2024-05-09T06:37:23.765 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:23.765 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.765 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:23.765 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-05-09T06:37:23.765 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.765 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.765 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:23.765 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Jaeger query container image", 2024-05-09T06:37:23.765 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.765 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.765 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.766 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.766 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_keepalived": { 2024-05-09T06:37:23.766 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_keepalived", 2024-05-09T06:37:23.766 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:23.766 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.766 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:23.766 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-05-09T06:37:23.766 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.766 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.766 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:23.766 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Keepalived container image", 2024-05-09T06:37:23.766 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.767 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.767 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.767 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.767 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_loki": { 2024-05-09T06:37:23.767 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_loki", 2024-05-09T06:37:23.767 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:23.767 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.767 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:23.767 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-05-09T06:37:23.767 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.767 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.767 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:23.768 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Loki container image", 2024-05-09T06:37:23.768 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.768 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.768 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.768 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.768 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_node_exporter": { 2024-05-09T06:37:23.768 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_node_exporter", 2024-05-09T06:37:23.768 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:23.768 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.768 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:23.768 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-05-09T06:37:23.768 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.768 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.769 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:23.769 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Prometheus container image", 2024-05-09T06:37:23.769 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.769 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.769 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.769 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.769 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_nvmeof": { 2024-05-09T06:37:23.769 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_nvmeof", 2024-05-09T06:37:23.769 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:23.769 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.769 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:23.769 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-05-09T06:37:23.769 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.770 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.770 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:23.770 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Nvme-of container image", 2024-05-09T06:37:23.770 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.770 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.770 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.770 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.770 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_prometheus": { 2024-05-09T06:37:23.770 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_prometheus", 2024-05-09T06:37:23.770 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:23.770 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.770 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:23.770 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-05-09T06:37:23.770 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.771 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.771 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:23.771 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Prometheus container image", 2024-05-09T06:37:23.771 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.771 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.771 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.771 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.771 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_promtail": { 2024-05-09T06:37:23.771 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_promtail", 2024-05-09T06:37:23.771 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:23.771 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.771 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:23.771 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-05-09T06:37:23.772 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.772 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.772 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:23.772 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Promtail container image", 2024-05-09T06:37:23.772 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.772 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.772 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.772 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.772 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_samba": { 2024-05-09T06:37:23.772 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_samba", 2024-05-09T06:37:23.772 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:23.772 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.772 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:23.772 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-05-09T06:37:23.772 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.773 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.773 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:23.773 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Samba/SMB container image", 2024-05-09T06:37:23.773 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.773 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.773 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.773 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.773 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_snmp_gateway": { 2024-05-09T06:37:23.773 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_snmp_gateway", 2024-05-09T06:37:23.773 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:23.773 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.773 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:23.773 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-05-09T06:37:23.773 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.774 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.774 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:23.774 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "SNMP Gateway container image", 2024-05-09T06:37:23.774 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.774 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.774 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.774 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.774 INFO:teuthology.orchestra.run.smithi003.stdout: "container_init": { 2024-05-09T06:37:23.774 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_init", 2024-05-09T06:37:23.774 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:23.774 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.774 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:23.774 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-05-09T06:37:23.774 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.775 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.775 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:23.775 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Run podman/docker with `--init`", 2024-05-09T06:37:23.775 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.775 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.775 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.775 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.775 INFO:teuthology.orchestra.run.smithi003.stdout: "daemon_cache_timeout": { 2024-05-09T06:37:23.775 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "daemon_cache_timeout", 2024-05-09T06:37:23.775 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-05-09T06:37:23.775 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.775 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:23.775 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "600", 2024-05-09T06:37:23.775 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.775 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.776 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:23.776 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-05-09T06:37:23.776 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.776 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.776 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.776 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.776 INFO:teuthology.orchestra.run.smithi003.stdout: "default_cephadm_command_timeout": { 2024-05-09T06:37:23.776 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "default_cephadm_command_timeout", 2024-05-09T06:37:23.776 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-09T06:37:23.776 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.776 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:23.776 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "900", 2024-05-09T06:37:23.776 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.776 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.776 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:23.777 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-05-09T06:37:23.777 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.777 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.777 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.777 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.777 INFO:teuthology.orchestra.run.smithi003.stdout: "default_registry": { 2024-05-09T06:37:23.777 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "default_registry", 2024-05-09T06:37:23.777 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:23.777 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.777 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:23.777 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "docker.io", 2024-05-09T06:37:23.777 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.777 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.777 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:23.777 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-05-09T06:37:23.777 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.777 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.778 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.778 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.778 INFO:teuthology.orchestra.run.smithi003.stdout: "device_cache_timeout": { 2024-05-09T06:37:23.778 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "device_cache_timeout", 2024-05-09T06:37:23.778 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-05-09T06:37:23.778 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.778 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:23.778 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "1800", 2024-05-09T06:37:23.778 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.778 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.778 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:23.778 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "seconds to cache device inventory", 2024-05-09T06:37:23.778 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.778 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.779 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.779 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.779 INFO:teuthology.orchestra.run.smithi003.stdout: "device_enhanced_scan": { 2024-05-09T06:37:23.779 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "device_enhanced_scan", 2024-05-09T06:37:23.779 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:23.779 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.779 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:23.779 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-09T06:37:23.779 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.779 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.779 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:23.779 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Use libstoragemgmt during device scans", 2024-05-09T06:37:23.779 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.779 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.779 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.779 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.779 INFO:teuthology.orchestra.run.smithi003.stdout: "facts_cache_timeout": { 2024-05-09T06:37:23.780 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "facts_cache_timeout", 2024-05-09T06:37:23.780 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-05-09T06:37:23.780 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.780 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:23.780 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "60", 2024-05-09T06:37:23.780 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.780 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.780 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:23.780 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "seconds to cache host facts data", 2024-05-09T06:37:23.780 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.780 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.780 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.780 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.781 INFO:teuthology.orchestra.run.smithi003.stdout: "grafana_dashboards_path": { 2024-05-09T06:37:23.781 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "grafana_dashboards_path", 2024-05-09T06:37:23.781 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:23.781 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.781 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:23.781 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-05-09T06:37:23.781 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.781 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.781 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:23.781 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-05-09T06:37:23.781 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.781 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.781 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.781 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.781 INFO:teuthology.orchestra.run.smithi003.stdout: "host_check_interval": { 2024-05-09T06:37:23.781 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "host_check_interval", 2024-05-09T06:37:23.782 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-05-09T06:37:23.782 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.782 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:23.782 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "600", 2024-05-09T06:37:23.782 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.782 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.782 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:23.782 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "how frequently to perform a host check", 2024-05-09T06:37:23.782 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.782 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.782 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.782 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.782 INFO:teuthology.orchestra.run.smithi003.stdout: "hw_monitoring": { 2024-05-09T06:37:23.782 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "hw_monitoring", 2024-05-09T06:37:23.783 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:23.783 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.783 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:23.783 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-09T06:37:23.783 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.783 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.783 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:23.783 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-05-09T06:37:23.783 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.783 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.783 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.783 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.783 INFO:teuthology.orchestra.run.smithi003.stdout: "inventory_list_all": { 2024-05-09T06:37:23.783 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "inventory_list_all", 2024-05-09T06:37:23.784 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:23.784 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.784 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:23.784 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-09T06:37:23.784 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.784 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.784 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:23.784 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-05-09T06:37:23.784 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.784 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.784 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.784 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.784 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-05-09T06:37:23.784 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-05-09T06:37:23.785 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:23.785 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.785 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:23.785 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-09T06:37:23.785 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.785 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.785 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-09T06:37:23.785 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-09T06:37:23.785 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-09T06:37:23.785 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-09T06:37:23.785 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-09T06:37:23.785 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-09T06:37:23.785 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-09T06:37:23.785 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-09T06:37:23.786 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:23.786 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.786 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.786 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.786 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.786 INFO:teuthology.orchestra.run.smithi003.stdout: "log_refresh_metadata": { 2024-05-09T06:37:23.786 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_refresh_metadata", 2024-05-09T06:37:23.786 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:23.786 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.786 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:23.786 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-09T06:37:23.786 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.786 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.786 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:23.787 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-05-09T06:37:23.787 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.787 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.787 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.787 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.787 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-05-09T06:37:23.787 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-05-09T06:37:23.787 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:23.787 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.787 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:23.787 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-05-09T06:37:23.787 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.787 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.787 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:23.788 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-05-09T06:37:23.788 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.788 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.788 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.788 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.788 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-05-09T06:37:23.788 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-05-09T06:37:23.788 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:23.788 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.788 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:23.788 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-05-09T06:37:23.788 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.788 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.788 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-09T06:37:23.789 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-09T06:37:23.789 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-09T06:37:23.789 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-09T06:37:23.789 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-09T06:37:23.789 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-09T06:37:23.789 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-09T06:37:23.789 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-09T06:37:23.789 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:23.789 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.789 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.789 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.789 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.789 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-05-09T06:37:23.789 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-05-09T06:37:23.789 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:23.790 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.790 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:23.790 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-09T06:37:23.790 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.790 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.790 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:23.790 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:23.790 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.790 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.790 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.790 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.790 INFO:teuthology.orchestra.run.smithi003.stdout: "manage_etc_ceph_ceph_conf": { 2024-05-09T06:37:23.791 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-05-09T06:37:23.791 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:23.791 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.791 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:23.791 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-09T06:37:23.791 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.791 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.791 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:23.791 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-05-09T06:37:23.791 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.791 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.791 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.791 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.791 INFO:teuthology.orchestra.run.smithi003.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-05-09T06:37:23.791 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-05-09T06:37:23.792 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:23.792 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.792 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:23.792 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "*", 2024-05-09T06:37:23.792 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.792 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.792 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:23.792 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-05-09T06:37:23.792 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.792 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.792 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.792 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.792 INFO:teuthology.orchestra.run.smithi003.stdout: "max_count_per_host": { 2024-05-09T06:37:23.792 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "max_count_per_host", 2024-05-09T06:37:23.793 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-09T06:37:23.793 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.793 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:23.793 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "10", 2024-05-09T06:37:23.793 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.793 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.793 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:23.793 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "max number of daemons per service per host", 2024-05-09T06:37:23.793 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.793 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.793 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.793 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.793 INFO:teuthology.orchestra.run.smithi003.stdout: "max_osd_draining_count": { 2024-05-09T06:37:23.793 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "max_osd_draining_count", 2024-05-09T06:37:23.793 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-09T06:37:23.793 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.794 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:23.794 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "10", 2024-05-09T06:37:23.794 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.794 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.794 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:23.794 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-05-09T06:37:23.794 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.794 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.794 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.794 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.794 INFO:teuthology.orchestra.run.smithi003.stdout: "migration_current": { 2024-05-09T06:37:23.794 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "migration_current", 2024-05-09T06:37:23.794 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-09T06:37:23.794 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.795 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:23.795 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-09T06:37:23.795 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.795 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.795 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:23.795 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "internal - do not modify", 2024-05-09T06:37:23.795 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.795 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.795 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.795 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.795 INFO:teuthology.orchestra.run.smithi003.stdout: "mode": { 2024-05-09T06:37:23.795 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "mode", 2024-05-09T06:37:23.795 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:23.795 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.795 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:23.796 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "root", 2024-05-09T06:37:23.796 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.796 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.796 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-09T06:37:23.796 INFO:teuthology.orchestra.run.smithi003.stdout: "cephadm-package", 2024-05-09T06:37:23.796 INFO:teuthology.orchestra.run.smithi003.stdout: "root" 2024-05-09T06:37:23.796 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-09T06:37:23.796 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "mode for remote execution of cephadm", 2024-05-09T06:37:23.796 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.796 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.796 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.796 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.796 INFO:teuthology.orchestra.run.smithi003.stdout: "oob_default_addr": { 2024-05-09T06:37:23.796 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "oob_default_addr", 2024-05-09T06:37:23.797 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:23.797 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.797 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:23.797 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "169.254.1.1", 2024-05-09T06:37:23.797 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.797 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.797 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:23.797 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Default address for RedFish API (oob management).", 2024-05-09T06:37:23.797 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.797 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.797 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.797 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.797 INFO:teuthology.orchestra.run.smithi003.stdout: "prometheus_alerts_path": { 2024-05-09T06:37:23.798 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "prometheus_alerts_path", 2024-05-09T06:37:23.798 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:23.798 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.798 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:23.798 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-05-09T06:37:23.798 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.798 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.798 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:23.798 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-05-09T06:37:23.798 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.798 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.798 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.798 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.798 INFO:teuthology.orchestra.run.smithi003.stdout: "registry_insecure": { 2024-05-09T06:37:23.799 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "registry_insecure", 2024-05-09T06:37:23.799 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:23.799 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.799 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:23.799 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-09T06:37:23.799 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.799 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.799 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:23.799 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-05-09T06:37:23.799 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.799 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.799 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.799 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.799 INFO:teuthology.orchestra.run.smithi003.stdout: "registry_password": { 2024-05-09T06:37:23.800 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "registry_password", 2024-05-09T06:37:23.800 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:23.800 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.800 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:23.800 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-09T06:37:23.800 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.800 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.800 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:23.800 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-05-09T06:37:23.800 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.800 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.800 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.800 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.800 INFO:teuthology.orchestra.run.smithi003.stdout: "registry_url": { 2024-05-09T06:37:23.800 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "registry_url", 2024-05-09T06:37:23.800 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:23.801 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.801 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:23.801 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-09T06:37:23.801 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.801 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.801 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:23.801 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-05-09T06:37:23.801 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.801 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.801 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.801 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.801 INFO:teuthology.orchestra.run.smithi003.stdout: "registry_username": { 2024-05-09T06:37:23.801 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "registry_username", 2024-05-09T06:37:23.801 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:23.802 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.802 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:23.802 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-09T06:37:23.802 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.802 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.802 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:23.802 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-05-09T06:37:23.802 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.802 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.802 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.802 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.802 INFO:teuthology.orchestra.run.smithi003.stdout: "secure_monitoring_stack": { 2024-05-09T06:37:23.802 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "secure_monitoring_stack", 2024-05-09T06:37:23.802 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:23.803 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.803 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:23.803 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-09T06:37:23.803 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.803 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.803 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:23.803 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-05-09T06:37:23.803 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.803 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.803 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.803 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.803 INFO:teuthology.orchestra.run.smithi003.stdout: "service_discovery_port": { 2024-05-09T06:37:23.803 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "service_discovery_port", 2024-05-09T06:37:23.803 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-09T06:37:23.803 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.804 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:23.804 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "8765", 2024-05-09T06:37:23.804 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.804 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.804 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:23.804 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "cephadm service discovery port", 2024-05-09T06:37:23.804 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.804 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.804 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.804 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.804 INFO:teuthology.orchestra.run.smithi003.stdout: "ssh_config_file": { 2024-05-09T06:37:23.804 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "ssh_config_file", 2024-05-09T06:37:23.804 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:23.804 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.804 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:23.805 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-09T06:37:23.805 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.805 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.805 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:23.805 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-05-09T06:37:23.805 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.805 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.805 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.805 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.805 INFO:teuthology.orchestra.run.smithi003.stdout: "use_agent": { 2024-05-09T06:37:23.805 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "use_agent", 2024-05-09T06:37:23.805 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:23.805 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.805 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:23.806 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-09T06:37:23.806 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.806 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.806 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:23.806 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-05-09T06:37:23.806 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.806 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.806 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.806 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.806 INFO:teuthology.orchestra.run.smithi003.stdout: "use_repo_digest": { 2024-05-09T06:37:23.806 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "use_repo_digest", 2024-05-09T06:37:23.806 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:23.806 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.806 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:23.807 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-05-09T06:37:23.807 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.807 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.807 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:23.807 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-05-09T06:37:23.807 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.807 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.807 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.807 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.807 INFO:teuthology.orchestra.run.smithi003.stdout: "warn_on_failed_host_check": { 2024-05-09T06:37:23.807 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "warn_on_failed_host_check", 2024-05-09T06:37:23.807 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:23.807 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.808 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:23.808 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-05-09T06:37:23.808 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.808 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.808 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:23.808 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "raise a health warning if the host check fails", 2024-05-09T06:37:23.808 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.808 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.808 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.808 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.808 INFO:teuthology.orchestra.run.smithi003.stdout: "warn_on_stray_daemons": { 2024-05-09T06:37:23.808 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "warn_on_stray_daemons", 2024-05-09T06:37:23.808 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:23.808 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.809 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:23.809 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-05-09T06:37:23.809 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.809 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.809 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:23.809 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-05-09T06:37:23.809 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.809 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.809 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.809 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.809 INFO:teuthology.orchestra.run.smithi003.stdout: "warn_on_stray_hosts": { 2024-05-09T06:37:23.809 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "warn_on_stray_hosts", 2024-05-09T06:37:23.809 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:23.809 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.810 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:23.810 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-05-09T06:37:23.810 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.810 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.810 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:23.810 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-05-09T06:37:23.810 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.810 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.810 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.810 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-09T06:37:23.810 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-09T06:37:23.810 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.810 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-09T06:37:23.810 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "crash", 2024-05-09T06:37:23.810 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-05-09T06:37:23.811 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-05-09T06:37:23.811 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-05-09T06:37:23.811 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-05-09T06:37:23.811 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-05-09T06:37:23.811 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:23.811 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.811 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:23.811 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-09T06:37:23.811 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.811 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.811 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-09T06:37:23.811 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-09T06:37:23.811 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-09T06:37:23.811 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-09T06:37:23.812 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-09T06:37:23.812 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-09T06:37:23.812 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-09T06:37:23.812 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-09T06:37:23.812 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:23.812 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.812 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.812 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.812 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.812 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-05-09T06:37:23.812 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-05-09T06:37:23.812 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:23.812 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.813 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:23.813 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-09T06:37:23.813 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.813 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.813 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:23.813 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:23.813 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.813 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.813 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.813 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.813 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-05-09T06:37:23.813 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-05-09T06:37:23.813 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:23.813 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.813 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:23.813 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-05-09T06:37:23.814 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.814 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.814 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-09T06:37:23.814 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-09T06:37:23.814 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-09T06:37:23.814 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-09T06:37:23.814 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-09T06:37:23.814 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-09T06:37:23.814 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-09T06:37:23.814 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-09T06:37:23.814 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:23.814 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.814 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.814 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.814 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.815 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-05-09T06:37:23.815 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-05-09T06:37:23.815 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:23.815 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.815 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:23.815 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-09T06:37:23.815 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.815 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.815 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:23.815 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:23.815 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.815 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.816 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.816 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.816 INFO:teuthology.orchestra.run.smithi003.stdout: "retain_interval": { 2024-05-09T06:37:23.816 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "retain_interval", 2024-05-09T06:37:23.816 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-05-09T06:37:23.816 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.816 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:23.816 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "31536000", 2024-05-09T06:37:23.816 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.816 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.816 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:23.816 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "how long to retain crashes before pruning them", 2024-05-09T06:37:23.817 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.817 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.817 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.817 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.817 INFO:teuthology.orchestra.run.smithi003.stdout: "warn_recent_interval": { 2024-05-09T06:37:23.817 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "warn_recent_interval", 2024-05-09T06:37:23.817 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-05-09T06:37:23.817 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.817 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:23.817 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "1209600", 2024-05-09T06:37:23.817 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.817 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.817 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:23.817 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "time interval in which to warn about recent crashes", 2024-05-09T06:37:23.817 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.818 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.818 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.818 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-09T06:37:23.818 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-09T06:37:23.818 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.818 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-09T06:37:23.818 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "dashboard", 2024-05-09T06:37:23.818 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-05-09T06:37:23.818 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-05-09T06:37:23.818 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-05-09T06:37:23.818 INFO:teuthology.orchestra.run.smithi003.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-05-09T06:37:23.818 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-05-09T06:37:23.818 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-09T06:37:23.818 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.819 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:23.819 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "10", 2024-05-09T06:37:23.819 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.819 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.819 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:23.819 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:23.819 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.819 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.819 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.819 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.819 INFO:teuthology.orchestra.run.smithi003.stdout: "ALERTMANAGER_API_HOST": { 2024-05-09T06:37:23.819 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "ALERTMANAGER_API_HOST", 2024-05-09T06:37:23.819 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:23.819 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.819 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:23.820 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-09T06:37:23.820 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.820 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.820 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:23.820 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:23.820 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.820 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.820 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.820 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.820 INFO:teuthology.orchestra.run.smithi003.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-05-09T06:37:23.820 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-05-09T06:37:23.820 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:23.820 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.820 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:23.821 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-05-09T06:37:23.821 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.821 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.821 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:23.821 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:23.821 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.821 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.821 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.821 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.821 INFO:teuthology.orchestra.run.smithi003.stdout: "AUDIT_API_ENABLED": { 2024-05-09T06:37:23.821 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "AUDIT_API_ENABLED", 2024-05-09T06:37:23.821 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:23.821 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.822 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:23.822 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-09T06:37:23.822 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.822 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.822 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:23.822 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:23.822 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.822 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.822 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.822 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.822 INFO:teuthology.orchestra.run.smithi003.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-05-09T06:37:23.822 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-05-09T06:37:23.822 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:23.823 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.823 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:23.823 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-05-09T06:37:23.823 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.823 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.823 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:23.823 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:23.823 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.823 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.823 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.823 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.823 INFO:teuthology.orchestra.run.smithi003.stdout: "ENABLE_BROWSABLE_API": { 2024-05-09T06:37:23.823 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "ENABLE_BROWSABLE_API", 2024-05-09T06:37:23.823 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:23.824 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.824 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:23.824 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-05-09T06:37:23.824 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.824 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.824 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:23.824 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:23.824 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.824 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.824 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.824 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.824 INFO:teuthology.orchestra.run.smithi003.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-05-09T06:37:23.824 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-05-09T06:37:23.824 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:23.824 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.824 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:23.825 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-05-09T06:37:23.825 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.825 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.825 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:23.825 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:23.825 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.825 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.825 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.825 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.825 INFO:teuthology.orchestra.run.smithi003.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-05-09T06:37:23.825 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-05-09T06:37:23.825 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:23.825 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.825 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:23.825 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-05-09T06:37:23.825 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.825 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.826 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:23.826 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:23.826 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.826 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.826 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.826 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.826 INFO:teuthology.orchestra.run.smithi003.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-05-09T06:37:23.826 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-05-09T06:37:23.826 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:23.826 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.826 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:23.826 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-05-09T06:37:23.826 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.826 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.826 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:23.827 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:23.827 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.827 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.827 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.827 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.827 INFO:teuthology.orchestra.run.smithi003.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-05-09T06:37:23.827 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-05-09T06:37:23.827 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:23.827 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.827 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:23.827 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-05-09T06:37:23.827 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.827 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.827 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:23.827 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:23.827 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.828 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.828 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.828 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.828 INFO:teuthology.orchestra.run.smithi003.stdout: "FEATURE_TOGGLE_NFS": { 2024-05-09T06:37:23.828 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-05-09T06:37:23.828 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:23.828 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.828 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:23.828 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-05-09T06:37:23.828 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.828 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.828 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:23.828 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:23.829 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.829 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.829 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.829 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.829 INFO:teuthology.orchestra.run.smithi003.stdout: "FEATURE_TOGGLE_RBD": { 2024-05-09T06:37:23.829 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-05-09T06:37:23.829 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:23.829 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.829 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:23.829 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-05-09T06:37:23.829 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.829 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.829 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:23.829 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:23.830 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.830 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.830 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.830 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.830 INFO:teuthology.orchestra.run.smithi003.stdout: "FEATURE_TOGGLE_RGW": { 2024-05-09T06:37:23.830 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-05-09T06:37:23.830 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:23.830 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.830 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:23.830 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-05-09T06:37:23.830 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.830 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.830 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:23.830 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:23.830 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.831 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.831 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.831 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.831 INFO:teuthology.orchestra.run.smithi003.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-05-09T06:37:23.831 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-05-09T06:37:23.831 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:23.831 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.831 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:23.831 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-09T06:37:23.831 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.831 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.831 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:23.831 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:23.831 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.831 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.832 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.832 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.832 INFO:teuthology.orchestra.run.smithi003.stdout: "GRAFANA_API_PASSWORD": { 2024-05-09T06:37:23.832 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "GRAFANA_API_PASSWORD", 2024-05-09T06:37:23.832 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:23.832 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.832 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:23.832 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "admin", 2024-05-09T06:37:23.832 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.832 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.832 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:23.832 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:23.832 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.832 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.833 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.833 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.833 INFO:teuthology.orchestra.run.smithi003.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-05-09T06:37:23.833 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-05-09T06:37:23.833 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:23.833 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.833 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:23.833 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-05-09T06:37:23.833 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.833 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.833 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:23.833 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:23.833 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.833 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.833 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.833 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.834 INFO:teuthology.orchestra.run.smithi003.stdout: "GRAFANA_API_URL": { 2024-05-09T06:37:23.834 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "GRAFANA_API_URL", 2024-05-09T06:37:23.834 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:23.834 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.834 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:23.834 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-09T06:37:23.834 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.834 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.834 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:23.834 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:23.834 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.834 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.835 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.835 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.835 INFO:teuthology.orchestra.run.smithi003.stdout: "GRAFANA_API_USERNAME": { 2024-05-09T06:37:23.835 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "GRAFANA_API_USERNAME", 2024-05-09T06:37:23.835 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:23.835 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.835 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:23.835 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "admin", 2024-05-09T06:37:23.835 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.835 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.835 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:23.835 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:23.835 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.835 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.835 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.836 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.836 INFO:teuthology.orchestra.run.smithi003.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-05-09T06:37:23.836 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-05-09T06:37:23.836 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:23.836 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.836 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:23.836 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-09T06:37:23.836 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.836 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.836 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:23.836 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:23.836 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.836 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.836 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.836 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.836 INFO:teuthology.orchestra.run.smithi003.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-05-09T06:37:23.837 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-05-09T06:37:23.837 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:23.837 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.837 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:23.837 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-09T06:37:23.837 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.837 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.837 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:23.837 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:23.837 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.837 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.837 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.837 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.837 INFO:teuthology.orchestra.run.smithi003.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-05-09T06:37:23.837 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-05-09T06:37:23.838 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:23.838 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.838 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:23.838 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-05-09T06:37:23.838 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.838 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.838 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:23.838 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:23.838 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.838 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.838 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.838 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.838 INFO:teuthology.orchestra.run.smithi003.stdout: "ISSUE_TRACKER_API_KEY": { 2024-05-09T06:37:23.838 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-05-09T06:37:23.838 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:23.838 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.839 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:23.839 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-09T06:37:23.839 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.839 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.839 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:23.839 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:23.839 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.839 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.839 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.839 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.839 INFO:teuthology.orchestra.run.smithi003.stdout: "PROMETHEUS_API_HOST": { 2024-05-09T06:37:23.839 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PROMETHEUS_API_HOST", 2024-05-09T06:37:23.839 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:23.839 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.839 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:23.839 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-09T06:37:23.840 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.840 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.840 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:23.840 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:23.840 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.840 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.840 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.840 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.840 INFO:teuthology.orchestra.run.smithi003.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-05-09T06:37:23.840 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-05-09T06:37:23.840 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:23.840 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.840 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:23.840 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-05-09T06:37:23.840 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.841 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.841 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:23.841 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:23.841 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.841 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.841 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.841 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.841 INFO:teuthology.orchestra.run.smithi003.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-05-09T06:37:23.841 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-05-09T06:37:23.841 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:23.841 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.841 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:23.841 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-09T06:37:23.842 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.842 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.842 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:23.842 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:23.842 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.842 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.842 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.842 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.842 INFO:teuthology.orchestra.run.smithi003.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-05-09T06:37:23.842 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-05-09T06:37:23.842 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:23.842 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.843 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:23.843 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-09T06:37:23.843 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.843 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.843 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:23.843 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:23.843 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.843 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.843 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.843 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.843 INFO:teuthology.orchestra.run.smithi003.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-05-09T06:37:23.843 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-05-09T06:37:23.843 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:23.843 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.843 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:23.844 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-05-09T06:37:23.844 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.844 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.844 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:23.844 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:23.844 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.844 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.844 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.844 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.844 INFO:teuthology.orchestra.run.smithi003.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-05-09T06:37:23.844 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-05-09T06:37:23.844 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:23.844 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.844 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:23.844 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-05-09T06:37:23.844 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.844 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.845 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:23.845 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:23.845 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.845 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.845 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.845 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.845 INFO:teuthology.orchestra.run.smithi003.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-05-09T06:37:23.845 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-05-09T06:37:23.845 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:23.845 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.845 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:23.845 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-09T06:37:23.845 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.846 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.846 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:23.846 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:23.846 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.846 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.846 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.846 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.846 INFO:teuthology.orchestra.run.smithi003.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-05-09T06:37:23.846 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-05-09T06:37:23.846 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:23.846 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.846 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:23.846 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-09T06:37:23.846 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.846 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.847 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:23.847 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:23.847 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.847 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.847 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.847 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.847 INFO:teuthology.orchestra.run.smithi003.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-05-09T06:37:23.847 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-05-09T06:37:23.847 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:23.847 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.847 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:23.847 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-09T06:37:23.847 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.847 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.847 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:23.847 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:23.847 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.848 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.848 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.848 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.848 INFO:teuthology.orchestra.run.smithi003.stdout: "PWD_POLICY_ENABLED": { 2024-05-09T06:37:23.848 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PWD_POLICY_ENABLED", 2024-05-09T06:37:23.848 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:23.848 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.848 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:23.848 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-05-09T06:37:23.848 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.848 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.848 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:23.848 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:23.848 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.848 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.849 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.849 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.849 INFO:teuthology.orchestra.run.smithi003.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-05-09T06:37:23.849 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-05-09T06:37:23.849 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:23.849 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.849 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:23.849 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-05-09T06:37:23.849 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.849 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.850 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:23.850 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:23.850 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.850 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.850 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.850 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.850 INFO:teuthology.orchestra.run.smithi003.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-05-09T06:37:23.850 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-05-09T06:37:23.850 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-09T06:37:23.850 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.850 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:23.850 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "10", 2024-05-09T06:37:23.850 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.850 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.851 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:23.851 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:23.851 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.851 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.851 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.851 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.851 INFO:teuthology.orchestra.run.smithi003.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-05-09T06:37:23.851 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-05-09T06:37:23.851 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-09T06:37:23.851 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.851 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:23.851 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "8", 2024-05-09T06:37:23.851 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.851 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.851 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:23.852 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:23.852 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.852 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.852 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.852 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.852 INFO:teuthology.orchestra.run.smithi003.stdout: "REST_REQUESTS_TIMEOUT": { 2024-05-09T06:37:23.852 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-05-09T06:37:23.852 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-09T06:37:23.852 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.852 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:23.852 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "45", 2024-05-09T06:37:23.852 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.852 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.852 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:23.852 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:23.853 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.853 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.853 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.853 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.853 INFO:teuthology.orchestra.run.smithi003.stdout: "RGW_API_ACCESS_KEY": { 2024-05-09T06:37:23.853 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "RGW_API_ACCESS_KEY", 2024-05-09T06:37:23.853 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:23.853 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.853 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:23.853 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-09T06:37:23.853 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.853 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.853 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:23.853 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:23.854 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.854 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.854 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.854 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.854 INFO:teuthology.orchestra.run.smithi003.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-05-09T06:37:23.854 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-05-09T06:37:23.854 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:23.854 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.854 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:23.854 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "admin", 2024-05-09T06:37:23.854 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.854 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.854 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:23.854 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:23.854 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.855 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.855 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.855 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.855 INFO:teuthology.orchestra.run.smithi003.stdout: "RGW_API_SECRET_KEY": { 2024-05-09T06:37:23.855 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "RGW_API_SECRET_KEY", 2024-05-09T06:37:23.855 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:23.855 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.855 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:23.855 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-09T06:37:23.855 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.855 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.855 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:23.855 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:23.855 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.855 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.856 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.856 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.856 INFO:teuthology.orchestra.run.smithi003.stdout: "RGW_API_SSL_VERIFY": { 2024-05-09T06:37:23.856 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "RGW_API_SSL_VERIFY", 2024-05-09T06:37:23.856 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:23.856 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.856 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:23.856 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-05-09T06:37:23.856 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.856 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.856 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:23.856 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:23.856 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.856 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.856 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.857 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.857 INFO:teuthology.orchestra.run.smithi003.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-05-09T06:37:23.857 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-05-09T06:37:23.857 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-09T06:37:23.857 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.857 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:23.857 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-05-09T06:37:23.857 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.857 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.857 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:23.857 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:23.857 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.857 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.857 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.857 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.858 INFO:teuthology.orchestra.run.smithi003.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-05-09T06:37:23.858 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-05-09T06:37:23.858 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-09T06:37:23.858 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.858 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:23.858 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "10", 2024-05-09T06:37:23.858 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.858 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.858 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:23.858 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:23.858 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.858 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.859 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.859 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.859 INFO:teuthology.orchestra.run.smithi003.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-05-09T06:37:23.859 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-05-09T06:37:23.859 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-09T06:37:23.859 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.859 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:23.859 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "5", 2024-05-09T06:37:23.859 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.859 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.859 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:23.859 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:23.859 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.860 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.860 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.860 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.860 INFO:teuthology.orchestra.run.smithi003.stdout: "cross_origin_url": { 2024-05-09T06:37:23.860 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "cross_origin_url", 2024-05-09T06:37:23.860 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:23.860 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.860 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:23.860 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-09T06:37:23.860 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.860 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.861 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:23.861 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:23.861 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.861 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.861 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.861 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.861 INFO:teuthology.orchestra.run.smithi003.stdout: "crt_file": { 2024-05-09T06:37:23.861 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "crt_file", 2024-05-09T06:37:23.861 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:23.861 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.861 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:23.861 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-09T06:37:23.862 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.862 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.862 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:23.862 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:23.862 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.862 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.862 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.862 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.862 INFO:teuthology.orchestra.run.smithi003.stdout: "debug": { 2024-05-09T06:37:23.862 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "debug", 2024-05-09T06:37:23.863 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:23.863 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.863 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:23.863 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-09T06:37:23.863 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.863 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.863 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:23.863 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Enable/disable debug options", 2024-05-09T06:37:23.863 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.863 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.863 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.863 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.863 INFO:teuthology.orchestra.run.smithi003.stdout: "jwt_token_ttl": { 2024-05-09T06:37:23.864 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "jwt_token_ttl", 2024-05-09T06:37:23.864 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-09T06:37:23.864 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.864 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:23.864 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "28800", 2024-05-09T06:37:23.864 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.864 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.864 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:23.864 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:23.864 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.864 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.864 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.864 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.864 INFO:teuthology.orchestra.run.smithi003.stdout: "key_file": { 2024-05-09T06:37:23.865 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "key_file", 2024-05-09T06:37:23.865 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:23.865 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.865 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:23.865 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-09T06:37:23.865 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.865 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.865 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:23.865 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:23.865 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.865 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.865 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.865 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.865 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-05-09T06:37:23.865 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-05-09T06:37:23.866 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:23.866 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.866 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:23.866 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-09T06:37:23.866 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.866 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.866 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-09T06:37:23.866 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-09T06:37:23.866 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-09T06:37:23.866 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-09T06:37:23.866 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-09T06:37:23.866 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-09T06:37:23.866 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-09T06:37:23.867 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-09T06:37:23.867 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:23.867 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.867 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.867 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.867 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.867 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-05-09T06:37:23.867 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-05-09T06:37:23.867 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:23.867 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.867 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:23.867 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-09T06:37:23.867 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.867 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.868 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:23.868 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:23.868 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.868 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.868 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.868 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.868 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-05-09T06:37:23.868 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-05-09T06:37:23.868 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:23.868 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.868 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:23.868 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-05-09T06:37:23.868 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.869 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.869 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-09T06:37:23.869 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-09T06:37:23.869 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-09T06:37:23.869 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-09T06:37:23.869 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-09T06:37:23.869 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-09T06:37:23.869 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-09T06:37:23.869 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-09T06:37:23.869 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:23.869 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.869 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.869 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.869 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.870 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-05-09T06:37:23.870 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-05-09T06:37:23.870 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:23.870 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.870 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:23.870 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-09T06:37:23.870 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.870 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.870 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:23.870 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:23.870 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.870 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.870 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.871 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.871 INFO:teuthology.orchestra.run.smithi003.stdout: "motd": { 2024-05-09T06:37:23.871 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "motd", 2024-05-09T06:37:23.871 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:23.871 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.871 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:23.871 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-09T06:37:23.871 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.871 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.871 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:23.871 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "The message of the day", 2024-05-09T06:37:23.871 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.871 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.871 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.872 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.872 INFO:teuthology.orchestra.run.smithi003.stdout: "redirect_resolve_ip_addr": { 2024-05-09T06:37:23.872 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "redirect_resolve_ip_addr", 2024-05-09T06:37:23.872 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:23.872 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.872 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:23.872 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-09T06:37:23.872 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.872 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.872 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:23.872 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:23.872 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.872 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.872 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.872 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.872 INFO:teuthology.orchestra.run.smithi003.stdout: "server_addr": { 2024-05-09T06:37:23.873 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "server_addr", 2024-05-09T06:37:23.873 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:23.873 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.873 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:23.873 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "::", 2024-05-09T06:37:23.873 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.873 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.873 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:23.873 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:23.873 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.873 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.873 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.873 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.873 INFO:teuthology.orchestra.run.smithi003.stdout: "server_port": { 2024-05-09T06:37:23.874 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "server_port", 2024-05-09T06:37:23.874 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-09T06:37:23.874 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.874 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:23.874 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "8080", 2024-05-09T06:37:23.874 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.874 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.874 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:23.874 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:23.874 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.874 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.874 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.874 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.874 INFO:teuthology.orchestra.run.smithi003.stdout: "ssl": { 2024-05-09T06:37:23.874 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "ssl", 2024-05-09T06:37:23.875 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:23.875 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.875 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:23.875 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-05-09T06:37:23.875 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.875 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.875 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:23.875 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:23.875 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.875 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.875 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.875 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.875 INFO:teuthology.orchestra.run.smithi003.stdout: "ssl_server_port": { 2024-05-09T06:37:23.876 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "ssl_server_port", 2024-05-09T06:37:23.876 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-09T06:37:23.876 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.876 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:23.876 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "8443", 2024-05-09T06:37:23.876 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.876 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.876 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:23.876 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:23.876 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.876 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.876 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.876 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.877 INFO:teuthology.orchestra.run.smithi003.stdout: "standby_behaviour": { 2024-05-09T06:37:23.877 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "standby_behaviour", 2024-05-09T06:37:23.877 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:23.877 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.877 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:23.877 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "redirect", 2024-05-09T06:37:23.877 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.877 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.877 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-09T06:37:23.877 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-09T06:37:23.877 INFO:teuthology.orchestra.run.smithi003.stdout: "redirect" 2024-05-09T06:37:23.877 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-09T06:37:23.877 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:23.878 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.878 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.878 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.878 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.878 INFO:teuthology.orchestra.run.smithi003.stdout: "standby_error_status_code": { 2024-05-09T06:37:23.878 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "standby_error_status_code", 2024-05-09T06:37:23.878 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-09T06:37:23.878 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.878 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:23.878 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "500", 2024-05-09T06:37:23.878 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "400", 2024-05-09T06:37:23.878 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "599", 2024-05-09T06:37:23.878 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:23.878 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:23.879 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.879 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.879 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.879 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.879 INFO:teuthology.orchestra.run.smithi003.stdout: "url_prefix": { 2024-05-09T06:37:23.879 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "url_prefix", 2024-05-09T06:37:23.879 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:23.879 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.879 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:23.879 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-09T06:37:23.879 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.879 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.879 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:23.879 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:23.879 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.879 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.880 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.880 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-09T06:37:23.880 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-09T06:37:23.880 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.880 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-09T06:37:23.880 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "devicehealth", 2024-05-09T06:37:23.880 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-05-09T06:37:23.880 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-05-09T06:37:23.880 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-05-09T06:37:23.880 INFO:teuthology.orchestra.run.smithi003.stdout: "enable_monitoring": { 2024-05-09T06:37:23.880 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "enable_monitoring", 2024-05-09T06:37:23.880 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:23.880 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.880 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:23.880 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-05-09T06:37:23.881 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.881 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.881 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:23.881 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "monitor device health metrics", 2024-05-09T06:37:23.881 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.881 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.881 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.881 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.881 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-05-09T06:37:23.881 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-05-09T06:37:23.881 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:23.881 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.881 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:23.881 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-09T06:37:23.881 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.881 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.882 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-09T06:37:23.882 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-09T06:37:23.882 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-09T06:37:23.882 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-09T06:37:23.882 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-09T06:37:23.882 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-09T06:37:23.882 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-09T06:37:23.882 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-09T06:37:23.882 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:23.882 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.882 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.882 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.882 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.882 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-05-09T06:37:23.883 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-05-09T06:37:23.883 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:23.883 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.883 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:23.883 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-09T06:37:23.883 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.883 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.883 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:23.883 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:23.883 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.883 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.883 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.883 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.883 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-05-09T06:37:23.883 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-05-09T06:37:23.884 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:23.884 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.884 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:23.884 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-05-09T06:37:23.884 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.884 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.884 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-09T06:37:23.884 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-09T06:37:23.884 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-09T06:37:23.884 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-09T06:37:23.884 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-09T06:37:23.884 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-09T06:37:23.884 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-09T06:37:23.884 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-09T06:37:23.885 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:23.885 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.885 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.885 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.885 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.885 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-05-09T06:37:23.885 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-05-09T06:37:23.885 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:23.885 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.885 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:23.885 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-09T06:37:23.885 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.885 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.885 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:23.885 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:23.885 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.886 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.886 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.886 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.886 INFO:teuthology.orchestra.run.smithi003.stdout: "mark_out_threshold": { 2024-05-09T06:37:23.886 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "mark_out_threshold", 2024-05-09T06:37:23.886 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-05-09T06:37:23.886 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.886 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:23.886 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "2419200", 2024-05-09T06:37:23.886 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.886 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.886 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:23.886 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-05-09T06:37:23.886 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.886 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.887 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.887 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.887 INFO:teuthology.orchestra.run.smithi003.stdout: "pool_name": { 2024-05-09T06:37:23.887 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "pool_name", 2024-05-09T06:37:23.887 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:23.887 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.887 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:23.887 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "device_health_metrics", 2024-05-09T06:37:23.887 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.887 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.887 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:23.887 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "name of pool in which to store device health metrics", 2024-05-09T06:37:23.887 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.887 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.887 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.888 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.888 INFO:teuthology.orchestra.run.smithi003.stdout: "retention_period": { 2024-05-09T06:37:23.888 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "retention_period", 2024-05-09T06:37:23.888 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-05-09T06:37:23.888 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.888 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:23.888 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "15552000", 2024-05-09T06:37:23.888 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.888 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.888 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:23.888 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "how long to retain device health metrics", 2024-05-09T06:37:23.888 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.888 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.888 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.889 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.889 INFO:teuthology.orchestra.run.smithi003.stdout: "scrape_frequency": { 2024-05-09T06:37:23.889 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "scrape_frequency", 2024-05-09T06:37:23.889 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-05-09T06:37:23.889 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.889 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:23.889 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "86400", 2024-05-09T06:37:23.889 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.889 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.889 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:23.889 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "how frequently to scrape device health metrics", 2024-05-09T06:37:23.889 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.889 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.889 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.889 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.890 INFO:teuthology.orchestra.run.smithi003.stdout: "self_heal": { 2024-05-09T06:37:23.890 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "self_heal", 2024-05-09T06:37:23.890 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:23.890 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.890 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:23.890 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-05-09T06:37:23.890 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.890 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.890 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:23.890 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-05-09T06:37:23.890 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.891 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.891 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.891 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.891 INFO:teuthology.orchestra.run.smithi003.stdout: "sleep_interval": { 2024-05-09T06:37:23.891 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sleep_interval", 2024-05-09T06:37:23.891 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-05-09T06:37:23.891 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.891 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:23.891 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "600", 2024-05-09T06:37:23.891 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.891 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.891 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:23.891 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "how frequently to wake up and check device health", 2024-05-09T06:37:23.891 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.891 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.892 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.892 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.892 INFO:teuthology.orchestra.run.smithi003.stdout: "warn_threshold": { 2024-05-09T06:37:23.892 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "warn_threshold", 2024-05-09T06:37:23.892 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-05-09T06:37:23.892 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.892 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:23.892 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "7257600", 2024-05-09T06:37:23.892 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.892 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.892 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:23.893 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-05-09T06:37:23.893 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.893 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.893 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.893 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-09T06:37:23.893 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-09T06:37:23.893 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.893 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-09T06:37:23.893 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "diskprediction_local", 2024-05-09T06:37:23.893 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-05-09T06:37:23.893 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-05-09T06:37:23.893 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-05-09T06:37:23.893 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-05-09T06:37:23.893 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-05-09T06:37:23.894 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:23.894 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.894 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:23.894 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-09T06:37:23.894 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.894 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.894 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-09T06:37:23.894 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-09T06:37:23.894 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-09T06:37:23.894 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-09T06:37:23.894 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-09T06:37:23.894 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-09T06:37:23.895 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-09T06:37:23.895 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-09T06:37:23.895 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:23.895 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.895 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.895 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.895 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.895 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-05-09T06:37:23.895 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-05-09T06:37:23.895 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:23.895 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.895 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:23.895 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-09T06:37:23.895 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.896 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.896 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:23.896 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:23.896 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.896 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.896 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.896 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.896 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-05-09T06:37:23.896 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-05-09T06:37:23.896 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:23.896 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.896 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:23.896 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-05-09T06:37:23.897 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.897 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.897 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-09T06:37:23.897 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-09T06:37:23.897 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-09T06:37:23.897 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-09T06:37:23.897 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-09T06:37:23.897 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-09T06:37:23.897 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-09T06:37:23.897 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-09T06:37:23.897 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:23.897 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.897 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.898 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.898 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.898 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-05-09T06:37:23.898 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-05-09T06:37:23.898 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:23.898 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.898 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:23.898 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-09T06:37:23.898 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.898 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.898 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:23.898 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:23.899 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.899 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.899 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.899 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.899 INFO:teuthology.orchestra.run.smithi003.stdout: "predict_interval": { 2024-05-09T06:37:23.899 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "predict_interval", 2024-05-09T06:37:23.899 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:23.899 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.899 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:23.899 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "86400", 2024-05-09T06:37:23.899 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.899 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.899 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:23.899 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:23.899 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.900 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.900 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.900 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.900 INFO:teuthology.orchestra.run.smithi003.stdout: "predictor_model": { 2024-05-09T06:37:23.900 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "predictor_model", 2024-05-09T06:37:23.900 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:23.900 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.900 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:23.900 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "prophetstor", 2024-05-09T06:37:23.900 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.900 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.900 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:23.901 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:23.901 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.901 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.901 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.901 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.901 INFO:teuthology.orchestra.run.smithi003.stdout: "sleep_interval": { 2024-05-09T06:37:23.901 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sleep_interval", 2024-05-09T06:37:23.901 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:23.901 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.901 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:23.901 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "600", 2024-05-09T06:37:23.901 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.901 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.901 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:23.902 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:23.902 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.902 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.902 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.902 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-09T06:37:23.902 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-09T06:37:23.902 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.902 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-09T06:37:23.902 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "influx", 2024-05-09T06:37:23.902 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": false, 2024-05-09T06:37:23.902 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "influxdb python module not found", 2024-05-09T06:37:23.902 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-05-09T06:37:23.902 INFO:teuthology.orchestra.run.smithi003.stdout: "batch_size": { 2024-05-09T06:37:23.902 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "batch_size", 2024-05-09T06:37:23.902 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-09T06:37:23.903 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.903 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:23.903 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "5000", 2024-05-09T06:37:23.903 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.903 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.903 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:23.903 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-05-09T06:37:23.903 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.903 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.903 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.903 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.903 INFO:teuthology.orchestra.run.smithi003.stdout: "database": { 2024-05-09T06:37:23.903 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "database", 2024-05-09T06:37:23.903 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:23.904 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.904 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:23.904 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "ceph", 2024-05-09T06:37:23.904 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.904 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.904 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:23.904 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-05-09T06:37:23.904 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.904 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.904 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.904 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.904 INFO:teuthology.orchestra.run.smithi003.stdout: "hostname": { 2024-05-09T06:37:23.904 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "hostname", 2024-05-09T06:37:23.904 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:23.904 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.905 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:23.905 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-09T06:37:23.905 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.905 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.905 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:23.905 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "InfluxDB server hostname", 2024-05-09T06:37:23.905 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.905 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.905 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.905 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.905 INFO:teuthology.orchestra.run.smithi003.stdout: "interval": { 2024-05-09T06:37:23.905 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "interval", 2024-05-09T06:37:23.905 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-05-09T06:37:23.905 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.905 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:23.905 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "30", 2024-05-09T06:37:23.906 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "5", 2024-05-09T06:37:23.906 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.906 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:23.906 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-05-09T06:37:23.906 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.906 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.906 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.906 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.906 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-05-09T06:37:23.906 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-05-09T06:37:23.906 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:23.906 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.906 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:23.906 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-09T06:37:23.906 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.907 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.907 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-09T06:37:23.907 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-09T06:37:23.907 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-09T06:37:23.907 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-09T06:37:23.907 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-09T06:37:23.907 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-09T06:37:23.907 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-09T06:37:23.907 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-09T06:37:23.907 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:23.907 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.907 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.907 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.907 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.907 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-05-09T06:37:23.907 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-05-09T06:37:23.908 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:23.908 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.908 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:23.908 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-09T06:37:23.908 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.908 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.908 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:23.908 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:23.908 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.908 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.908 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.908 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.908 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-05-09T06:37:23.908 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-05-09T06:37:23.909 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:23.909 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.909 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:23.909 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-05-09T06:37:23.909 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.909 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.909 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-09T06:37:23.909 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-09T06:37:23.909 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-09T06:37:23.909 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-09T06:37:23.909 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-09T06:37:23.909 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-09T06:37:23.909 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-09T06:37:23.909 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-09T06:37:23.909 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:23.910 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.910 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.910 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.910 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.910 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-05-09T06:37:23.910 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-05-09T06:37:23.910 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:23.910 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.910 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:23.910 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-09T06:37:23.910 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.910 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.910 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:23.910 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:23.911 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.911 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.911 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.911 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.911 INFO:teuthology.orchestra.run.smithi003.stdout: "password": { 2024-05-09T06:37:23.911 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "password", 2024-05-09T06:37:23.911 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:23.911 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.911 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:23.911 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-09T06:37:23.911 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.911 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.911 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:23.911 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "password of InfluxDB server user", 2024-05-09T06:37:23.911 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.911 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.912 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.912 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.912 INFO:teuthology.orchestra.run.smithi003.stdout: "port": { 2024-05-09T06:37:23.912 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "port", 2024-05-09T06:37:23.912 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-09T06:37:23.912 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.912 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:23.912 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "8086", 2024-05-09T06:37:23.912 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.912 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.912 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:23.912 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "InfluxDB server port", 2024-05-09T06:37:23.912 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.912 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.912 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.913 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.913 INFO:teuthology.orchestra.run.smithi003.stdout: "ssl": { 2024-05-09T06:37:23.913 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "ssl", 2024-05-09T06:37:23.913 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:23.913 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.913 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:23.913 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "false", 2024-05-09T06:37:23.913 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.913 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.913 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:23.913 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-05-09T06:37:23.913 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.913 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.913 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.913 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.913 INFO:teuthology.orchestra.run.smithi003.stdout: "threads": { 2024-05-09T06:37:23.914 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "threads", 2024-05-09T06:37:23.914 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-09T06:37:23.914 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.914 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:23.914 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "5", 2024-05-09T06:37:23.914 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "1", 2024-05-09T06:37:23.914 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "32", 2024-05-09T06:37:23.914 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:23.914 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-05-09T06:37:23.914 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.914 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.914 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.914 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.914 INFO:teuthology.orchestra.run.smithi003.stdout: "username": { 2024-05-09T06:37:23.914 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "username", 2024-05-09T06:37:23.915 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:23.915 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.915 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:23.915 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-09T06:37:23.915 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.915 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.915 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:23.915 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "username of InfluxDB server user", 2024-05-09T06:37:23.915 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.915 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.915 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.915 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.915 INFO:teuthology.orchestra.run.smithi003.stdout: "verify_ssl": { 2024-05-09T06:37:23.915 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "verify_ssl", 2024-05-09T06:37:23.916 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:23.916 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.916 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:23.916 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "true", 2024-05-09T06:37:23.916 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.916 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.916 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:23.916 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-05-09T06:37:23.916 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.916 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.916 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.916 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-09T06:37:23.917 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-09T06:37:23.917 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.917 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-09T06:37:23.917 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "insights", 2024-05-09T06:37:23.917 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-05-09T06:37:23.917 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-05-09T06:37:23.917 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-05-09T06:37:23.917 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-05-09T06:37:23.917 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-05-09T06:37:23.917 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:23.917 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.917 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:23.917 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-09T06:37:23.917 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.917 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.918 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-09T06:37:23.918 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-09T06:37:23.918 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-09T06:37:23.918 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-09T06:37:23.918 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-09T06:37:23.918 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-09T06:37:23.918 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-09T06:37:23.918 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-09T06:37:23.918 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:23.918 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.918 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.918 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.919 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.919 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-05-09T06:37:23.919 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-05-09T06:37:23.919 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:23.919 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.919 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:23.919 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-09T06:37:23.919 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.919 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.919 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:23.919 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:23.919 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.919 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.920 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.920 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.920 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-05-09T06:37:23.920 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-05-09T06:37:23.920 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:23.920 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.920 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:23.920 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-05-09T06:37:23.920 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.920 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.920 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-09T06:37:23.920 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-09T06:37:23.920 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-09T06:37:23.920 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-09T06:37:23.920 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-09T06:37:23.921 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-09T06:37:23.921 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-09T06:37:23.921 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-09T06:37:23.921 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:23.921 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.921 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.921 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.921 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.921 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-05-09T06:37:23.921 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-05-09T06:37:23.921 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:23.921 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.921 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:23.921 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-09T06:37:23.922 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.922 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.922 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:23.922 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:23.922 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.922 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.922 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.922 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-09T06:37:23.922 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-09T06:37:23.922 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.922 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-09T06:37:23.922 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "iostat", 2024-05-09T06:37:23.922 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-05-09T06:37:23.922 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-05-09T06:37:23.923 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-05-09T06:37:23.923 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-05-09T06:37:23.923 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-05-09T06:37:23.923 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:23.923 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.923 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:23.923 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-09T06:37:23.923 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.923 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.923 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-09T06:37:23.923 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-09T06:37:23.923 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-09T06:37:23.923 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-09T06:37:23.923 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-09T06:37:23.923 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-09T06:37:23.924 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-09T06:37:23.924 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-09T06:37:23.924 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:23.924 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.924 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.924 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.924 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.924 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-05-09T06:37:23.924 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-05-09T06:37:23.924 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:23.924 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.924 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:23.924 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-09T06:37:23.924 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.924 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.925 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:23.925 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:23.925 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.925 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.925 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.925 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.925 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-05-09T06:37:23.925 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-05-09T06:37:23.925 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:23.925 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.925 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:23.925 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-05-09T06:37:23.925 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.926 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.926 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-09T06:37:23.926 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-09T06:37:23.926 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-09T06:37:23.926 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-09T06:37:23.926 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-09T06:37:23.926 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-09T06:37:23.926 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-09T06:37:23.926 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-09T06:37:23.926 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:23.926 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.926 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.926 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.926 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.927 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-05-09T06:37:23.927 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-05-09T06:37:23.927 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:23.927 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.927 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:23.927 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-09T06:37:23.927 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.927 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.927 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:23.927 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:23.927 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.927 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.927 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.927 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-09T06:37:23.928 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-09T06:37:23.928 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.928 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-09T06:37:23.928 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "localpool", 2024-05-09T06:37:23.928 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-05-09T06:37:23.928 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-05-09T06:37:23.928 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-05-09T06:37:23.928 INFO:teuthology.orchestra.run.smithi003.stdout: "failure_domain": { 2024-05-09T06:37:23.928 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "failure_domain", 2024-05-09T06:37:23.928 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:23.928 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.928 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:23.928 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "host", 2024-05-09T06:37:23.928 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.928 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.929 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:23.929 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "failure domain for any created local pool", 2024-05-09T06:37:23.929 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-05-09T06:37:23.929 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.929 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.929 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.929 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-05-09T06:37:23.929 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-05-09T06:37:23.929 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:23.929 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.929 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:23.929 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-09T06:37:23.929 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.929 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.929 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-09T06:37:23.930 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-09T06:37:23.930 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-09T06:37:23.930 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-09T06:37:23.930 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-09T06:37:23.930 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-09T06:37:23.930 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-09T06:37:23.930 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-09T06:37:23.930 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:23.930 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.930 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.930 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.930 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.931 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-05-09T06:37:23.931 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-05-09T06:37:23.931 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:23.931 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.931 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:23.931 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-09T06:37:23.931 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.931 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.931 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:23.931 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:23.931 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.931 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.931 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.932 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.932 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-05-09T06:37:23.932 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-05-09T06:37:23.932 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:23.932 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.932 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:23.932 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-05-09T06:37:23.932 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.932 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.932 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-09T06:37:23.932 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-09T06:37:23.932 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-09T06:37:23.932 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-09T06:37:23.932 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-09T06:37:23.932 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-09T06:37:23.933 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-09T06:37:23.933 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-09T06:37:23.933 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:23.933 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.933 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.933 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.933 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.933 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-05-09T06:37:23.933 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-05-09T06:37:23.933 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:23.933 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.933 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:23.933 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-09T06:37:23.933 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.933 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.934 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:23.934 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:23.934 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.934 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.934 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.934 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.934 INFO:teuthology.orchestra.run.smithi003.stdout: "min_size": { 2024-05-09T06:37:23.934 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "min_size", 2024-05-09T06:37:23.934 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-09T06:37:23.934 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.934 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:23.934 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-09T06:37:23.934 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.934 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.935 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:23.935 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "default min_size for any created local pool", 2024-05-09T06:37:23.935 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-05-09T06:37:23.935 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.935 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.935 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.935 INFO:teuthology.orchestra.run.smithi003.stdout: "num_rep": { 2024-05-09T06:37:23.935 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "num_rep", 2024-05-09T06:37:23.935 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-09T06:37:23.935 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.935 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:23.935 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "3", 2024-05-09T06:37:23.935 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.935 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.935 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:23.935 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "default replica count for any created local pool", 2024-05-09T06:37:23.936 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.936 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.936 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.936 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.936 INFO:teuthology.orchestra.run.smithi003.stdout: "pg_num": { 2024-05-09T06:37:23.936 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "pg_num", 2024-05-09T06:37:23.936 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-09T06:37:23.936 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.936 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:23.936 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "128", 2024-05-09T06:37:23.936 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.936 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.937 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:23.937 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "default pg_num for any created local pool", 2024-05-09T06:37:23.937 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.937 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.937 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.937 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.937 INFO:teuthology.orchestra.run.smithi003.stdout: "prefix": { 2024-05-09T06:37:23.937 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "prefix", 2024-05-09T06:37:23.937 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:23.937 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.937 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:23.937 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-09T06:37:23.938 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.938 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.938 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:23.938 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "name prefix for any created local pool", 2024-05-09T06:37:23.938 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.938 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.938 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.938 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.938 INFO:teuthology.orchestra.run.smithi003.stdout: "subtree": { 2024-05-09T06:37:23.938 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "subtree", 2024-05-09T06:37:23.938 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:23.938 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.939 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:23.939 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "rack", 2024-05-09T06:37:23.939 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.939 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.939 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:23.939 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "CRUSH level for which to create a local pool", 2024-05-09T06:37:23.939 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-05-09T06:37:23.939 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.939 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.939 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-09T06:37:23.939 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-09T06:37:23.939 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.939 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-09T06:37:23.939 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "mds_autoscaler", 2024-05-09T06:37:23.940 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-05-09T06:37:23.940 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-05-09T06:37:23.940 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-05-09T06:37:23.940 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-05-09T06:37:23.940 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-05-09T06:37:23.940 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:23.940 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.940 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:23.940 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-09T06:37:23.940 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.940 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.940 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-09T06:37:23.940 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-09T06:37:23.940 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-09T06:37:23.940 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-09T06:37:23.941 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-09T06:37:23.941 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-09T06:37:23.941 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-09T06:37:23.941 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-09T06:37:23.941 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:23.941 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.941 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.941 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.941 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.941 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-05-09T06:37:23.941 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-05-09T06:37:23.941 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:23.942 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.942 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:23.942 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-09T06:37:23.942 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.942 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.942 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:23.942 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:23.942 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.942 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.942 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.942 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.942 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-05-09T06:37:23.942 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-05-09T06:37:23.943 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:23.943 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.943 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:23.943 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-05-09T06:37:23.943 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.943 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.943 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-09T06:37:23.943 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-09T06:37:23.943 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-09T06:37:23.943 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-09T06:37:23.943 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-09T06:37:23.943 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-09T06:37:23.943 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-09T06:37:23.943 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-09T06:37:23.944 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:23.944 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.944 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.944 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.944 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.944 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-05-09T06:37:23.944 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-05-09T06:37:23.944 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:23.944 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.944 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:23.944 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-09T06:37:23.944 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.944 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.944 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:23.944 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:23.945 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.945 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.945 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.945 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-09T06:37:23.945 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-09T06:37:23.945 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.945 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-09T06:37:23.945 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "mirroring", 2024-05-09T06:37:23.945 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-05-09T06:37:23.945 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-05-09T06:37:23.945 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-05-09T06:37:23.945 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-05-09T06:37:23.945 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-05-09T06:37:23.945 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:23.945 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.946 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:23.946 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-09T06:37:23.946 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.946 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.946 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-09T06:37:23.946 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-09T06:37:23.946 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-09T06:37:23.946 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-09T06:37:23.946 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-09T06:37:23.946 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-09T06:37:23.946 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-09T06:37:23.946 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-09T06:37:23.946 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:23.946 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.947 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.947 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.947 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.947 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-05-09T06:37:23.947 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-05-09T06:37:23.947 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:23.947 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.947 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:23.947 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-09T06:37:23.947 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.947 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.947 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:23.947 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:23.947 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.947 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.948 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.948 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.948 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-05-09T06:37:23.948 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-05-09T06:37:23.948 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:23.948 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.948 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:23.948 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-05-09T06:37:23.948 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.948 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.948 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-09T06:37:23.948 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-09T06:37:23.948 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-09T06:37:23.948 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-09T06:37:23.948 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-09T06:37:23.949 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-09T06:37:23.949 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-09T06:37:23.949 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-09T06:37:23.949 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:23.949 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.949 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.949 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.949 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.949 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-05-09T06:37:23.949 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-05-09T06:37:23.949 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:23.949 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.949 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:23.949 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-09T06:37:23.950 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.950 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.950 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:23.950 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:23.950 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.950 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.950 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.950 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-09T06:37:23.950 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-09T06:37:23.950 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.950 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-09T06:37:23.950 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "nfs", 2024-05-09T06:37:23.950 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-05-09T06:37:23.951 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-05-09T06:37:23.951 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-05-09T06:37:23.951 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-05-09T06:37:23.951 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-05-09T06:37:23.951 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:23.951 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.951 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:23.951 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-09T06:37:23.951 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.951 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.951 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-09T06:37:23.951 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-09T06:37:23.951 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-09T06:37:23.951 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-09T06:37:23.951 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-09T06:37:23.952 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-09T06:37:23.952 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-09T06:37:23.952 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-09T06:37:23.952 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:23.952 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.952 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.952 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.952 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.952 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-05-09T06:37:23.952 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-05-09T06:37:23.952 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:23.952 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.952 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:23.952 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-09T06:37:23.953 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.953 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.953 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:23.953 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:23.953 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.953 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.953 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.953 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.953 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-05-09T06:37:23.953 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-05-09T06:37:23.953 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:23.953 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.953 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:23.954 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-05-09T06:37:23.954 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.954 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.954 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-09T06:37:23.954 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-09T06:37:23.954 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-09T06:37:23.954 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-09T06:37:23.954 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-09T06:37:23.954 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-09T06:37:23.954 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-09T06:37:23.954 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-09T06:37:23.954 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:23.954 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.955 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.955 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.955 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.955 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-05-09T06:37:23.955 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-05-09T06:37:23.955 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:23.955 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.955 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:23.955 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-09T06:37:23.955 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.955 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.955 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:23.955 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:23.955 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.955 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.955 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.956 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-09T06:37:23.956 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-09T06:37:23.956 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.956 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-09T06:37:23.956 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "orchestrator", 2024-05-09T06:37:23.956 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-05-09T06:37:23.956 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-05-09T06:37:23.956 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-05-09T06:37:23.956 INFO:teuthology.orchestra.run.smithi003.stdout: "fail_fs": { 2024-05-09T06:37:23.956 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "fail_fs", 2024-05-09T06:37:23.956 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:23.956 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.956 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:23.956 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-09T06:37:23.957 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.957 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.957 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:23.957 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-05-09T06:37:23.957 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.957 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.957 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.957 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.957 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-05-09T06:37:23.957 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-05-09T06:37:23.957 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:23.957 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.957 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:23.957 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-09T06:37:23.957 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.958 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.958 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-09T06:37:23.958 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-09T06:37:23.958 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-09T06:37:23.958 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-09T06:37:23.958 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-09T06:37:23.958 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-09T06:37:23.958 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-09T06:37:23.958 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-09T06:37:23.958 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:23.958 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.958 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.958 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.959 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.959 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-05-09T06:37:23.959 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-05-09T06:37:23.959 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:23.959 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.959 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:23.959 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-09T06:37:23.959 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.959 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.959 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:23.960 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:23.960 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.960 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.960 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.960 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.960 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-05-09T06:37:23.960 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-05-09T06:37:23.960 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:23.960 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.960 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:23.961 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-05-09T06:37:23.961 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.961 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.961 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-09T06:37:23.961 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-09T06:37:23.961 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-09T06:37:23.961 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-09T06:37:23.961 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-09T06:37:23.961 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-09T06:37:23.961 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-09T06:37:23.961 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-09T06:37:23.961 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:23.961 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.962 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.962 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.962 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.962 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-05-09T06:37:23.962 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-05-09T06:37:23.962 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:23.962 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.962 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:23.962 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-09T06:37:23.962 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.962 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.962 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:23.962 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:23.963 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.963 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.963 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.963 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.963 INFO:teuthology.orchestra.run.smithi003.stdout: "orchestrator": { 2024-05-09T06:37:23.963 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "orchestrator", 2024-05-09T06:37:23.963 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:23.963 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.963 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:23.963 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-09T06:37:23.963 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.963 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.963 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-09T06:37:23.964 INFO:teuthology.orchestra.run.smithi003.stdout: "cephadm", 2024-05-09T06:37:23.964 INFO:teuthology.orchestra.run.smithi003.stdout: "rook", 2024-05-09T06:37:23.964 INFO:teuthology.orchestra.run.smithi003.stdout: "test_orchestrator" 2024-05-09T06:37:23.964 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-09T06:37:23.964 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Orchestrator backend", 2024-05-09T06:37:23.964 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.964 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.964 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.964 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-09T06:37:23.964 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-09T06:37:23.964 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.964 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-09T06:37:23.964 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "osd_perf_query", 2024-05-09T06:37:23.964 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-05-09T06:37:23.964 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-05-09T06:37:23.965 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-05-09T06:37:23.965 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-05-09T06:37:23.965 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-05-09T06:37:23.965 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:23.965 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.965 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:23.965 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-09T06:37:23.965 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.965 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.965 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-09T06:37:23.965 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-09T06:37:23.965 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-09T06:37:23.965 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-09T06:37:23.965 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-09T06:37:23.966 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-09T06:37:23.966 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-09T06:37:23.966 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-09T06:37:23.966 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:23.966 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.966 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.966 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.966 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.966 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-05-09T06:37:23.966 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-05-09T06:37:23.966 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:23.966 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.966 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:23.966 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-09T06:37:23.967 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.967 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.967 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:23.967 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:23.967 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.967 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.967 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.967 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.967 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-05-09T06:37:23.967 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-05-09T06:37:23.967 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:23.967 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.967 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:23.967 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-05-09T06:37:23.968 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.968 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.968 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-09T06:37:23.968 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-09T06:37:23.968 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-09T06:37:23.968 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-09T06:37:23.968 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-09T06:37:23.968 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-09T06:37:23.968 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-09T06:37:23.968 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-09T06:37:23.968 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:23.968 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.968 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.969 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.969 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.969 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-05-09T06:37:23.969 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-05-09T06:37:23.969 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:23.969 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.969 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:23.969 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-09T06:37:23.969 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.969 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.969 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:23.969 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:23.969 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.969 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.969 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.969 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-09T06:37:23.969 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-09T06:37:23.970 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.970 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-09T06:37:23.970 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "osd_support", 2024-05-09T06:37:23.970 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-05-09T06:37:23.970 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-05-09T06:37:23.970 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-05-09T06:37:23.970 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-05-09T06:37:23.970 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-05-09T06:37:23.970 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:23.970 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.970 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:23.970 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-09T06:37:23.970 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.970 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.970 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-09T06:37:23.971 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-09T06:37:23.971 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-09T06:37:23.971 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-09T06:37:23.971 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-09T06:37:23.971 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-09T06:37:23.971 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-09T06:37:23.971 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-09T06:37:23.971 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:23.971 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.971 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.971 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.971 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.971 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-05-09T06:37:23.971 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-05-09T06:37:23.971 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:23.972 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.972 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:23.972 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-09T06:37:23.972 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.972 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.972 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:23.972 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:23.972 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.972 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.972 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.972 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.972 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-05-09T06:37:23.972 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-05-09T06:37:23.972 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:23.973 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.973 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:23.973 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-05-09T06:37:23.973 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.973 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.973 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-09T06:37:23.973 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-09T06:37:23.973 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-09T06:37:23.973 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-09T06:37:23.973 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-09T06:37:23.973 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-09T06:37:23.973 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-09T06:37:23.973 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-09T06:37:23.973 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:23.973 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.974 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.974 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.974 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.974 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-05-09T06:37:23.974 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-05-09T06:37:23.974 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:23.974 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.974 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:23.974 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-09T06:37:23.974 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.974 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.974 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:23.974 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:23.974 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.974 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.974 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.975 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-09T06:37:23.975 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-09T06:37:23.975 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.975 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-09T06:37:23.975 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "pg_autoscaler", 2024-05-09T06:37:23.975 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-05-09T06:37:23.975 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-05-09T06:37:23.975 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-05-09T06:37:23.975 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-05-09T06:37:23.975 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-05-09T06:37:23.975 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:23.975 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.975 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:23.975 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-09T06:37:23.975 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.975 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.976 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-09T06:37:23.976 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-09T06:37:23.976 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-09T06:37:23.976 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-09T06:37:23.976 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-09T06:37:23.976 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-09T06:37:23.976 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-09T06:37:23.976 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-09T06:37:23.976 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:23.976 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.976 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.976 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.976 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.977 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-05-09T06:37:23.977 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-05-09T06:37:23.977 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:23.977 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.977 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:23.977 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-09T06:37:23.977 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.977 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.977 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:23.977 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:23.977 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.977 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.977 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.978 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.978 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-05-09T06:37:23.978 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-05-09T06:37:23.978 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:23.978 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.978 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:23.978 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-05-09T06:37:23.978 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.978 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.978 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-09T06:37:23.978 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-09T06:37:23.978 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-09T06:37:23.978 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-09T06:37:23.978 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-09T06:37:23.978 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-09T06:37:23.979 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-09T06:37:23.979 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-09T06:37:23.979 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:23.979 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.979 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.979 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.979 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.979 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-05-09T06:37:23.979 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-05-09T06:37:23.979 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:23.979 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.979 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:23.979 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-09T06:37:23.979 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.979 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.979 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:23.980 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:23.980 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.980 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.980 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.980 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.980 INFO:teuthology.orchestra.run.smithi003.stdout: "sleep_interval": { 2024-05-09T06:37:23.980 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sleep_interval", 2024-05-09T06:37:23.980 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-05-09T06:37:23.980 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.980 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:23.980 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "60", 2024-05-09T06:37:23.980 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.981 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.981 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:23.981 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:23.981 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.981 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.981 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.981 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.981 INFO:teuthology.orchestra.run.smithi003.stdout: "threshold": { 2024-05-09T06:37:23.981 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "threshold", 2024-05-09T06:37:23.981 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "float", 2024-05-09T06:37:23.981 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.981 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:23.981 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "3.0", 2024-05-09T06:37:23.982 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "1.0", 2024-05-09T06:37:23.982 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.982 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:23.982 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "scaling threshold", 2024-05-09T06:37:23.982 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-05-09T06:37:23.982 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.982 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.982 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-09T06:37:23.982 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-09T06:37:23.982 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.982 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-09T06:37:23.982 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "progress", 2024-05-09T06:37:23.982 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-05-09T06:37:23.982 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-05-09T06:37:23.982 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-05-09T06:37:23.983 INFO:teuthology.orchestra.run.smithi003.stdout: "allow_pg_recovery_event": { 2024-05-09T06:37:23.983 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "allow_pg_recovery_event", 2024-05-09T06:37:23.983 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:23.983 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.983 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:23.983 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-09T06:37:23.983 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.983 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.983 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:23.983 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "allow the module to show pg recovery progress", 2024-05-09T06:37:23.983 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.983 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.983 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.983 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.984 INFO:teuthology.orchestra.run.smithi003.stdout: "enabled": { 2024-05-09T06:37:23.984 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "enabled", 2024-05-09T06:37:23.984 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:23.984 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.984 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:23.984 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-05-09T06:37:23.984 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.984 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.984 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:23.984 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:23.984 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.984 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.985 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.985 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.985 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-05-09T06:37:23.985 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-05-09T06:37:23.985 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:23.985 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.985 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:23.985 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-09T06:37:23.985 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.985 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.985 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-09T06:37:23.985 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-09T06:37:23.985 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-09T06:37:23.985 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-09T06:37:23.986 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-09T06:37:23.986 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-09T06:37:23.986 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-09T06:37:23.986 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-09T06:37:23.986 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:23.986 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.986 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.986 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.986 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.986 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-05-09T06:37:23.986 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-05-09T06:37:23.986 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:23.986 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.986 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:23.987 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-09T06:37:23.987 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.987 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.987 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:23.987 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:23.987 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.987 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.987 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.987 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.987 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-05-09T06:37:23.987 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-05-09T06:37:23.987 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:23.987 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.988 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:23.988 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-05-09T06:37:23.988 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.988 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.988 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-09T06:37:23.988 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-09T06:37:23.988 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-09T06:37:23.988 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-09T06:37:23.988 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-09T06:37:23.988 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-09T06:37:23.988 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-09T06:37:23.988 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-09T06:37:23.988 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:23.988 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.989 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.989 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.989 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.989 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-05-09T06:37:23.989 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-05-09T06:37:23.989 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:23.989 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.989 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:23.989 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-09T06:37:23.989 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.989 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.989 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:23.989 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:23.990 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.990 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.990 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.990 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.990 INFO:teuthology.orchestra.run.smithi003.stdout: "max_completed_events": { 2024-05-09T06:37:23.990 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "max_completed_events", 2024-05-09T06:37:23.990 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-09T06:37:23.990 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.990 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:23.990 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "50", 2024-05-09T06:37:23.990 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.990 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.990 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:23.990 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "number of past completed events to remember", 2024-05-09T06:37:23.991 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.991 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.991 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.991 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.991 INFO:teuthology.orchestra.run.smithi003.stdout: "sleep_interval": { 2024-05-09T06:37:23.991 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sleep_interval", 2024-05-09T06:37:23.991 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-05-09T06:37:23.991 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.991 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:23.991 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "5", 2024-05-09T06:37:23.991 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.991 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.991 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:23.991 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "how long the module is going to sleep", 2024-05-09T06:37:23.992 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.992 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.992 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.992 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-09T06:37:23.992 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-09T06:37:23.992 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.992 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-09T06:37:23.992 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "prometheus", 2024-05-09T06:37:23.992 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-05-09T06:37:23.992 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-05-09T06:37:23.992 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-05-09T06:37:23.992 INFO:teuthology.orchestra.run.smithi003.stdout: "cache": { 2024-05-09T06:37:23.992 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "cache", 2024-05-09T06:37:23.992 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:23.992 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.993 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:23.993 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-05-09T06:37:23.993 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.993 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.993 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:23.993 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:23.993 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.993 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.993 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.993 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.993 INFO:teuthology.orchestra.run.smithi003.stdout: "exclude_perf_counters": { 2024-05-09T06:37:23.993 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "exclude_perf_counters", 2024-05-09T06:37:23.993 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:23.993 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.993 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:23.994 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-05-09T06:37:23.994 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.994 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.994 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:23.994 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-05-09T06:37:23.994 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-05-09T06:37:23.994 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.994 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.994 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.994 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-05-09T06:37:23.994 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-05-09T06:37:23.994 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:23.994 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.994 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:23.994 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-09T06:37:23.995 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.995 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.995 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-09T06:37:23.995 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-09T06:37:23.995 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-09T06:37:23.995 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-09T06:37:23.995 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-09T06:37:23.995 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-09T06:37:23.995 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-09T06:37:23.995 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-09T06:37:23.995 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:23.995 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.995 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.995 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.995 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.996 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-05-09T06:37:23.996 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-05-09T06:37:23.996 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:23.996 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.996 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:23.996 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-09T06:37:23.996 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.996 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.996 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:23.996 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:23.996 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.996 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.996 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.996 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.996 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-05-09T06:37:23.997 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-05-09T06:37:23.997 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:23.997 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.997 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:23.997 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-05-09T06:37:23.997 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.997 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.997 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-09T06:37:23.997 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-09T06:37:23.997 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-09T06:37:23.997 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-09T06:37:23.997 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-09T06:37:23.997 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-09T06:37:23.997 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-09T06:37:23.998 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-09T06:37:23.998 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:23.998 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.998 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.998 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.998 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.998 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-05-09T06:37:23.998 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-05-09T06:37:23.998 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:23.998 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.998 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:23.998 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-09T06:37:23.998 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.998 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.999 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:23.999 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:23.999 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:23.999 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:23.999 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:23.999 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:23.999 INFO:teuthology.orchestra.run.smithi003.stdout: "rbd_stats_pools": { 2024-05-09T06:37:23.999 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rbd_stats_pools", 2024-05-09T06:37:23.999 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:23.999 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:23.999 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:23.999 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-09T06:37:23.999 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:23.999 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:23.999 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.000 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.000 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.000 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.000 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.000 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.000 INFO:teuthology.orchestra.run.smithi003.stdout: "rbd_stats_pools_refresh_interval": { 2024-05-09T06:37:24.000 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-05-09T06:37:24.000 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-09T06:37:24.000 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.000 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.000 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "300", 2024-05-09T06:37:24.000 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.000 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.000 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.000 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.000 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.001 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.001 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.001 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.001 INFO:teuthology.orchestra.run.smithi003.stdout: "scrape_interval": { 2024-05-09T06:37:24.001 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "scrape_interval", 2024-05-09T06:37:24.001 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "float", 2024-05-09T06:37:24.001 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.001 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.001 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "15.0", 2024-05-09T06:37:24.001 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.001 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.001 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.001 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.001 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.001 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.002 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.002 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.002 INFO:teuthology.orchestra.run.smithi003.stdout: "server_addr": { 2024-05-09T06:37:24.002 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "server_addr", 2024-05-09T06:37:24.002 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:24.002 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.002 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.002 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "::", 2024-05-09T06:37:24.002 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.002 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.002 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.002 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-05-09T06:37:24.002 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.002 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.002 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.003 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.003 INFO:teuthology.orchestra.run.smithi003.stdout: "server_port": { 2024-05-09T06:37:24.003 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "server_port", 2024-05-09T06:37:24.003 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-09T06:37:24.003 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.003 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.003 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "9283", 2024-05-09T06:37:24.003 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.003 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.003 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.003 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-05-09T06:37:24.003 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.003 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.003 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.004 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.004 INFO:teuthology.orchestra.run.smithi003.stdout: "stale_cache_strategy": { 2024-05-09T06:37:24.004 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "stale_cache_strategy", 2024-05-09T06:37:24.004 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:24.004 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.004 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.004 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "log", 2024-05-09T06:37:24.004 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.004 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.004 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.004 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.004 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.004 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.004 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.005 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.005 INFO:teuthology.orchestra.run.smithi003.stdout: "standby_behaviour": { 2024-05-09T06:37:24.005 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "standby_behaviour", 2024-05-09T06:37:24.005 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:24.005 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.005 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.005 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "default", 2024-05-09T06:37:24.005 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.005 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.005 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-09T06:37:24.005 INFO:teuthology.orchestra.run.smithi003.stdout: "default", 2024-05-09T06:37:24.005 INFO:teuthology.orchestra.run.smithi003.stdout: "error" 2024-05-09T06:37:24.005 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-09T06:37:24.005 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.006 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.006 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.006 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.006 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.006 INFO:teuthology.orchestra.run.smithi003.stdout: "standby_error_status_code": { 2024-05-09T06:37:24.006 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "standby_error_status_code", 2024-05-09T06:37:24.006 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-09T06:37:24.006 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.006 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.006 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "500", 2024-05-09T06:37:24.006 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "400", 2024-05-09T06:37:24.006 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "599", 2024-05-09T06:37:24.006 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.007 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.007 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.007 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.007 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.007 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-09T06:37:24.007 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-09T06:37:24.007 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.007 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-09T06:37:24.007 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rbd_support", 2024-05-09T06:37:24.007 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-05-09T06:37:24.007 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-05-09T06:37:24.007 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-05-09T06:37:24.007 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-05-09T06:37:24.007 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-05-09T06:37:24.008 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:24.008 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.008 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.008 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-09T06:37:24.008 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.008 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.008 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-09T06:37:24.008 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-09T06:37:24.008 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-09T06:37:24.008 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-09T06:37:24.008 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-09T06:37:24.008 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-09T06:37:24.008 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-09T06:37:24.009 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-09T06:37:24.009 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.009 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.009 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.009 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.009 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.009 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-05-09T06:37:24.009 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-05-09T06:37:24.009 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:24.009 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.009 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.009 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-09T06:37:24.009 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.009 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.010 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.010 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.010 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.010 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.010 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.010 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.010 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-05-09T06:37:24.010 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-05-09T06:37:24.010 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:24.010 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.010 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.010 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-05-09T06:37:24.010 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.011 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.011 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-09T06:37:24.011 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-09T06:37:24.011 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-09T06:37:24.011 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-09T06:37:24.011 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-09T06:37:24.011 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-09T06:37:24.011 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-09T06:37:24.011 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-09T06:37:24.011 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.011 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.011 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.011 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.012 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.012 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-05-09T06:37:24.012 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-05-09T06:37:24.012 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:24.012 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.012 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.012 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-09T06:37:24.012 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.012 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.013 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.013 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.013 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.013 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.013 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.013 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.013 INFO:teuthology.orchestra.run.smithi003.stdout: "max_concurrent_snap_create": { 2024-05-09T06:37:24.013 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "max_concurrent_snap_create", 2024-05-09T06:37:24.013 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-09T06:37:24.013 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.013 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.013 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "10", 2024-05-09T06:37:24.013 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.013 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.014 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.014 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.014 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.014 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.014 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.014 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.014 INFO:teuthology.orchestra.run.smithi003.stdout: "mirror_snapshot_schedule": { 2024-05-09T06:37:24.014 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "mirror_snapshot_schedule", 2024-05-09T06:37:24.014 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:24.014 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.014 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.014 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-09T06:37:24.014 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.014 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.015 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.015 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.015 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.015 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.015 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.015 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.015 INFO:teuthology.orchestra.run.smithi003.stdout: "trash_purge_schedule": { 2024-05-09T06:37:24.015 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "trash_purge_schedule", 2024-05-09T06:37:24.015 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:24.015 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.015 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.015 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-09T06:37:24.015 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.015 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.016 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.016 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.016 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.016 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.016 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.016 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-09T06:37:24.016 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-09T06:37:24.016 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.016 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-09T06:37:24.016 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "restful", 2024-05-09T06:37:24.016 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-05-09T06:37:24.016 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-05-09T06:37:24.016 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-05-09T06:37:24.016 INFO:teuthology.orchestra.run.smithi003.stdout: "enable_auth": { 2024-05-09T06:37:24.017 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "enable_auth", 2024-05-09T06:37:24.017 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:24.017 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.017 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.017 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-05-09T06:37:24.017 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.017 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.017 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.017 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.017 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.017 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.017 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.017 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.017 INFO:teuthology.orchestra.run.smithi003.stdout: "key_file": { 2024-05-09T06:37:24.017 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "key_file", 2024-05-09T06:37:24.017 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:24.018 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.018 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.018 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-09T06:37:24.018 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.018 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.018 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.018 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.018 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.018 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.018 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.018 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.018 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-05-09T06:37:24.018 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-05-09T06:37:24.018 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:24.018 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.019 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.019 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-09T06:37:24.019 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.019 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.019 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-09T06:37:24.019 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-09T06:37:24.019 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-09T06:37:24.019 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-09T06:37:24.019 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-09T06:37:24.019 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-09T06:37:24.019 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-09T06:37:24.019 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-09T06:37:24.019 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.019 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.020 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.020 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.020 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.020 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-05-09T06:37:24.020 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-05-09T06:37:24.020 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:24.020 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.020 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.020 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-09T06:37:24.020 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.020 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.020 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.020 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.020 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.020 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.020 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.021 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.021 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-05-09T06:37:24.021 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-05-09T06:37:24.021 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:24.021 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.021 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.021 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-05-09T06:37:24.021 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.021 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.021 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-09T06:37:24.021 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-09T06:37:24.021 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-09T06:37:24.021 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-09T06:37:24.021 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-09T06:37:24.022 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-09T06:37:24.022 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-09T06:37:24.022 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-09T06:37:24.022 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.022 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.022 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.022 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.022 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.022 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-05-09T06:37:24.022 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-05-09T06:37:24.022 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:24.022 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.022 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.022 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-09T06:37:24.022 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.023 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.023 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.023 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.023 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.023 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.023 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.023 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.023 INFO:teuthology.orchestra.run.smithi003.stdout: "server_addr": { 2024-05-09T06:37:24.023 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "server_addr", 2024-05-09T06:37:24.023 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:24.023 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.023 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.023 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-09T06:37:24.023 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.023 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.024 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.024 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.024 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.024 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.024 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.024 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.024 INFO:teuthology.orchestra.run.smithi003.stdout: "server_port": { 2024-05-09T06:37:24.024 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "server_port", 2024-05-09T06:37:24.024 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:24.024 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.024 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.024 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-09T06:37:24.024 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.024 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.024 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.024 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.025 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.025 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.025 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.025 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-09T06:37:24.025 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-09T06:37:24.025 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.025 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-09T06:37:24.025 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rgw", 2024-05-09T06:37:24.025 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-05-09T06:37:24.025 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-05-09T06:37:24.025 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-05-09T06:37:24.025 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-05-09T06:37:24.025 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-05-09T06:37:24.025 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:24.026 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.026 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.026 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-09T06:37:24.026 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.026 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.026 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-09T06:37:24.026 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-09T06:37:24.026 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-09T06:37:24.026 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-09T06:37:24.026 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-09T06:37:24.026 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-09T06:37:24.026 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-09T06:37:24.026 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-09T06:37:24.026 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.026 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.027 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.027 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.027 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.027 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-05-09T06:37:24.027 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-05-09T06:37:24.027 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:24.027 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.027 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.027 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-09T06:37:24.027 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.027 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.027 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.027 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.027 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.028 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.028 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.028 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.028 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-05-09T06:37:24.028 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-05-09T06:37:24.028 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:24.028 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.028 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.028 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-05-09T06:37:24.028 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.028 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.028 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-09T06:37:24.028 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-09T06:37:24.028 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-09T06:37:24.028 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-09T06:37:24.029 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-09T06:37:24.029 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-09T06:37:24.029 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-09T06:37:24.029 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-09T06:37:24.029 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.029 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.029 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.029 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.029 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.029 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-05-09T06:37:24.029 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-05-09T06:37:24.029 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:24.029 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.030 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.030 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-09T06:37:24.030 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.030 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.030 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.030 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.030 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.030 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.030 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.030 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-09T06:37:24.030 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-09T06:37:24.030 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.030 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-09T06:37:24.030 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rook", 2024-05-09T06:37:24.031 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-05-09T06:37:24.031 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-05-09T06:37:24.031 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-05-09T06:37:24.031 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-05-09T06:37:24.031 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-05-09T06:37:24.031 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:24.031 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.031 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.031 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-09T06:37:24.031 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.031 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.031 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-09T06:37:24.031 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-09T06:37:24.031 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-09T06:37:24.031 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-09T06:37:24.032 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-09T06:37:24.032 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-09T06:37:24.032 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-09T06:37:24.032 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-09T06:37:24.032 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.032 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.032 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.032 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.032 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.032 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-05-09T06:37:24.032 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-05-09T06:37:24.032 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:24.032 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.032 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.032 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-09T06:37:24.033 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.033 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.033 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.033 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.033 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.033 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.033 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.033 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.033 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-05-09T06:37:24.033 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-05-09T06:37:24.033 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:24.033 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.033 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.033 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-05-09T06:37:24.034 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.034 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.034 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-09T06:37:24.034 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-09T06:37:24.034 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-09T06:37:24.034 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-09T06:37:24.034 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-09T06:37:24.034 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-09T06:37:24.034 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-09T06:37:24.034 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-09T06:37:24.034 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.034 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.034 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.034 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.035 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.035 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-05-09T06:37:24.035 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-05-09T06:37:24.035 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:24.035 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.035 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.035 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-09T06:37:24.035 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.035 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.035 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.035 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.035 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.035 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.035 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.035 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.036 INFO:teuthology.orchestra.run.smithi003.stdout: "storage_class": { 2024-05-09T06:37:24.036 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "storage_class", 2024-05-09T06:37:24.036 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:24.036 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.036 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.036 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "local", 2024-05-09T06:37:24.036 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.036 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.036 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.036 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-05-09T06:37:24.036 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.036 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.036 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.037 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-09T06:37:24.037 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-09T06:37:24.037 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.037 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-09T06:37:24.037 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "selftest", 2024-05-09T06:37:24.037 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-05-09T06:37:24.037 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-05-09T06:37:24.037 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-05-09T06:37:24.037 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-05-09T06:37:24.037 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-05-09T06:37:24.037 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:24.037 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.037 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.037 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-09T06:37:24.038 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.038 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.038 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-09T06:37:24.038 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-09T06:37:24.038 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-09T06:37:24.038 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-09T06:37:24.038 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-09T06:37:24.038 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-09T06:37:24.038 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-09T06:37:24.038 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-09T06:37:24.038 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.038 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.038 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.039 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.039 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.039 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-05-09T06:37:24.039 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-05-09T06:37:24.039 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:24.039 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.039 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.039 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-09T06:37:24.039 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.039 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.039 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.039 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.039 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.040 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.040 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.040 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.040 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-05-09T06:37:24.040 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-05-09T06:37:24.040 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:24.040 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.040 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.040 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-05-09T06:37:24.040 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.040 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.040 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-09T06:37:24.041 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-09T06:37:24.041 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-09T06:37:24.041 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-09T06:37:24.041 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-09T06:37:24.041 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-09T06:37:24.041 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-09T06:37:24.041 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-09T06:37:24.041 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.041 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.041 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.041 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.041 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.041 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-05-09T06:37:24.042 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-05-09T06:37:24.042 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:24.042 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.042 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.042 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-09T06:37:24.042 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.042 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.042 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.042 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.042 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.042 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.042 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.042 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.043 INFO:teuthology.orchestra.run.smithi003.stdout: "roption1": { 2024-05-09T06:37:24.043 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "roption1", 2024-05-09T06:37:24.043 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:24.043 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.043 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.043 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-09T06:37:24.043 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.043 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.043 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.043 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.043 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.043 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.043 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.043 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.043 INFO:teuthology.orchestra.run.smithi003.stdout: "roption2": { 2024-05-09T06:37:24.044 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "roption2", 2024-05-09T06:37:24.044 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:24.044 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.044 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.044 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "xyz", 2024-05-09T06:37:24.044 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.044 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.044 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.044 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.044 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.044 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.044 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.044 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.044 INFO:teuthology.orchestra.run.smithi003.stdout: "rwoption1": { 2024-05-09T06:37:24.044 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rwoption1", 2024-05-09T06:37:24.044 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:24.045 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.045 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.045 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-09T06:37:24.045 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.045 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.045 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.045 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.045 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.045 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.045 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.045 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.045 INFO:teuthology.orchestra.run.smithi003.stdout: "rwoption2": { 2024-05-09T06:37:24.045 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rwoption2", 2024-05-09T06:37:24.045 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-09T06:37:24.045 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.046 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.046 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-09T06:37:24.046 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.046 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.046 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.046 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.046 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.046 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.046 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.046 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.046 INFO:teuthology.orchestra.run.smithi003.stdout: "rwoption3": { 2024-05-09T06:37:24.046 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rwoption3", 2024-05-09T06:37:24.046 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "float", 2024-05-09T06:37:24.046 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.047 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.047 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-09T06:37:24.047 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.047 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.047 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.047 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.047 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.047 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.047 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.047 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.047 INFO:teuthology.orchestra.run.smithi003.stdout: "rwoption4": { 2024-05-09T06:37:24.047 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rwoption4", 2024-05-09T06:37:24.047 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:24.047 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.047 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.048 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-09T06:37:24.048 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.048 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.048 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.048 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.048 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.048 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.048 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.048 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.048 INFO:teuthology.orchestra.run.smithi003.stdout: "rwoption5": { 2024-05-09T06:37:24.048 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rwoption5", 2024-05-09T06:37:24.048 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:24.048 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.048 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.048 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-09T06:37:24.049 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.049 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.049 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.049 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.049 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.049 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.049 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.049 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.049 INFO:teuthology.orchestra.run.smithi003.stdout: "rwoption6": { 2024-05-09T06:37:24.049 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rwoption6", 2024-05-09T06:37:24.049 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:24.049 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.049 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.050 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-05-09T06:37:24.050 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.050 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.050 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.050 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.050 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.050 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.050 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.050 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.050 INFO:teuthology.orchestra.run.smithi003.stdout: "rwoption7": { 2024-05-09T06:37:24.050 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rwoption7", 2024-05-09T06:37:24.050 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-09T06:37:24.050 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.050 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.051 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-09T06:37:24.051 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "1", 2024-05-09T06:37:24.051 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "42", 2024-05-09T06:37:24.051 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.051 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.051 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.051 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.051 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.051 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.051 INFO:teuthology.orchestra.run.smithi003.stdout: "testkey": { 2024-05-09T06:37:24.051 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "testkey", 2024-05-09T06:37:24.051 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:24.051 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.052 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.052 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-09T06:37:24.052 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.052 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.052 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.052 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.052 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.052 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.052 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.052 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.052 INFO:teuthology.orchestra.run.smithi003.stdout: "testlkey": { 2024-05-09T06:37:24.052 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "testlkey", 2024-05-09T06:37:24.052 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:24.052 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.052 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.053 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-09T06:37:24.053 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.053 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.053 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.053 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.053 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.053 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.053 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.053 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.053 INFO:teuthology.orchestra.run.smithi003.stdout: "testnewline": { 2024-05-09T06:37:24.053 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "testnewline", 2024-05-09T06:37:24.053 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:24.053 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.054 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.054 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-09T06:37:24.054 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.054 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.054 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.054 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.054 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.054 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.054 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.054 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-09T06:37:24.054 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-09T06:37:24.054 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.054 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-09T06:37:24.055 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "snap_schedule", 2024-05-09T06:37:24.055 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-05-09T06:37:24.055 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-05-09T06:37:24.055 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-05-09T06:37:24.055 INFO:teuthology.orchestra.run.smithi003.stdout: "allow_m_granularity": { 2024-05-09T06:37:24.055 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "allow_m_granularity", 2024-05-09T06:37:24.055 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:24.055 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.055 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.055 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-09T06:37:24.055 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.055 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.055 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.055 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "allow minute scheduled snapshots", 2024-05-09T06:37:24.055 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.056 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.056 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.056 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.056 INFO:teuthology.orchestra.run.smithi003.stdout: "dump_on_update": { 2024-05-09T06:37:24.056 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "dump_on_update", 2024-05-09T06:37:24.056 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:24.056 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.056 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.056 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-09T06:37:24.056 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.056 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.056 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.056 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "dump database to debug log on update", 2024-05-09T06:37:24.056 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.056 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.057 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.057 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.057 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-05-09T06:37:24.057 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-05-09T06:37:24.057 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:24.057 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.057 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.057 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-09T06:37:24.057 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.057 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.057 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-09T06:37:24.057 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-09T06:37:24.057 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-09T06:37:24.057 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-09T06:37:24.057 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-09T06:37:24.057 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-09T06:37:24.058 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-09T06:37:24.058 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-09T06:37:24.058 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.058 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.058 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.058 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.058 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.058 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-05-09T06:37:24.058 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-05-09T06:37:24.058 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:24.058 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.058 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.058 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-09T06:37:24.058 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.058 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.059 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.059 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.059 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.059 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.059 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.059 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.059 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-05-09T06:37:24.059 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-05-09T06:37:24.059 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:24.059 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.059 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.059 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-05-09T06:37:24.059 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.059 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.060 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-09T06:37:24.060 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-09T06:37:24.060 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-09T06:37:24.060 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-09T06:37:24.060 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-09T06:37:24.060 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-09T06:37:24.060 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-09T06:37:24.060 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-09T06:37:24.060 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.060 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.060 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.060 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.060 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.060 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-05-09T06:37:24.060 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-05-09T06:37:24.060 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:24.061 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.061 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.061 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-09T06:37:24.061 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.061 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.061 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.061 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.061 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.061 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.061 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.061 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-09T06:37:24.061 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-09T06:37:24.061 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.061 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-09T06:37:24.061 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "stats", 2024-05-09T06:37:24.062 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-05-09T06:37:24.062 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-05-09T06:37:24.062 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-05-09T06:37:24.062 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-05-09T06:37:24.062 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-05-09T06:37:24.062 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:24.062 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.062 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.062 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-09T06:37:24.062 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.062 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.062 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-09T06:37:24.063 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-09T06:37:24.063 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-09T06:37:24.063 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-09T06:37:24.063 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-09T06:37:24.063 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-09T06:37:24.063 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-09T06:37:24.063 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-09T06:37:24.063 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.063 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.063 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.063 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.063 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.063 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-05-09T06:37:24.063 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-05-09T06:37:24.064 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:24.064 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.064 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.064 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-09T06:37:24.064 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.064 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.064 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.064 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.064 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.064 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.065 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.065 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.065 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-05-09T06:37:24.065 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-05-09T06:37:24.065 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:24.065 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.065 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.065 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-05-09T06:37:24.065 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.065 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.065 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-09T06:37:24.065 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-09T06:37:24.065 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-09T06:37:24.065 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-09T06:37:24.066 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-09T06:37:24.066 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-09T06:37:24.066 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-09T06:37:24.066 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-09T06:37:24.066 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.066 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.066 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.066 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.066 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.066 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-05-09T06:37:24.066 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-05-09T06:37:24.066 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:24.066 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.066 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.067 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-09T06:37:24.067 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.067 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.067 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.067 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.067 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.067 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.067 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.067 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-09T06:37:24.067 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-09T06:37:24.067 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.067 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-09T06:37:24.067 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "status", 2024-05-09T06:37:24.068 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-05-09T06:37:24.068 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-05-09T06:37:24.068 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-05-09T06:37:24.068 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-05-09T06:37:24.068 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-05-09T06:37:24.068 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:24.068 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.068 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.068 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-09T06:37:24.068 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.068 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.068 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-09T06:37:24.068 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-09T06:37:24.069 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-09T06:37:24.069 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-09T06:37:24.069 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-09T06:37:24.069 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-09T06:37:24.069 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-09T06:37:24.069 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-09T06:37:24.069 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.069 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.069 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.069 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.069 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.069 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-05-09T06:37:24.069 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-05-09T06:37:24.069 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:24.070 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.070 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.070 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-09T06:37:24.070 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.070 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.070 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.070 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.070 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.070 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.070 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.070 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.071 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-05-09T06:37:24.071 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-05-09T06:37:24.071 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:24.071 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.071 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.071 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-05-09T06:37:24.071 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.071 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.071 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-09T06:37:24.071 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-09T06:37:24.071 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-09T06:37:24.071 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-09T06:37:24.071 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-09T06:37:24.072 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-09T06:37:24.072 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-09T06:37:24.072 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-09T06:37:24.072 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.072 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.072 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.072 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.072 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.072 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-05-09T06:37:24.072 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-05-09T06:37:24.072 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:24.072 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.072 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.073 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-09T06:37:24.073 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.073 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.073 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.073 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.073 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.073 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.073 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.073 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-09T06:37:24.073 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-09T06:37:24.073 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.073 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-09T06:37:24.073 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "telegraf", 2024-05-09T06:37:24.073 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-05-09T06:37:24.074 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-05-09T06:37:24.074 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-05-09T06:37:24.074 INFO:teuthology.orchestra.run.smithi003.stdout: "address": { 2024-05-09T06:37:24.074 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "address", 2024-05-09T06:37:24.074 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:24.074 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.074 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.074 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-05-09T06:37:24.074 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.074 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.074 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.074 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.074 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.074 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.075 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.075 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.075 INFO:teuthology.orchestra.run.smithi003.stdout: "interval": { 2024-05-09T06:37:24.075 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "interval", 2024-05-09T06:37:24.075 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-05-09T06:37:24.075 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.075 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.075 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "15", 2024-05-09T06:37:24.075 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.075 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.075 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.075 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.075 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.076 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.076 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.076 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.076 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-05-09T06:37:24.076 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-05-09T06:37:24.076 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:24.076 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.076 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.076 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-09T06:37:24.076 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.076 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.076 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-09T06:37:24.076 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-09T06:37:24.077 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-09T06:37:24.077 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-09T06:37:24.077 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-09T06:37:24.077 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-09T06:37:24.077 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-09T06:37:24.077 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-09T06:37:24.077 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.077 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.077 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.077 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.077 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.077 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-05-09T06:37:24.077 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-05-09T06:37:24.077 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:24.077 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.078 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.078 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-09T06:37:24.078 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.078 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.078 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.078 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.078 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.078 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.078 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.078 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.078 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-05-09T06:37:24.078 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-05-09T06:37:24.078 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:24.078 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.078 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.079 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-05-09T06:37:24.079 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.079 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.079 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-09T06:37:24.079 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-09T06:37:24.079 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-09T06:37:24.079 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-09T06:37:24.079 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-09T06:37:24.079 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-09T06:37:24.079 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-09T06:37:24.079 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-09T06:37:24.079 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.079 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.079 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.079 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.079 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.079 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-05-09T06:37:24.080 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-05-09T06:37:24.080 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:24.080 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.080 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.080 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-09T06:37:24.080 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.080 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.080 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.080 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.080 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.080 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.080 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.080 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-09T06:37:24.080 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-09T06:37:24.080 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.080 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-09T06:37:24.080 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "telemetry", 2024-05-09T06:37:24.081 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-05-09T06:37:24.081 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-05-09T06:37:24.081 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-05-09T06:37:24.081 INFO:teuthology.orchestra.run.smithi003.stdout: "channel_basic": { 2024-05-09T06:37:24.081 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "channel_basic", 2024-05-09T06:37:24.081 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:24.081 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.081 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.081 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-05-09T06:37:24.081 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.081 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.081 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.081 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Share basic cluster information (size, version)", 2024-05-09T06:37:24.081 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.081 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.082 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.082 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.082 INFO:teuthology.orchestra.run.smithi003.stdout: "channel_crash": { 2024-05-09T06:37:24.082 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "channel_crash", 2024-05-09T06:37:24.082 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:24.082 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.082 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.082 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-05-09T06:37:24.082 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.082 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.082 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.082 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-05-09T06:37:24.082 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.082 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.083 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.083 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.083 INFO:teuthology.orchestra.run.smithi003.stdout: "channel_device": { 2024-05-09T06:37:24.083 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "channel_device", 2024-05-09T06:37:24.083 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:24.083 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.083 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.083 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-05-09T06:37:24.083 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.083 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.083 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.083 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-05-09T06:37:24.083 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.083 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.084 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.084 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.084 INFO:teuthology.orchestra.run.smithi003.stdout: "channel_ident": { 2024-05-09T06:37:24.084 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "channel_ident", 2024-05-09T06:37:24.084 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:24.084 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.084 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.084 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-09T06:37:24.084 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.084 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.084 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.084 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-05-09T06:37:24.084 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.084 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.084 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.084 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.085 INFO:teuthology.orchestra.run.smithi003.stdout: "channel_perf": { 2024-05-09T06:37:24.085 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "channel_perf", 2024-05-09T06:37:24.085 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:24.085 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.085 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.085 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-09T06:37:24.085 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.085 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.085 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.085 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Share various performance metrics of a cluster", 2024-05-09T06:37:24.085 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.085 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.085 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.086 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.086 INFO:teuthology.orchestra.run.smithi003.stdout: "contact": { 2024-05-09T06:37:24.086 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "contact", 2024-05-09T06:37:24.086 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:24.086 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.086 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.086 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-09T06:37:24.086 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.086 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.086 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.086 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.086 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.086 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.086 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.087 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.087 INFO:teuthology.orchestra.run.smithi003.stdout: "description": { 2024-05-09T06:37:24.087 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "description", 2024-05-09T06:37:24.087 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:24.087 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.087 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.087 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-09T06:37:24.087 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.087 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.087 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.087 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.087 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.087 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.087 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.087 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.087 INFO:teuthology.orchestra.run.smithi003.stdout: "device_url": { 2024-05-09T06:37:24.088 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "device_url", 2024-05-09T06:37:24.088 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:24.088 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.088 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.088 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-05-09T06:37:24.088 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.088 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.088 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.088 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.088 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.088 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.088 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.088 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.088 INFO:teuthology.orchestra.run.smithi003.stdout: "enabled": { 2024-05-09T06:37:24.088 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "enabled", 2024-05-09T06:37:24.089 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:24.089 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.089 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.089 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-09T06:37:24.089 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.089 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.089 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.089 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.089 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.089 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.089 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.089 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.089 INFO:teuthology.orchestra.run.smithi003.stdout: "interval": { 2024-05-09T06:37:24.090 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "interval", 2024-05-09T06:37:24.090 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-09T06:37:24.090 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.090 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.090 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "24", 2024-05-09T06:37:24.090 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "8", 2024-05-09T06:37:24.090 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.090 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.090 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.090 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.090 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.090 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.090 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.090 INFO:teuthology.orchestra.run.smithi003.stdout: "last_opt_revision": { 2024-05-09T06:37:24.091 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "last_opt_revision", 2024-05-09T06:37:24.091 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-09T06:37:24.091 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.091 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.091 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "1", 2024-05-09T06:37:24.091 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.091 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.091 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.091 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.093 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.093 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.094 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.094 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.094 INFO:teuthology.orchestra.run.smithi003.stdout: "leaderboard": { 2024-05-09T06:37:24.094 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "leaderboard", 2024-05-09T06:37:24.094 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:24.094 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.094 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.094 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-09T06:37:24.094 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.094 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.094 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.094 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.095 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.095 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.095 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.095 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.095 INFO:teuthology.orchestra.run.smithi003.stdout: "leaderboard_description": { 2024-05-09T06:37:24.095 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "leaderboard_description", 2024-05-09T06:37:24.095 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:24.095 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.095 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.095 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-09T06:37:24.095 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.096 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.096 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.096 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.096 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.096 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.096 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.096 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.096 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-05-09T06:37:24.096 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-05-09T06:37:24.096 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:24.096 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.096 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.097 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-09T06:37:24.097 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.097 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.097 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-09T06:37:24.097 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-09T06:37:24.097 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-09T06:37:24.097 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-09T06:37:24.097 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-09T06:37:24.097 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-09T06:37:24.097 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-09T06:37:24.097 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-09T06:37:24.098 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.098 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.098 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.098 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.098 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.098 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-05-09T06:37:24.098 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-05-09T06:37:24.098 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:24.098 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.098 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.098 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-09T06:37:24.098 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.098 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.099 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.099 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.099 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.099 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.099 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.099 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.099 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-05-09T06:37:24.099 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-05-09T06:37:24.099 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:24.099 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.099 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.099 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-05-09T06:37:24.099 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.099 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.100 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-09T06:37:24.100 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-09T06:37:24.100 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-09T06:37:24.100 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-09T06:37:24.100 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-09T06:37:24.100 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-09T06:37:24.100 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-09T06:37:24.100 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-09T06:37:24.100 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.100 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.100 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.101 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.101 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.101 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-05-09T06:37:24.101 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-05-09T06:37:24.101 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:24.101 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.101 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.101 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-09T06:37:24.101 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.101 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.101 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.101 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.101 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.101 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.102 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.102 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.102 INFO:teuthology.orchestra.run.smithi003.stdout: "organization": { 2024-05-09T06:37:24.102 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "organization", 2024-05-09T06:37:24.102 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:24.102 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.102 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.102 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-09T06:37:24.102 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.102 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.102 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.102 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.102 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.103 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.103 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.103 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.103 INFO:teuthology.orchestra.run.smithi003.stdout: "proxy": { 2024-05-09T06:37:24.103 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "proxy", 2024-05-09T06:37:24.103 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:24.103 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.103 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.103 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-09T06:37:24.103 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.103 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.103 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.103 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.103 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.103 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.104 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.104 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.104 INFO:teuthology.orchestra.run.smithi003.stdout: "url": { 2024-05-09T06:37:24.104 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "url", 2024-05-09T06:37:24.104 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:24.104 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.104 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.104 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-05-09T06:37:24.104 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.104 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.104 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.104 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.104 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.104 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.104 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.105 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-09T06:37:24.105 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-09T06:37:24.105 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.105 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-09T06:37:24.105 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "test_orchestrator", 2024-05-09T06:37:24.105 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-05-09T06:37:24.105 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-05-09T06:37:24.105 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-05-09T06:37:24.105 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-05-09T06:37:24.105 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-05-09T06:37:24.105 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:24.105 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.105 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.105 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-09T06:37:24.105 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.106 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.106 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-09T06:37:24.106 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-09T06:37:24.106 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-09T06:37:24.106 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-09T06:37:24.106 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-09T06:37:24.106 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-09T06:37:24.106 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-09T06:37:24.106 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-09T06:37:24.106 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.106 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.106 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.106 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.107 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.107 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-05-09T06:37:24.107 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-05-09T06:37:24.107 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:24.107 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.107 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.107 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-09T06:37:24.107 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.107 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.107 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.107 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.107 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.107 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.107 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.108 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.108 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-05-09T06:37:24.108 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-05-09T06:37:24.108 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:24.108 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.108 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.108 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-05-09T06:37:24.108 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.108 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.108 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-09T06:37:24.108 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-09T06:37:24.108 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-09T06:37:24.108 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-09T06:37:24.108 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-09T06:37:24.108 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-09T06:37:24.109 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-09T06:37:24.109 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-09T06:37:24.109 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.109 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.109 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.109 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.109 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.109 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-05-09T06:37:24.109 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-05-09T06:37:24.109 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:24.109 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.109 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.109 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-09T06:37:24.109 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.109 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.109 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.110 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.110 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.110 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.110 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.110 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-09T06:37:24.110 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-09T06:37:24.110 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.110 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-09T06:37:24.110 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "volumes", 2024-05-09T06:37:24.110 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-05-09T06:37:24.110 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-05-09T06:37:24.110 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-05-09T06:37:24.110 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-05-09T06:37:24.110 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-05-09T06:37:24.110 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:24.111 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.111 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.111 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-09T06:37:24.111 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.111 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.111 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-09T06:37:24.111 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-09T06:37:24.111 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-09T06:37:24.111 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-09T06:37:24.111 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-09T06:37:24.111 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-09T06:37:24.111 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-09T06:37:24.111 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-09T06:37:24.111 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.111 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.112 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.112 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.112 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.112 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-05-09T06:37:24.112 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-05-09T06:37:24.112 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:24.112 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.112 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.112 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-09T06:37:24.112 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.112 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.112 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.112 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.112 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.112 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.113 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.113 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.113 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-05-09T06:37:24.113 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-05-09T06:37:24.113 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:24.113 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.113 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.113 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-05-09T06:37:24.113 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.113 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.113 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-09T06:37:24.113 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-09T06:37:24.113 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-09T06:37:24.113 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-09T06:37:24.113 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-09T06:37:24.114 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-09T06:37:24.114 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-09T06:37:24.114 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-09T06:37:24.114 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.114 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.114 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.114 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.114 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.114 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-05-09T06:37:24.114 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-05-09T06:37:24.114 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:24.114 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.114 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.114 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-09T06:37:24.114 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.115 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.115 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.115 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.115 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.115 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.115 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.115 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.115 INFO:teuthology.orchestra.run.smithi003.stdout: "max_concurrent_clones": { 2024-05-09T06:37:24.115 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "max_concurrent_clones", 2024-05-09T06:37:24.115 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-09T06:37:24.115 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.115 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.115 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "4", 2024-05-09T06:37:24.115 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.115 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.116 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.116 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Number of asynchronous cloner threads", 2024-05-09T06:37:24.116 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.116 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.116 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.116 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.116 INFO:teuthology.orchestra.run.smithi003.stdout: "periodic_async_work": { 2024-05-09T06:37:24.116 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "periodic_async_work", 2024-05-09T06:37:24.116 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:24.116 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.116 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.116 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-09T06:37:24.116 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.116 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.116 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.116 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Periodically check for async work", 2024-05-09T06:37:24.117 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.117 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.117 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.117 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.117 INFO:teuthology.orchestra.run.smithi003.stdout: "snapshot_clone_delay": { 2024-05-09T06:37:24.117 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "snapshot_clone_delay", 2024-05-09T06:37:24.117 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-09T06:37:24.117 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.117 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.117 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-05-09T06:37:24.117 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.117 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.117 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.117 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-05-09T06:37:24.117 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.118 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.118 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.118 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.118 INFO:teuthology.orchestra.run.smithi003.stdout: "snapshot_clone_no_wait": { 2024-05-09T06:37:24.118 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "snapshot_clone_no_wait", 2024-05-09T06:37:24.118 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:24.118 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.118 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.118 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-05-09T06:37:24.118 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.118 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.118 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.118 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-05-09T06:37:24.118 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.118 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.119 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.119 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-09T06:37:24.119 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-09T06:37:24.119 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.119 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-09T06:37:24.119 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "zabbix", 2024-05-09T06:37:24.119 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-05-09T06:37:24.119 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-05-09T06:37:24.119 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-05-09T06:37:24.119 INFO:teuthology.orchestra.run.smithi003.stdout: "discovery_interval": { 2024-05-09T06:37:24.119 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "discovery_interval", 2024-05-09T06:37:24.119 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "uint", 2024-05-09T06:37:24.119 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.119 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.119 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "100", 2024-05-09T06:37:24.119 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.120 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.120 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.120 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.120 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.120 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.120 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.120 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.120 INFO:teuthology.orchestra.run.smithi003.stdout: "identifier": { 2024-05-09T06:37:24.120 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "identifier", 2024-05-09T06:37:24.120 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:24.120 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.120 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.120 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-09T06:37:24.120 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.120 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.121 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.121 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.121 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.121 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.121 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.121 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.121 INFO:teuthology.orchestra.run.smithi003.stdout: "interval": { 2024-05-09T06:37:24.121 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "interval", 2024-05-09T06:37:24.121 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-05-09T06:37:24.121 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.121 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.121 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "60", 2024-05-09T06:37:24.121 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.121 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.121 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.121 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.122 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.122 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.122 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.122 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.122 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-05-09T06:37:24.122 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-05-09T06:37:24.122 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:24.122 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.122 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.122 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-09T06:37:24.122 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.122 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.122 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-09T06:37:24.122 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-09T06:37:24.122 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-09T06:37:24.123 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-09T06:37:24.123 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-09T06:37:24.123 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-09T06:37:24.123 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-09T06:37:24.123 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-09T06:37:24.123 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.123 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.123 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.123 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.123 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.123 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-05-09T06:37:24.123 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-05-09T06:37:24.123 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:24.123 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.123 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.124 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-09T06:37:24.124 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.124 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.124 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.124 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.124 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.124 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.124 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.124 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.124 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-05-09T06:37:24.124 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-05-09T06:37:24.124 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:24.124 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.124 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.124 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-05-09T06:37:24.125 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.125 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.125 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-09T06:37:24.125 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-09T06:37:24.125 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-09T06:37:24.125 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-09T06:37:24.125 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-09T06:37:24.125 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-09T06:37:24.125 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-09T06:37:24.125 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-09T06:37:24.125 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.125 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.125 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.125 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.125 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.125 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-05-09T06:37:24.126 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-05-09T06:37:24.126 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:24.126 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.126 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.126 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-09T06:37:24.126 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.126 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.126 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.126 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.126 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.126 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.126 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.126 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.126 INFO:teuthology.orchestra.run.smithi003.stdout: "zabbix_host": { 2024-05-09T06:37:24.127 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "zabbix_host", 2024-05-09T06:37:24.127 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:24.127 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.127 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.127 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-09T06:37:24.127 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.127 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.127 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.127 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.127 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.127 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.127 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.127 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.127 INFO:teuthology.orchestra.run.smithi003.stdout: "zabbix_port": { 2024-05-09T06:37:24.127 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "zabbix_port", 2024-05-09T06:37:24.128 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-09T06:37:24.128 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.128 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.128 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "10051", 2024-05-09T06:37:24.128 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.128 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.128 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.128 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.128 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.128 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.128 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.128 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.128 INFO:teuthology.orchestra.run.smithi003.stdout: "zabbix_sender": { 2024-05-09T06:37:24.128 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "zabbix_sender", 2024-05-09T06:37:24.128 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:24.128 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.129 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.129 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-05-09T06:37:24.129 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.129 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.129 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.129 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.129 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.129 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.129 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.129 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-09T06:37:24.129 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-09T06:37:24.129 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-09T06:37:24.129 INFO:teuthology.orchestra.run.smithi003.stdout: ] 2024-05-09T06:37:24.129 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-09T06:37:24.129 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-09T06:37:24.129 INFO:teuthology.orchestra.run.smithi003.stdout: "modules": [ 2024-05-09T06:37:24.130 INFO:teuthology.orchestra.run.smithi003.stdout: "iostat", 2024-05-09T06:37:24.130 INFO:teuthology.orchestra.run.smithi003.stdout: "nfs", 2024-05-09T06:37:24.130 INFO:teuthology.orchestra.run.smithi003.stdout: "restful" 2024-05-09T06:37:24.130 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-09T06:37:24.130 INFO:teuthology.orchestra.run.smithi003.stdout: "available_modules": [ 2024-05-09T06:37:24.130 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-09T06:37:24.130 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "alerts", 2024-05-09T06:37:24.130 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-05-09T06:37:24.130 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-05-09T06:37:24.130 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-05-09T06:37:24.130 INFO:teuthology.orchestra.run.smithi003.stdout: "interval": { 2024-05-09T06:37:24.130 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "interval", 2024-05-09T06:37:24.130 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-05-09T06:37:24.130 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.130 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.131 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "60", 2024-05-09T06:37:24.131 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.131 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.131 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.131 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "How frequently to reexamine health status", 2024-05-09T06:37:24.131 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.131 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.131 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.131 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.131 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-05-09T06:37:24.131 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-05-09T06:37:24.131 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:24.131 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.131 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.131 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-09T06:37:24.131 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.132 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.132 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-09T06:37:24.132 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-09T06:37:24.132 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-09T06:37:24.132 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-09T06:37:24.132 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-09T06:37:24.132 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-09T06:37:24.132 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-09T06:37:24.132 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-09T06:37:24.132 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.132 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.132 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.132 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.132 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.132 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-05-09T06:37:24.133 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-05-09T06:37:24.133 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:24.133 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.133 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.133 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-09T06:37:24.133 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.133 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.133 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.133 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.133 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.133 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.133 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.133 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.133 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-05-09T06:37:24.133 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-05-09T06:37:24.134 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:24.134 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.134 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.134 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-05-09T06:37:24.134 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.134 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.134 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-09T06:37:24.134 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-09T06:37:24.134 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-09T06:37:24.134 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-09T06:37:24.134 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-09T06:37:24.134 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-09T06:37:24.134 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-09T06:37:24.134 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-09T06:37:24.135 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.135 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.135 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.135 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.135 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.135 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-05-09T06:37:24.135 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-05-09T06:37:24.135 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:24.135 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.135 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.135 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-09T06:37:24.135 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.135 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.135 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.135 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.136 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.136 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.136 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.136 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.136 INFO:teuthology.orchestra.run.smithi003.stdout: "smtp_destination": { 2024-05-09T06:37:24.136 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "smtp_destination", 2024-05-09T06:37:24.136 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:24.136 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.136 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.136 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-09T06:37:24.136 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.136 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.136 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.136 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Email address to send alerts to", 2024-05-09T06:37:24.136 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.137 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.137 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.137 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.137 INFO:teuthology.orchestra.run.smithi003.stdout: "smtp_from_name": { 2024-05-09T06:37:24.137 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "smtp_from_name", 2024-05-09T06:37:24.137 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:24.137 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.137 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.137 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "Ceph", 2024-05-09T06:37:24.137 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.137 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.137 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.137 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Email From: name", 2024-05-09T06:37:24.137 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.137 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.138 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.138 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.138 INFO:teuthology.orchestra.run.smithi003.stdout: "smtp_host": { 2024-05-09T06:37:24.138 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "smtp_host", 2024-05-09T06:37:24.138 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:24.138 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.138 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.138 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-09T06:37:24.138 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.138 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.138 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.138 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "SMTP server", 2024-05-09T06:37:24.138 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.139 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.139 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.139 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.139 INFO:teuthology.orchestra.run.smithi003.stdout: "smtp_password": { 2024-05-09T06:37:24.139 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "smtp_password", 2024-05-09T06:37:24.139 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:24.139 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.139 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.139 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-09T06:37:24.139 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.139 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.139 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.139 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Password to authenticate with", 2024-05-09T06:37:24.139 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.139 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.140 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.140 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.140 INFO:teuthology.orchestra.run.smithi003.stdout: "smtp_port": { 2024-05-09T06:37:24.140 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "smtp_port", 2024-05-09T06:37:24.140 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-09T06:37:24.140 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.140 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.140 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "465", 2024-05-09T06:37:24.140 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.140 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.140 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.140 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "SMTP port", 2024-05-09T06:37:24.140 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.140 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.140 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.140 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.140 INFO:teuthology.orchestra.run.smithi003.stdout: "smtp_sender": { 2024-05-09T06:37:24.141 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "smtp_sender", 2024-05-09T06:37:24.141 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:24.141 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.141 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.141 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-09T06:37:24.141 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.141 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.141 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.141 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "SMTP envelope sender", 2024-05-09T06:37:24.141 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.141 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.141 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.141 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.141 INFO:teuthology.orchestra.run.smithi003.stdout: "smtp_ssl": { 2024-05-09T06:37:24.142 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "smtp_ssl", 2024-05-09T06:37:24.142 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:24.142 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.142 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.142 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-05-09T06:37:24.142 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.142 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.142 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.142 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Use SSL to connect to SMTP server", 2024-05-09T06:37:24.142 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.142 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.142 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.142 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.142 INFO:teuthology.orchestra.run.smithi003.stdout: "smtp_user": { 2024-05-09T06:37:24.143 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "smtp_user", 2024-05-09T06:37:24.143 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:24.143 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.143 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.143 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-09T06:37:24.143 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.143 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.143 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.143 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "User to authenticate as", 2024-05-09T06:37:24.143 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.143 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.143 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.143 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-09T06:37:24.143 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-09T06:37:24.143 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.144 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-09T06:37:24.144 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "balancer", 2024-05-09T06:37:24.144 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-05-09T06:37:24.144 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-05-09T06:37:24.144 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-05-09T06:37:24.144 INFO:teuthology.orchestra.run.smithi003.stdout: "active": { 2024-05-09T06:37:24.144 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "active", 2024-05-09T06:37:24.144 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:24.144 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.144 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.144 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-05-09T06:37:24.144 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.144 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.144 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.144 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "automatically balance PGs across cluster", 2024-05-09T06:37:24.144 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.144 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.145 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.145 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.145 INFO:teuthology.orchestra.run.smithi003.stdout: "begin_time": { 2024-05-09T06:37:24.145 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "begin_time", 2024-05-09T06:37:24.145 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:24.145 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.145 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.145 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0000", 2024-05-09T06:37:24.145 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.145 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.145 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.145 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "beginning time of day to automatically balance", 2024-05-09T06:37:24.145 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-09T06:37:24.145 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.145 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.146 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.146 INFO:teuthology.orchestra.run.smithi003.stdout: "begin_weekday": { 2024-05-09T06:37:24.146 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "begin_weekday", 2024-05-09T06:37:24.146 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "uint", 2024-05-09T06:37:24.146 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.146 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.146 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-05-09T06:37:24.146 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "0", 2024-05-09T06:37:24.146 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "6", 2024-05-09T06:37:24.146 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.146 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-05-09T06:37:24.146 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-09T06:37:24.146 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.147 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.147 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.147 INFO:teuthology.orchestra.run.smithi003.stdout: "crush_compat_max_iterations": { 2024-05-09T06:37:24.147 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "crush_compat_max_iterations", 2024-05-09T06:37:24.147 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "uint", 2024-05-09T06:37:24.147 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.147 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.147 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "25", 2024-05-09T06:37:24.147 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "1", 2024-05-09T06:37:24.147 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "250", 2024-05-09T06:37:24.147 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.147 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-05-09T06:37:24.147 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.147 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.147 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.147 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.148 INFO:teuthology.orchestra.run.smithi003.stdout: "crush_compat_metrics": { 2024-05-09T06:37:24.148 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "crush_compat_metrics", 2024-05-09T06:37:24.148 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:24.148 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.148 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.148 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "pgs,objects,bytes", 2024-05-09T06:37:24.148 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.148 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.148 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.148 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-05-09T06:37:24.148 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-05-09T06:37:24.148 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.148 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.148 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.149 INFO:teuthology.orchestra.run.smithi003.stdout: "crush_compat_step": { 2024-05-09T06:37:24.149 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "crush_compat_step", 2024-05-09T06:37:24.149 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "float", 2024-05-09T06:37:24.149 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.149 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.149 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0.5", 2024-05-09T06:37:24.149 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "0.001", 2024-05-09T06:37:24.149 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "0.999", 2024-05-09T06:37:24.149 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.149 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "aggressiveness of optimization", 2024-05-09T06:37:24.149 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-05-09T06:37:24.149 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.149 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.149 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.149 INFO:teuthology.orchestra.run.smithi003.stdout: "end_time": { 2024-05-09T06:37:24.150 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "end_time", 2024-05-09T06:37:24.150 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:24.150 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.150 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.150 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "2359", 2024-05-09T06:37:24.150 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.150 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.150 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.150 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "ending time of day to automatically balance", 2024-05-09T06:37:24.150 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-09T06:37:24.150 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.150 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.150 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.150 INFO:teuthology.orchestra.run.smithi003.stdout: "end_weekday": { 2024-05-09T06:37:24.151 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "end_weekday", 2024-05-09T06:37:24.151 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "uint", 2024-05-09T06:37:24.151 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.151 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.151 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-05-09T06:37:24.151 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "0", 2024-05-09T06:37:24.151 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "6", 2024-05-09T06:37:24.151 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.151 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-05-09T06:37:24.151 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-09T06:37:24.151 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.151 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.151 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.151 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-05-09T06:37:24.151 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-05-09T06:37:24.152 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:24.152 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.152 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.152 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-09T06:37:24.152 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.152 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.152 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-09T06:37:24.152 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-09T06:37:24.152 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-09T06:37:24.152 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-09T06:37:24.152 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-09T06:37:24.152 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-09T06:37:24.152 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-09T06:37:24.152 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-09T06:37:24.152 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.153 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.153 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.153 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.153 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.153 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-05-09T06:37:24.153 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-05-09T06:37:24.153 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:24.153 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.153 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.153 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-09T06:37:24.153 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.153 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.153 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.153 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.154 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.154 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.154 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.154 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.154 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-05-09T06:37:24.154 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-05-09T06:37:24.154 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:24.154 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.154 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.154 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-05-09T06:37:24.154 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.154 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.154 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-09T06:37:24.154 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-09T06:37:24.154 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-09T06:37:24.155 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-09T06:37:24.155 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-09T06:37:24.155 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-09T06:37:24.155 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-09T06:37:24.155 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-09T06:37:24.155 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.155 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.155 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.155 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.155 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.155 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-05-09T06:37:24.155 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-05-09T06:37:24.155 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:24.155 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.156 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.156 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-09T06:37:24.156 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.156 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.156 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.156 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.156 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.156 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.156 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.156 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.156 INFO:teuthology.orchestra.run.smithi003.stdout: "min_score": { 2024-05-09T06:37:24.156 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "min_score", 2024-05-09T06:37:24.156 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "float", 2024-05-09T06:37:24.156 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.157 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.157 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-05-09T06:37:24.157 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.157 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.157 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.157 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-05-09T06:37:24.157 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.157 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.157 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.157 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.157 INFO:teuthology.orchestra.run.smithi003.stdout: "mode": { 2024-05-09T06:37:24.157 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "mode", 2024-05-09T06:37:24.157 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:24.157 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.157 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.158 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "upmap", 2024-05-09T06:37:24.158 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.158 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.158 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-09T06:37:24.158 INFO:teuthology.orchestra.run.smithi003.stdout: "crush-compat", 2024-05-09T06:37:24.158 INFO:teuthology.orchestra.run.smithi003.stdout: "none", 2024-05-09T06:37:24.158 INFO:teuthology.orchestra.run.smithi003.stdout: "read", 2024-05-09T06:37:24.158 INFO:teuthology.orchestra.run.smithi003.stdout: "upmap", 2024-05-09T06:37:24.158 INFO:teuthology.orchestra.run.smithi003.stdout: "upmap-read" 2024-05-09T06:37:24.158 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-09T06:37:24.158 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Balancer mode", 2024-05-09T06:37:24.158 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.158 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.158 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.158 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.159 INFO:teuthology.orchestra.run.smithi003.stdout: "pool_ids": { 2024-05-09T06:37:24.159 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "pool_ids", 2024-05-09T06:37:24.159 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:24.159 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.159 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.159 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-09T06:37:24.159 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.159 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.159 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.159 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-05-09T06:37:24.159 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.159 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.159 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.159 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.160 INFO:teuthology.orchestra.run.smithi003.stdout: "sleep_interval": { 2024-05-09T06:37:24.160 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sleep_interval", 2024-05-09T06:37:24.160 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-05-09T06:37:24.160 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.160 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.160 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "60", 2024-05-09T06:37:24.160 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.160 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.160 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.160 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-05-09T06:37:24.160 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.160 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.160 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.160 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.160 INFO:teuthology.orchestra.run.smithi003.stdout: "upmap_max_deviation": { 2024-05-09T06:37:24.161 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "upmap_max_deviation", 2024-05-09T06:37:24.161 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-09T06:37:24.161 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.161 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.161 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "5", 2024-05-09T06:37:24.161 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "1", 2024-05-09T06:37:24.161 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.161 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.161 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "deviation below which no optimization is attempted", 2024-05-09T06:37:24.161 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-05-09T06:37:24.161 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.161 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.161 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.161 INFO:teuthology.orchestra.run.smithi003.stdout: "upmap_max_optimizations": { 2024-05-09T06:37:24.161 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "upmap_max_optimizations", 2024-05-09T06:37:24.162 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "uint", 2024-05-09T06:37:24.162 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.162 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.162 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "10", 2024-05-09T06:37:24.162 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.162 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.162 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.162 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-05-09T06:37:24.162 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.162 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.162 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.162 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-09T06:37:24.162 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-09T06:37:24.162 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.162 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-09T06:37:24.163 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "cephadm", 2024-05-09T06:37:24.163 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-05-09T06:37:24.163 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-05-09T06:37:24.163 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-05-09T06:37:24.163 INFO:teuthology.orchestra.run.smithi003.stdout: "agent_down_multiplier": { 2024-05-09T06:37:24.163 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "agent_down_multiplier", 2024-05-09T06:37:24.163 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "float", 2024-05-09T06:37:24.163 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.163 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.163 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "3.0", 2024-05-09T06:37:24.163 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.163 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.163 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.163 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-05-09T06:37:24.164 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.164 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.164 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.164 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.164 INFO:teuthology.orchestra.run.smithi003.stdout: "agent_refresh_rate": { 2024-05-09T06:37:24.164 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "agent_refresh_rate", 2024-05-09T06:37:24.164 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-05-09T06:37:24.164 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.164 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.164 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "20", 2024-05-09T06:37:24.164 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.164 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.164 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.164 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-05-09T06:37:24.165 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.165 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.165 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.165 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.165 INFO:teuthology.orchestra.run.smithi003.stdout: "agent_starting_port": { 2024-05-09T06:37:24.165 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "agent_starting_port", 2024-05-09T06:37:24.165 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-09T06:37:24.165 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.165 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.165 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "4721", 2024-05-09T06:37:24.165 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.165 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.165 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.165 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-05-09T06:37:24.166 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.166 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.166 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.166 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.166 INFO:teuthology.orchestra.run.smithi003.stdout: "allow_ptrace": { 2024-05-09T06:37:24.166 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "allow_ptrace", 2024-05-09T06:37:24.166 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:24.166 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.166 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.166 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-09T06:37:24.166 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.166 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.166 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.166 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-05-09T06:37:24.166 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-05-09T06:37:24.167 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.167 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.167 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.167 INFO:teuthology.orchestra.run.smithi003.stdout: "autotune_interval": { 2024-05-09T06:37:24.167 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "autotune_interval", 2024-05-09T06:37:24.167 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-05-09T06:37:24.167 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.167 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.167 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "600", 2024-05-09T06:37:24.167 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.167 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.167 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.167 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "how frequently to autotune daemon memory", 2024-05-09T06:37:24.167 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.167 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.168 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.168 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.168 INFO:teuthology.orchestra.run.smithi003.stdout: "autotune_memory_target_ratio": { 2024-05-09T06:37:24.168 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "autotune_memory_target_ratio", 2024-05-09T06:37:24.168 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "float", 2024-05-09T06:37:24.168 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.168 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.168 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0.7", 2024-05-09T06:37:24.168 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.168 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.168 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.168 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-05-09T06:37:24.168 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.168 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.169 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.169 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.169 INFO:teuthology.orchestra.run.smithi003.stdout: "cephadm_log_destination": { 2024-05-09T06:37:24.169 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "cephadm_log_destination", 2024-05-09T06:37:24.169 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:24.169 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.169 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.169 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-09T06:37:24.169 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.169 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.169 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-09T06:37:24.169 INFO:teuthology.orchestra.run.smithi003.stdout: "file", 2024-05-09T06:37:24.169 INFO:teuthology.orchestra.run.smithi003.stdout: "file,syslog", 2024-05-09T06:37:24.169 INFO:teuthology.orchestra.run.smithi003.stdout: "syslog" 2024-05-09T06:37:24.169 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-09T06:37:24.170 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-05-09T06:37:24.170 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.170 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.170 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.170 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.170 INFO:teuthology.orchestra.run.smithi003.stdout: "cgroups_split": { 2024-05-09T06:37:24.170 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "cgroups_split", 2024-05-09T06:37:24.170 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:24.170 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.170 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.170 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-05-09T06:37:24.170 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.170 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.171 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.171 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-05-09T06:37:24.171 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.171 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.171 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.171 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.171 INFO:teuthology.orchestra.run.smithi003.stdout: "config_checks_enabled": { 2024-05-09T06:37:24.171 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "config_checks_enabled", 2024-05-09T06:37:24.171 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:24.171 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.171 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.171 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-09T06:37:24.171 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.171 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.171 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.172 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-05-09T06:37:24.172 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.172 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.172 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.172 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.172 INFO:teuthology.orchestra.run.smithi003.stdout: "config_dashboard": { 2024-05-09T06:37:24.172 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "config_dashboard", 2024-05-09T06:37:24.172 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:24.172 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.172 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.172 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-05-09T06:37:24.172 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.172 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.172 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.172 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-05-09T06:37:24.173 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.173 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.173 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.173 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.173 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_alertmanager": { 2024-05-09T06:37:24.173 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_alertmanager", 2024-05-09T06:37:24.173 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:24.173 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.173 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.173 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-05-09T06:37:24.173 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.173 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.173 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.173 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Prometheus container image", 2024-05-09T06:37:24.174 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.174 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.174 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.174 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.174 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_base": { 2024-05-09T06:37:24.174 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_base", 2024-05-09T06:37:24.174 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:24.174 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.174 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.174 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/ceph/ceph", 2024-05-09T06:37:24.174 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.174 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.174 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.174 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Container image name, without the tag", 2024-05-09T06:37:24.174 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.175 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.175 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.175 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.175 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_elasticsearch": { 2024-05-09T06:37:24.175 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_elasticsearch", 2024-05-09T06:37:24.175 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:24.175 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.175 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.175 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-05-09T06:37:24.175 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.175 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.175 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.175 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "elasticsearch container image", 2024-05-09T06:37:24.175 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.175 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.176 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.176 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.176 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_grafana": { 2024-05-09T06:37:24.176 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_grafana", 2024-05-09T06:37:24.176 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:24.176 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.176 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.176 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-05-09T06:37:24.176 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.176 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.176 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.176 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Prometheus container image", 2024-05-09T06:37:24.176 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.176 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.177 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.177 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.177 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_haproxy": { 2024-05-09T06:37:24.177 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_haproxy", 2024-05-09T06:37:24.177 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:24.177 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.177 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.177 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-05-09T06:37:24.177 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.177 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.177 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.177 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "HAproxy container image", 2024-05-09T06:37:24.177 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.178 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.178 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.178 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.178 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_jaeger_agent": { 2024-05-09T06:37:24.178 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_jaeger_agent", 2024-05-09T06:37:24.178 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:24.178 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.178 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.178 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-05-09T06:37:24.178 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.178 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.178 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.178 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Jaeger agent container image", 2024-05-09T06:37:24.178 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.179 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.179 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.179 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.179 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_jaeger_collector": { 2024-05-09T06:37:24.179 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_jaeger_collector", 2024-05-09T06:37:24.179 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:24.179 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.179 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.179 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-05-09T06:37:24.179 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.179 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.179 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.179 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Jaeger collector container image", 2024-05-09T06:37:24.179 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.179 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.180 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.180 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.180 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_jaeger_query": { 2024-05-09T06:37:24.180 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_jaeger_query", 2024-05-09T06:37:24.180 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:24.180 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.180 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.180 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-05-09T06:37:24.180 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.180 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.180 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.180 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Jaeger query container image", 2024-05-09T06:37:24.180 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.180 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.180 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.181 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.181 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_keepalived": { 2024-05-09T06:37:24.181 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_keepalived", 2024-05-09T06:37:24.181 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:24.181 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.181 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.181 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-05-09T06:37:24.181 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.181 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.181 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.181 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Keepalived container image", 2024-05-09T06:37:24.181 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.181 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.181 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.182 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.182 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_loki": { 2024-05-09T06:37:24.182 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_loki", 2024-05-09T06:37:24.182 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:24.182 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.182 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.182 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-05-09T06:37:24.182 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.182 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.182 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.182 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Loki container image", 2024-05-09T06:37:24.182 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.182 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.182 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.182 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.183 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_node_exporter": { 2024-05-09T06:37:24.183 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_node_exporter", 2024-05-09T06:37:24.183 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:24.183 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.183 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.183 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-05-09T06:37:24.183 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.183 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.183 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.183 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Prometheus container image", 2024-05-09T06:37:24.183 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.183 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.183 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.183 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.184 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_nvmeof": { 2024-05-09T06:37:24.184 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_nvmeof", 2024-05-09T06:37:24.184 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:24.184 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.184 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.184 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-05-09T06:37:24.184 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.184 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.184 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.184 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Nvme-of container image", 2024-05-09T06:37:24.184 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.184 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.184 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.184 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.184 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_prometheus": { 2024-05-09T06:37:24.185 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_prometheus", 2024-05-09T06:37:24.185 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:24.185 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.185 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.185 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-05-09T06:37:24.185 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.185 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.185 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.185 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Prometheus container image", 2024-05-09T06:37:24.185 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.185 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.185 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.185 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.185 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_promtail": { 2024-05-09T06:37:24.185 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_promtail", 2024-05-09T06:37:24.186 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:24.186 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.186 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.186 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-05-09T06:37:24.186 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.186 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.186 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.186 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Promtail container image", 2024-05-09T06:37:24.186 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.186 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.186 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.186 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.186 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_samba": { 2024-05-09T06:37:24.186 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_samba", 2024-05-09T06:37:24.187 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:24.187 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.187 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.187 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-05-09T06:37:24.187 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.187 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.187 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.187 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Samba/SMB container image", 2024-05-09T06:37:24.187 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.187 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.187 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.187 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.187 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_snmp_gateway": { 2024-05-09T06:37:24.187 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_snmp_gateway", 2024-05-09T06:37:24.188 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:24.188 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.188 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.188 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-05-09T06:37:24.188 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.188 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.188 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.188 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "SNMP Gateway container image", 2024-05-09T06:37:24.188 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.188 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.188 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.189 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.189 INFO:teuthology.orchestra.run.smithi003.stdout: "container_init": { 2024-05-09T06:37:24.189 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_init", 2024-05-09T06:37:24.189 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:24.189 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.189 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.189 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-05-09T06:37:24.189 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.189 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.189 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.189 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Run podman/docker with `--init`", 2024-05-09T06:37:24.189 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.190 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.190 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.190 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.190 INFO:teuthology.orchestra.run.smithi003.stdout: "daemon_cache_timeout": { 2024-05-09T06:37:24.190 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "daemon_cache_timeout", 2024-05-09T06:37:24.190 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-05-09T06:37:24.190 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.190 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.190 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "600", 2024-05-09T06:37:24.190 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.190 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.190 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.190 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-05-09T06:37:24.190 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.191 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.191 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.191 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.191 INFO:teuthology.orchestra.run.smithi003.stdout: "default_cephadm_command_timeout": { 2024-05-09T06:37:24.191 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "default_cephadm_command_timeout", 2024-05-09T06:37:24.191 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-09T06:37:24.191 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.191 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.191 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "900", 2024-05-09T06:37:24.191 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.191 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.191 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.191 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-05-09T06:37:24.191 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.191 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.192 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.192 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.192 INFO:teuthology.orchestra.run.smithi003.stdout: "default_registry": { 2024-05-09T06:37:24.192 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "default_registry", 2024-05-09T06:37:24.192 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:24.192 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.192 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.192 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "docker.io", 2024-05-09T06:37:24.192 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.192 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.192 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.192 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-05-09T06:37:24.192 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.192 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.192 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.192 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.193 INFO:teuthology.orchestra.run.smithi003.stdout: "device_cache_timeout": { 2024-05-09T06:37:24.193 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "device_cache_timeout", 2024-05-09T06:37:24.193 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-05-09T06:37:24.193 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.193 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.193 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "1800", 2024-05-09T06:37:24.193 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.193 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.193 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.193 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "seconds to cache device inventory", 2024-05-09T06:37:24.193 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.193 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.193 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.193 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.193 INFO:teuthology.orchestra.run.smithi003.stdout: "device_enhanced_scan": { 2024-05-09T06:37:24.194 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "device_enhanced_scan", 2024-05-09T06:37:24.194 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:24.194 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.194 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.194 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-09T06:37:24.194 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.194 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.194 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.194 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Use libstoragemgmt during device scans", 2024-05-09T06:37:24.194 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.194 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.194 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.194 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.194 INFO:teuthology.orchestra.run.smithi003.stdout: "facts_cache_timeout": { 2024-05-09T06:37:24.194 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "facts_cache_timeout", 2024-05-09T06:37:24.195 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-05-09T06:37:24.195 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.195 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.195 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "60", 2024-05-09T06:37:24.195 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.195 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.195 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.195 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "seconds to cache host facts data", 2024-05-09T06:37:24.195 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.195 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.195 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.195 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.195 INFO:teuthology.orchestra.run.smithi003.stdout: "grafana_dashboards_path": { 2024-05-09T06:37:24.195 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "grafana_dashboards_path", 2024-05-09T06:37:24.195 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:24.196 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.196 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.196 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-05-09T06:37:24.196 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.196 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.196 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.196 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-05-09T06:37:24.196 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.196 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.196 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.196 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.196 INFO:teuthology.orchestra.run.smithi003.stdout: "host_check_interval": { 2024-05-09T06:37:24.196 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "host_check_interval", 2024-05-09T06:37:24.196 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-05-09T06:37:24.196 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.197 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.197 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "600", 2024-05-09T06:37:24.197 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.197 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.197 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.197 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "how frequently to perform a host check", 2024-05-09T06:37:24.197 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.197 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.197 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.197 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.197 INFO:teuthology.orchestra.run.smithi003.stdout: "hw_monitoring": { 2024-05-09T06:37:24.197 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "hw_monitoring", 2024-05-09T06:37:24.197 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:24.197 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.198 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.198 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-09T06:37:24.198 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.198 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.198 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.198 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-05-09T06:37:24.198 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.198 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.198 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.198 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.198 INFO:teuthology.orchestra.run.smithi003.stdout: "inventory_list_all": { 2024-05-09T06:37:24.198 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "inventory_list_all", 2024-05-09T06:37:24.198 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:24.198 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.199 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.199 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-09T06:37:24.199 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.199 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.199 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.199 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-05-09T06:37:24.199 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.199 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.199 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.199 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.199 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-05-09T06:37:24.199 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-05-09T06:37:24.199 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:24.199 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.199 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.200 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-09T06:37:24.200 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.200 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.200 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-09T06:37:24.200 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-09T06:37:24.200 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-09T06:37:24.200 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-09T06:37:24.200 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-09T06:37:24.200 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-09T06:37:24.200 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-09T06:37:24.200 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-09T06:37:24.200 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.200 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.201 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.201 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.201 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.201 INFO:teuthology.orchestra.run.smithi003.stdout: "log_refresh_metadata": { 2024-05-09T06:37:24.201 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_refresh_metadata", 2024-05-09T06:37:24.201 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:24.201 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.201 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.201 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-09T06:37:24.201 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.201 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.201 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.201 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-05-09T06:37:24.201 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.202 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.202 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.202 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.202 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-05-09T06:37:24.202 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-05-09T06:37:24.202 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:24.202 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.202 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.202 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-05-09T06:37:24.202 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.202 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.202 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.202 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-05-09T06:37:24.202 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.203 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.203 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.203 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.203 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-05-09T06:37:24.203 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-05-09T06:37:24.203 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:24.203 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.203 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.203 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-05-09T06:37:24.203 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.203 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.203 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-09T06:37:24.203 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-09T06:37:24.203 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-09T06:37:24.204 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-09T06:37:24.204 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-09T06:37:24.204 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-09T06:37:24.204 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-09T06:37:24.204 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-09T06:37:24.204 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.204 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.204 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.204 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.204 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.204 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-05-09T06:37:24.204 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-05-09T06:37:24.204 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:24.204 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.204 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.205 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-09T06:37:24.205 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.205 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.205 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.205 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.205 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.205 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.205 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.205 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.205 INFO:teuthology.orchestra.run.smithi003.stdout: "manage_etc_ceph_ceph_conf": { 2024-05-09T06:37:24.205 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-05-09T06:37:24.205 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:24.205 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.206 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.206 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-09T06:37:24.206 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.206 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.206 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.206 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-05-09T06:37:24.206 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.206 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.206 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.206 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.206 INFO:teuthology.orchestra.run.smithi003.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-05-09T06:37:24.206 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-05-09T06:37:24.206 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:24.206 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.207 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.207 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "*", 2024-05-09T06:37:24.207 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.207 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.207 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.207 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-05-09T06:37:24.207 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.207 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.207 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.207 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.207 INFO:teuthology.orchestra.run.smithi003.stdout: "max_count_per_host": { 2024-05-09T06:37:24.207 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "max_count_per_host", 2024-05-09T06:37:24.207 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-09T06:37:24.207 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.208 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.208 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "10", 2024-05-09T06:37:24.208 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.208 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.208 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.208 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "max number of daemons per service per host", 2024-05-09T06:37:24.208 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.208 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.208 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.208 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.208 INFO:teuthology.orchestra.run.smithi003.stdout: "max_osd_draining_count": { 2024-05-09T06:37:24.208 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "max_osd_draining_count", 2024-05-09T06:37:24.208 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-09T06:37:24.209 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.209 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.209 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "10", 2024-05-09T06:37:24.209 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.209 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.209 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.209 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-05-09T06:37:24.209 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.209 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.209 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.209 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.209 INFO:teuthology.orchestra.run.smithi003.stdout: "migration_current": { 2024-05-09T06:37:24.209 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "migration_current", 2024-05-09T06:37:24.209 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-09T06:37:24.210 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.210 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.210 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-09T06:37:24.210 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.210 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.210 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.210 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "internal - do not modify", 2024-05-09T06:37:24.210 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.210 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.210 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.210 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.210 INFO:teuthology.orchestra.run.smithi003.stdout: "mode": { 2024-05-09T06:37:24.210 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "mode", 2024-05-09T06:37:24.210 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:24.211 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.211 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.211 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "root", 2024-05-09T06:37:24.211 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.211 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.211 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-09T06:37:24.211 INFO:teuthology.orchestra.run.smithi003.stdout: "cephadm-package", 2024-05-09T06:37:24.211 INFO:teuthology.orchestra.run.smithi003.stdout: "root" 2024-05-09T06:37:24.211 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-09T06:37:24.211 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "mode for remote execution of cephadm", 2024-05-09T06:37:24.211 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.211 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.211 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.212 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.212 INFO:teuthology.orchestra.run.smithi003.stdout: "oob_default_addr": { 2024-05-09T06:37:24.212 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "oob_default_addr", 2024-05-09T06:37:24.212 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:24.212 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.212 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.212 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "169.254.1.1", 2024-05-09T06:37:24.212 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.212 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.212 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.212 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Default address for RedFish API (oob management).", 2024-05-09T06:37:24.212 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.212 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.213 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.213 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.213 INFO:teuthology.orchestra.run.smithi003.stdout: "prometheus_alerts_path": { 2024-05-09T06:37:24.213 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "prometheus_alerts_path", 2024-05-09T06:37:24.213 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:24.213 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.213 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.213 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-05-09T06:37:24.213 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.213 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.213 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.213 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-05-09T06:37:24.213 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.214 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.214 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.214 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.214 INFO:teuthology.orchestra.run.smithi003.stdout: "registry_insecure": { 2024-05-09T06:37:24.214 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "registry_insecure", 2024-05-09T06:37:24.214 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:24.214 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.214 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.214 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-09T06:37:24.214 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.214 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.214 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.214 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-05-09T06:37:24.214 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.215 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.215 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.215 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.215 INFO:teuthology.orchestra.run.smithi003.stdout: "registry_password": { 2024-05-09T06:37:24.215 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "registry_password", 2024-05-09T06:37:24.215 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:24.215 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.215 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.215 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-09T06:37:24.215 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.215 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.215 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.216 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-05-09T06:37:24.216 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.216 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.216 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.216 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.216 INFO:teuthology.orchestra.run.smithi003.stdout: "registry_url": { 2024-05-09T06:37:24.216 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "registry_url", 2024-05-09T06:37:24.216 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:24.216 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.216 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.216 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-09T06:37:24.216 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.216 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.216 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.217 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-05-09T06:37:24.217 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.217 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.217 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.217 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.217 INFO:teuthology.orchestra.run.smithi003.stdout: "registry_username": { 2024-05-09T06:37:24.217 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "registry_username", 2024-05-09T06:37:24.217 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:24.217 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.217 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.217 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-09T06:37:24.217 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.217 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.217 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.218 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-05-09T06:37:24.218 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.218 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.218 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.218 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.218 INFO:teuthology.orchestra.run.smithi003.stdout: "secure_monitoring_stack": { 2024-05-09T06:37:24.218 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "secure_monitoring_stack", 2024-05-09T06:37:24.218 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:24.218 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.218 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.218 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-09T06:37:24.218 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.218 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.218 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.218 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-05-09T06:37:24.219 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.219 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.219 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.219 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.219 INFO:teuthology.orchestra.run.smithi003.stdout: "service_discovery_port": { 2024-05-09T06:37:24.219 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "service_discovery_port", 2024-05-09T06:37:24.219 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-09T06:37:24.219 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.219 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.219 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "8765", 2024-05-09T06:37:24.219 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.219 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.219 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.220 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "cephadm service discovery port", 2024-05-09T06:37:24.220 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.220 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.220 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.220 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.220 INFO:teuthology.orchestra.run.smithi003.stdout: "ssh_config_file": { 2024-05-09T06:37:24.220 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "ssh_config_file", 2024-05-09T06:37:24.220 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:24.220 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.220 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.220 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-09T06:37:24.220 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.220 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.221 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.221 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-05-09T06:37:24.221 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.221 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.221 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.221 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.221 INFO:teuthology.orchestra.run.smithi003.stdout: "use_agent": { 2024-05-09T06:37:24.221 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "use_agent", 2024-05-09T06:37:24.221 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:24.221 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.221 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.221 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-09T06:37:24.222 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.222 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.222 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.222 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-05-09T06:37:24.222 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.222 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.222 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.222 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.222 INFO:teuthology.orchestra.run.smithi003.stdout: "use_repo_digest": { 2024-05-09T06:37:24.222 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "use_repo_digest", 2024-05-09T06:37:24.223 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:24.223 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.223 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.223 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-05-09T06:37:24.223 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.223 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.223 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.223 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-05-09T06:37:24.223 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.223 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.223 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.223 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.223 INFO:teuthology.orchestra.run.smithi003.stdout: "warn_on_failed_host_check": { 2024-05-09T06:37:24.223 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "warn_on_failed_host_check", 2024-05-09T06:37:24.224 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:24.224 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.224 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.224 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-05-09T06:37:24.224 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.224 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.224 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.224 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "raise a health warning if the host check fails", 2024-05-09T06:37:24.224 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.224 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.224 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.224 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.224 INFO:teuthology.orchestra.run.smithi003.stdout: "warn_on_stray_daemons": { 2024-05-09T06:37:24.224 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "warn_on_stray_daemons", 2024-05-09T06:37:24.224 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:24.224 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.225 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.225 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-05-09T06:37:24.225 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.225 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.225 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.225 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-05-09T06:37:24.225 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.225 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.225 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.225 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.225 INFO:teuthology.orchestra.run.smithi003.stdout: "warn_on_stray_hosts": { 2024-05-09T06:37:24.225 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "warn_on_stray_hosts", 2024-05-09T06:37:24.225 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:24.226 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.226 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.226 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-05-09T06:37:24.226 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.226 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.226 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.226 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-05-09T06:37:24.226 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.226 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.226 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.226 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-09T06:37:24.226 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-09T06:37:24.226 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.227 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-09T06:37:24.227 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "crash", 2024-05-09T06:37:24.227 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-05-09T06:37:24.227 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-05-09T06:37:24.227 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-05-09T06:37:24.227 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-05-09T06:37:24.227 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-05-09T06:37:24.227 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:24.227 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.227 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.227 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-09T06:37:24.227 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.227 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.227 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-09T06:37:24.227 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-09T06:37:24.227 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-09T06:37:24.228 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-09T06:37:24.230 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-09T06:37:24.231 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-09T06:37:24.231 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-09T06:37:24.231 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-09T06:37:24.231 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.231 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.231 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.231 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.231 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.231 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-05-09T06:37:24.231 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-05-09T06:37:24.231 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:24.231 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.231 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.231 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-09T06:37:24.232 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.232 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.232 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.232 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.232 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.232 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.232 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.232 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.232 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-05-09T06:37:24.232 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-05-09T06:37:24.232 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:24.232 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.232 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.232 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-05-09T06:37:24.232 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.233 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.233 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-09T06:37:24.233 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-09T06:37:24.233 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-09T06:37:24.233 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-09T06:37:24.233 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-09T06:37:24.233 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-09T06:37:24.233 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-09T06:37:24.233 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-09T06:37:24.233 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.233 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.233 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.234 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.234 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.234 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-05-09T06:37:24.234 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-05-09T06:37:24.234 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:24.234 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.234 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.234 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-09T06:37:24.234 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.234 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.234 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.234 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.234 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.234 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.235 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.235 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.235 INFO:teuthology.orchestra.run.smithi003.stdout: "retain_interval": { 2024-05-09T06:37:24.235 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "retain_interval", 2024-05-09T06:37:24.235 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-05-09T06:37:24.235 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.235 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.235 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "31536000", 2024-05-09T06:37:24.235 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.235 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.235 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.235 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "how long to retain crashes before pruning them", 2024-05-09T06:37:24.235 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.235 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.235 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.235 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.235 INFO:teuthology.orchestra.run.smithi003.stdout: "warn_recent_interval": { 2024-05-09T06:37:24.236 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "warn_recent_interval", 2024-05-09T06:37:24.236 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-05-09T06:37:24.236 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.236 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.236 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "1209600", 2024-05-09T06:37:24.236 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.236 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.236 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.236 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "time interval in which to warn about recent crashes", 2024-05-09T06:37:24.236 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.236 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.236 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.236 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-09T06:37:24.236 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-09T06:37:24.236 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.236 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-09T06:37:24.237 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "dashboard", 2024-05-09T06:37:24.237 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-05-09T06:37:24.237 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-05-09T06:37:24.237 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-05-09T06:37:24.237 INFO:teuthology.orchestra.run.smithi003.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-05-09T06:37:24.237 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-05-09T06:37:24.237 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-09T06:37:24.237 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.237 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.237 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "10", 2024-05-09T06:37:24.237 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.237 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.237 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.237 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.238 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.238 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.238 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.238 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.238 INFO:teuthology.orchestra.run.smithi003.stdout: "ALERTMANAGER_API_HOST": { 2024-05-09T06:37:24.238 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "ALERTMANAGER_API_HOST", 2024-05-09T06:37:24.238 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:24.238 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.238 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.238 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-09T06:37:24.238 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.238 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.238 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.238 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.239 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.239 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.239 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.239 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.239 INFO:teuthology.orchestra.run.smithi003.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-05-09T06:37:24.239 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-05-09T06:37:24.239 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:24.239 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.239 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.239 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-05-09T06:37:24.239 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.239 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.239 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.239 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.239 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.240 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.240 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.240 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.240 INFO:teuthology.orchestra.run.smithi003.stdout: "AUDIT_API_ENABLED": { 2024-05-09T06:37:24.240 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "AUDIT_API_ENABLED", 2024-05-09T06:37:24.240 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:24.240 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.240 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.240 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-09T06:37:24.240 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.240 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.240 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.240 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.240 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.240 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.240 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.241 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.241 INFO:teuthology.orchestra.run.smithi003.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-05-09T06:37:24.241 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-05-09T06:37:24.241 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:24.241 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.241 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.241 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-05-09T06:37:24.241 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.241 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.241 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.241 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.241 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.241 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.241 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.242 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.242 INFO:teuthology.orchestra.run.smithi003.stdout: "ENABLE_BROWSABLE_API": { 2024-05-09T06:37:24.242 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "ENABLE_BROWSABLE_API", 2024-05-09T06:37:24.242 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:24.242 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.242 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.242 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-05-09T06:37:24.242 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.242 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.242 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.242 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.242 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.242 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.242 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.242 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.243 INFO:teuthology.orchestra.run.smithi003.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-05-09T06:37:24.243 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-05-09T06:37:24.243 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:24.243 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.243 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.243 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-05-09T06:37:24.243 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.243 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.243 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.243 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.243 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.243 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.243 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.243 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.243 INFO:teuthology.orchestra.run.smithi003.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-05-09T06:37:24.244 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-05-09T06:37:24.244 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:24.244 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.244 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.244 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-05-09T06:37:24.244 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.244 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.244 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.244 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.244 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.244 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.244 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.244 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.244 INFO:teuthology.orchestra.run.smithi003.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-05-09T06:37:24.245 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-05-09T06:37:24.245 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:24.245 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.245 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.245 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-05-09T06:37:24.245 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.245 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.245 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.245 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.245 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.245 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.245 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.245 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.246 INFO:teuthology.orchestra.run.smithi003.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-05-09T06:37:24.246 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-05-09T06:37:24.246 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:24.246 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.246 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.246 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-05-09T06:37:24.246 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.246 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.246 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.246 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.246 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.246 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.246 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.246 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.247 INFO:teuthology.orchestra.run.smithi003.stdout: "FEATURE_TOGGLE_NFS": { 2024-05-09T06:37:24.247 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-05-09T06:37:24.247 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:24.247 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.247 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.247 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-05-09T06:37:24.247 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.247 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.247 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.247 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.247 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.247 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.247 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.247 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.248 INFO:teuthology.orchestra.run.smithi003.stdout: "FEATURE_TOGGLE_RBD": { 2024-05-09T06:37:24.248 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-05-09T06:37:24.248 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:24.248 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.248 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.248 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-05-09T06:37:24.248 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.248 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.248 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.248 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.248 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.248 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.248 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.248 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.248 INFO:teuthology.orchestra.run.smithi003.stdout: "FEATURE_TOGGLE_RGW": { 2024-05-09T06:37:24.249 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-05-09T06:37:24.249 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:24.249 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.249 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.249 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-05-09T06:37:24.249 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.249 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.249 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.249 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.249 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.249 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.249 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.249 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.249 INFO:teuthology.orchestra.run.smithi003.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-05-09T06:37:24.249 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-05-09T06:37:24.250 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:24.250 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.250 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.250 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-09T06:37:24.250 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.250 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.250 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.250 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.250 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.250 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.250 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.250 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.250 INFO:teuthology.orchestra.run.smithi003.stdout: "GRAFANA_API_PASSWORD": { 2024-05-09T06:37:24.250 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "GRAFANA_API_PASSWORD", 2024-05-09T06:37:24.251 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:24.251 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.251 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.251 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "admin", 2024-05-09T06:37:24.251 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.251 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.251 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.251 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.251 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.251 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.251 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.251 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.251 INFO:teuthology.orchestra.run.smithi003.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-05-09T06:37:24.252 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-05-09T06:37:24.252 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:24.252 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.252 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.252 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-05-09T06:37:24.252 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.252 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.252 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.252 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.252 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.252 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.252 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.252 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.252 INFO:teuthology.orchestra.run.smithi003.stdout: "GRAFANA_API_URL": { 2024-05-09T06:37:24.253 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "GRAFANA_API_URL", 2024-05-09T06:37:24.253 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:24.253 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.253 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.253 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-09T06:37:24.253 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.253 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.253 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.253 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.253 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.253 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.253 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.253 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.253 INFO:teuthology.orchestra.run.smithi003.stdout: "GRAFANA_API_USERNAME": { 2024-05-09T06:37:24.253 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "GRAFANA_API_USERNAME", 2024-05-09T06:37:24.254 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:24.254 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.254 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.254 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "admin", 2024-05-09T06:37:24.254 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.254 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.254 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.254 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.254 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.254 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.254 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.254 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.254 INFO:teuthology.orchestra.run.smithi003.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-05-09T06:37:24.254 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-05-09T06:37:24.254 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:24.255 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.255 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.255 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-09T06:37:24.255 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.255 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.255 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.255 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.255 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.255 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.255 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.255 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.255 INFO:teuthology.orchestra.run.smithi003.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-05-09T06:37:24.255 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-05-09T06:37:24.255 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:24.255 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.256 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.256 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-09T06:37:24.256 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.256 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.256 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.256 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.256 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.256 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.256 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.256 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.256 INFO:teuthology.orchestra.run.smithi003.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-05-09T06:37:24.256 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-05-09T06:37:24.256 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:24.256 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.256 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.256 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-05-09T06:37:24.257 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.257 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.257 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.257 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.257 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.257 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.257 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.257 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.257 INFO:teuthology.orchestra.run.smithi003.stdout: "ISSUE_TRACKER_API_KEY": { 2024-05-09T06:37:24.257 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-05-09T06:37:24.257 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:24.257 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.257 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.257 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-09T06:37:24.258 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.258 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.258 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.258 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.258 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.258 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.258 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.258 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.258 INFO:teuthology.orchestra.run.smithi003.stdout: "PROMETHEUS_API_HOST": { 2024-05-09T06:37:24.258 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PROMETHEUS_API_HOST", 2024-05-09T06:37:24.258 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:24.258 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.258 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.258 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-09T06:37:24.258 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.259 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.259 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.259 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.259 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.259 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.259 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.259 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.259 INFO:teuthology.orchestra.run.smithi003.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-05-09T06:37:24.259 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-05-09T06:37:24.259 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:24.259 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.259 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.259 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-05-09T06:37:24.259 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.259 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.259 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.260 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.260 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.260 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.260 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.260 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.260 INFO:teuthology.orchestra.run.smithi003.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-05-09T06:37:24.260 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-05-09T06:37:24.260 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:24.260 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.260 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.260 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-09T06:37:24.260 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.260 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.260 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.260 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.260 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.260 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.261 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.261 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.261 INFO:teuthology.orchestra.run.smithi003.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-05-09T06:37:24.261 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-05-09T06:37:24.261 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:24.261 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.261 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.261 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-09T06:37:24.261 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.261 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.261 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.261 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.261 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.261 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.261 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.262 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.262 INFO:teuthology.orchestra.run.smithi003.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-05-09T06:37:24.262 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-05-09T06:37:24.262 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:24.262 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.262 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.262 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-05-09T06:37:24.262 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.262 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.262 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.262 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.262 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.262 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.262 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.262 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.263 INFO:teuthology.orchestra.run.smithi003.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-05-09T06:37:24.263 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-05-09T06:37:24.263 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:24.263 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.263 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.263 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-05-09T06:37:24.263 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.263 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.263 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.263 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.263 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.263 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.264 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.264 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.264 INFO:teuthology.orchestra.run.smithi003.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-05-09T06:37:24.264 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-05-09T06:37:24.264 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:24.264 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.264 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.264 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-09T06:37:24.264 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.264 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.264 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.264 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.264 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.265 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.265 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.265 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.265 INFO:teuthology.orchestra.run.smithi003.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-05-09T06:37:24.265 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-05-09T06:37:24.265 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:24.265 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.265 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.265 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-09T06:37:24.265 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.265 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.265 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.265 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.265 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.265 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.266 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.266 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.266 INFO:teuthology.orchestra.run.smithi003.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-05-09T06:37:24.266 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-05-09T06:37:24.266 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:24.266 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.266 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.266 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-09T06:37:24.266 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.266 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.266 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.266 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.266 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.266 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.266 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.267 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.267 INFO:teuthology.orchestra.run.smithi003.stdout: "PWD_POLICY_ENABLED": { 2024-05-09T06:37:24.267 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PWD_POLICY_ENABLED", 2024-05-09T06:37:24.267 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:24.267 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.267 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.267 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-05-09T06:37:24.267 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.267 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.267 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.267 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.267 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.267 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.267 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.267 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.268 INFO:teuthology.orchestra.run.smithi003.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-05-09T06:37:24.268 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-05-09T06:37:24.268 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:24.268 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.268 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.268 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-05-09T06:37:24.268 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.268 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.268 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.268 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.268 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.268 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.268 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.268 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.268 INFO:teuthology.orchestra.run.smithi003.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-05-09T06:37:24.269 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-05-09T06:37:24.269 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-09T06:37:24.269 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.269 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.269 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "10", 2024-05-09T06:37:24.269 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.269 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.269 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.269 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.269 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.269 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.269 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.269 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.269 INFO:teuthology.orchestra.run.smithi003.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-05-09T06:37:24.269 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-05-09T06:37:24.269 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-09T06:37:24.270 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.270 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.270 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "8", 2024-05-09T06:37:24.270 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.270 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.270 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.270 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.270 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.270 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.270 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.270 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.271 INFO:teuthology.orchestra.run.smithi003.stdout: "REST_REQUESTS_TIMEOUT": { 2024-05-09T06:37:24.271 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-05-09T06:37:24.271 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-09T06:37:24.271 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.271 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.271 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "45", 2024-05-09T06:37:24.271 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.271 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.271 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.271 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.271 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.271 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.271 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.272 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.272 INFO:teuthology.orchestra.run.smithi003.stdout: "RGW_API_ACCESS_KEY": { 2024-05-09T06:37:24.272 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "RGW_API_ACCESS_KEY", 2024-05-09T06:37:24.272 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:24.272 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.272 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.272 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-09T06:37:24.272 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.272 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.272 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.272 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.272 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.272 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.272 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.272 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.273 INFO:teuthology.orchestra.run.smithi003.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-05-09T06:37:24.273 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-05-09T06:37:24.273 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:24.273 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.273 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.273 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "admin", 2024-05-09T06:37:24.273 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.273 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.273 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.273 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.273 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.273 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.273 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.273 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.273 INFO:teuthology.orchestra.run.smithi003.stdout: "RGW_API_SECRET_KEY": { 2024-05-09T06:37:24.274 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "RGW_API_SECRET_KEY", 2024-05-09T06:37:24.274 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:24.274 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.274 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.274 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-09T06:37:24.274 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.274 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.274 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.274 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.274 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.274 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.274 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.274 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.274 INFO:teuthology.orchestra.run.smithi003.stdout: "RGW_API_SSL_VERIFY": { 2024-05-09T06:37:24.275 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "RGW_API_SSL_VERIFY", 2024-05-09T06:37:24.275 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:24.275 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.275 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.275 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-05-09T06:37:24.275 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.275 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.275 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.275 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.275 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.275 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.275 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.275 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.275 INFO:teuthology.orchestra.run.smithi003.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-05-09T06:37:24.276 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-05-09T06:37:24.276 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-09T06:37:24.276 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.276 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.276 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-05-09T06:37:24.276 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.276 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.276 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.276 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.276 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.276 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.276 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.276 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.276 INFO:teuthology.orchestra.run.smithi003.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-05-09T06:37:24.277 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-05-09T06:37:24.277 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-09T06:37:24.277 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.277 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.277 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "10", 2024-05-09T06:37:24.277 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.277 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.277 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.277 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.277 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.277 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.277 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.277 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.278 INFO:teuthology.orchestra.run.smithi003.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-05-09T06:37:24.278 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-05-09T06:37:24.278 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-09T06:37:24.278 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.278 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.278 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "5", 2024-05-09T06:37:24.278 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.278 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.278 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.278 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.278 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.278 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.278 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.278 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.279 INFO:teuthology.orchestra.run.smithi003.stdout: "cross_origin_url": { 2024-05-09T06:37:24.279 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "cross_origin_url", 2024-05-09T06:37:24.279 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:24.279 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.279 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.279 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-09T06:37:24.279 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.279 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.279 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.279 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.279 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.279 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.280 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.280 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.280 INFO:teuthology.orchestra.run.smithi003.stdout: "crt_file": { 2024-05-09T06:37:24.280 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "crt_file", 2024-05-09T06:37:24.280 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:24.280 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.280 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.280 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-09T06:37:24.280 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.280 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.280 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.280 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.280 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.281 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.281 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.281 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.281 INFO:teuthology.orchestra.run.smithi003.stdout: "debug": { 2024-05-09T06:37:24.281 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "debug", 2024-05-09T06:37:24.281 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:24.281 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.281 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.281 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-09T06:37:24.281 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.281 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.281 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.281 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Enable/disable debug options", 2024-05-09T06:37:24.281 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.282 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.282 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.282 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.282 INFO:teuthology.orchestra.run.smithi003.stdout: "jwt_token_ttl": { 2024-05-09T06:37:24.282 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "jwt_token_ttl", 2024-05-09T06:37:24.282 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-09T06:37:24.282 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.282 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.282 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "28800", 2024-05-09T06:37:24.282 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.282 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.282 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.282 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.283 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.283 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.283 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.283 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.283 INFO:teuthology.orchestra.run.smithi003.stdout: "key_file": { 2024-05-09T06:37:24.283 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "key_file", 2024-05-09T06:37:24.283 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:24.283 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.283 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.283 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-09T06:37:24.283 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.283 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.283 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.283 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.283 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.284 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.284 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.284 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.284 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-05-09T06:37:24.284 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-05-09T06:37:24.284 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:24.284 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.284 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.284 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-09T06:37:24.284 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.284 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.284 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-09T06:37:24.284 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-09T06:37:24.284 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-09T06:37:24.284 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-09T06:37:24.285 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-09T06:37:24.285 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-09T06:37:24.285 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-09T06:37:24.285 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-09T06:37:24.285 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.285 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.285 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.285 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.285 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.285 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-05-09T06:37:24.285 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-05-09T06:37:24.285 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:24.285 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.285 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.285 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-09T06:37:24.286 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.286 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.286 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.286 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.286 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.286 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.286 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.286 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.286 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-05-09T06:37:24.286 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-05-09T06:37:24.286 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:24.286 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.286 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.286 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-05-09T06:37:24.287 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.287 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.287 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-09T06:37:24.287 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-09T06:37:24.287 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-09T06:37:24.287 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-09T06:37:24.287 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-09T06:37:24.287 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-09T06:37:24.287 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-09T06:37:24.287 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-09T06:37:24.287 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.287 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.287 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.287 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.287 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.288 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-05-09T06:37:24.288 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-05-09T06:37:24.288 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:24.288 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.288 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.288 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-09T06:37:24.288 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.288 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.288 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.288 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.288 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.288 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.288 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.288 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.289 INFO:teuthology.orchestra.run.smithi003.stdout: "motd": { 2024-05-09T06:37:24.289 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "motd", 2024-05-09T06:37:24.289 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:24.289 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.289 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.289 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-09T06:37:24.289 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.289 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.289 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.289 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "The message of the day", 2024-05-09T06:37:24.289 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.289 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.289 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.289 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.289 INFO:teuthology.orchestra.run.smithi003.stdout: "redirect_resolve_ip_addr": { 2024-05-09T06:37:24.289 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "redirect_resolve_ip_addr", 2024-05-09T06:37:24.290 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:24.290 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.290 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.290 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-09T06:37:24.290 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.290 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.290 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.290 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.290 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.290 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.290 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.290 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.290 INFO:teuthology.orchestra.run.smithi003.stdout: "server_addr": { 2024-05-09T06:37:24.291 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "server_addr", 2024-05-09T06:37:24.291 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:24.291 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.291 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.291 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "::", 2024-05-09T06:37:24.291 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.291 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.291 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.291 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.291 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.291 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.291 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.291 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.291 INFO:teuthology.orchestra.run.smithi003.stdout: "server_port": { 2024-05-09T06:37:24.291 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "server_port", 2024-05-09T06:37:24.292 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-09T06:37:24.292 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.292 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.292 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "8080", 2024-05-09T06:37:24.292 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.292 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.292 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.292 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.292 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.292 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.292 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.292 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.292 INFO:teuthology.orchestra.run.smithi003.stdout: "ssl": { 2024-05-09T06:37:24.292 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "ssl", 2024-05-09T06:37:24.293 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:24.293 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.293 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.293 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-05-09T06:37:24.293 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.293 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.293 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.293 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.293 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.293 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.293 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.293 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.293 INFO:teuthology.orchestra.run.smithi003.stdout: "ssl_server_port": { 2024-05-09T06:37:24.293 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "ssl_server_port", 2024-05-09T06:37:24.293 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-09T06:37:24.293 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.294 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.294 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "8443", 2024-05-09T06:37:24.294 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.294 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.294 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.294 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.294 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.294 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.294 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.294 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.294 INFO:teuthology.orchestra.run.smithi003.stdout: "standby_behaviour": { 2024-05-09T06:37:24.294 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "standby_behaviour", 2024-05-09T06:37:24.294 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:24.295 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.295 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.295 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "redirect", 2024-05-09T06:37:24.295 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.295 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.295 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-09T06:37:24.295 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-09T06:37:24.295 INFO:teuthology.orchestra.run.smithi003.stdout: "redirect" 2024-05-09T06:37:24.295 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-09T06:37:24.295 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.295 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.295 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.295 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.295 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.295 INFO:teuthology.orchestra.run.smithi003.stdout: "standby_error_status_code": { 2024-05-09T06:37:24.296 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "standby_error_status_code", 2024-05-09T06:37:24.296 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-09T06:37:24.296 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.296 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.296 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "500", 2024-05-09T06:37:24.296 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "400", 2024-05-09T06:37:24.296 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "599", 2024-05-09T06:37:24.296 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.296 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.296 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.296 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.296 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.296 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.297 INFO:teuthology.orchestra.run.smithi003.stdout: "url_prefix": { 2024-05-09T06:37:24.297 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "url_prefix", 2024-05-09T06:37:24.297 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:24.297 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.297 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.297 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-09T06:37:24.297 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.297 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.297 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.297 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.297 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.297 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.297 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.297 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-09T06:37:24.298 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-09T06:37:24.298 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.298 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-09T06:37:24.298 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "devicehealth", 2024-05-09T06:37:24.298 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-05-09T06:37:24.298 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-05-09T06:37:24.298 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-05-09T06:37:24.298 INFO:teuthology.orchestra.run.smithi003.stdout: "enable_monitoring": { 2024-05-09T06:37:24.298 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "enable_monitoring", 2024-05-09T06:37:24.298 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:24.298 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.298 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.298 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-05-09T06:37:24.299 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.299 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.299 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.299 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "monitor device health metrics", 2024-05-09T06:37:24.299 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.299 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.299 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.299 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.299 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-05-09T06:37:24.299 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-05-09T06:37:24.299 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:24.299 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.299 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.299 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-09T06:37:24.299 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.300 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.300 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-09T06:37:24.300 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-09T06:37:24.300 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-09T06:37:24.300 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-09T06:37:24.300 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-09T06:37:24.300 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-09T06:37:24.300 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-09T06:37:24.300 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-09T06:37:24.300 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.300 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.300 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.300 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.301 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.301 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-05-09T06:37:24.301 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-05-09T06:37:24.301 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:24.301 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.301 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.301 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-09T06:37:24.301 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.301 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.301 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.301 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.302 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.302 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.302 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.302 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.302 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-05-09T06:37:24.302 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-05-09T06:37:24.302 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:24.302 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.302 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.302 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-05-09T06:37:24.302 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.302 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.302 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-09T06:37:24.302 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-09T06:37:24.303 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-09T06:37:24.303 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-09T06:37:24.303 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-09T06:37:24.303 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-09T06:37:24.303 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-09T06:37:24.303 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-09T06:37:24.303 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.303 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.303 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.303 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.303 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.303 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-05-09T06:37:24.303 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-05-09T06:37:24.303 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:24.303 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.304 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.304 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-09T06:37:24.304 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.304 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.304 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.304 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.304 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.304 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.304 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.304 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.304 INFO:teuthology.orchestra.run.smithi003.stdout: "mark_out_threshold": { 2024-05-09T06:37:24.304 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "mark_out_threshold", 2024-05-09T06:37:24.304 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-05-09T06:37:24.304 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.304 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.305 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "2419200", 2024-05-09T06:37:24.305 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.305 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.305 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.305 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-05-09T06:37:24.305 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.305 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.305 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.305 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.305 INFO:teuthology.orchestra.run.smithi003.stdout: "pool_name": { 2024-05-09T06:37:24.305 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "pool_name", 2024-05-09T06:37:24.305 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:24.305 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.305 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.305 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "device_health_metrics", 2024-05-09T06:37:24.305 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.305 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.306 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.306 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "name of pool in which to store device health metrics", 2024-05-09T06:37:24.306 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.306 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.306 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.306 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.306 INFO:teuthology.orchestra.run.smithi003.stdout: "retention_period": { 2024-05-09T06:37:24.306 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "retention_period", 2024-05-09T06:37:24.306 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-05-09T06:37:24.306 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.306 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.306 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "15552000", 2024-05-09T06:37:24.306 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.306 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.306 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.307 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "how long to retain device health metrics", 2024-05-09T06:37:24.307 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.307 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.307 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.307 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.307 INFO:teuthology.orchestra.run.smithi003.stdout: "scrape_frequency": { 2024-05-09T06:37:24.307 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "scrape_frequency", 2024-05-09T06:37:24.307 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-05-09T06:37:24.307 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.307 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.307 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "86400", 2024-05-09T06:37:24.307 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.307 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.307 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.307 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "how frequently to scrape device health metrics", 2024-05-09T06:37:24.307 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.308 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.308 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.308 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.308 INFO:teuthology.orchestra.run.smithi003.stdout: "self_heal": { 2024-05-09T06:37:24.308 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "self_heal", 2024-05-09T06:37:24.308 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:24.308 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.308 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.308 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-05-09T06:37:24.308 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.308 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.308 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.308 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-05-09T06:37:24.308 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.308 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.308 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.308 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.309 INFO:teuthology.orchestra.run.smithi003.stdout: "sleep_interval": { 2024-05-09T06:37:24.309 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sleep_interval", 2024-05-09T06:37:24.309 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-05-09T06:37:24.309 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.309 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.309 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "600", 2024-05-09T06:37:24.309 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.309 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.309 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.309 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "how frequently to wake up and check device health", 2024-05-09T06:37:24.309 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.309 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.309 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.309 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.309 INFO:teuthology.orchestra.run.smithi003.stdout: "warn_threshold": { 2024-05-09T06:37:24.309 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "warn_threshold", 2024-05-09T06:37:24.310 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-05-09T06:37:24.310 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.310 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.310 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "7257600", 2024-05-09T06:37:24.310 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.310 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.310 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.310 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-05-09T06:37:24.310 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.310 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.310 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.310 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-09T06:37:24.310 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-09T06:37:24.311 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.311 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-09T06:37:24.311 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "diskprediction_local", 2024-05-09T06:37:24.311 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-05-09T06:37:24.311 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-05-09T06:37:24.311 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-05-09T06:37:24.311 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-05-09T06:37:24.311 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-05-09T06:37:24.311 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:24.311 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.311 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.311 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-09T06:37:24.311 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.311 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.312 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-09T06:37:24.312 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-09T06:37:24.312 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-09T06:37:24.312 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-09T06:37:24.312 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-09T06:37:24.312 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-09T06:37:24.312 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-09T06:37:24.312 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-09T06:37:24.312 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.312 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.312 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.312 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.312 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.312 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-05-09T06:37:24.313 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-05-09T06:37:24.313 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:24.313 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.313 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.313 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-09T06:37:24.313 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.313 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.313 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.313 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.313 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.313 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.313 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.313 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.313 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-05-09T06:37:24.313 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-05-09T06:37:24.314 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:24.314 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.314 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.314 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-05-09T06:37:24.314 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.314 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.314 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-09T06:37:24.314 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-09T06:37:24.314 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-09T06:37:24.314 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-09T06:37:24.314 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-09T06:37:24.314 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-09T06:37:24.314 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-09T06:37:24.314 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-09T06:37:24.314 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.315 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.315 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.315 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.315 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.315 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-05-09T06:37:24.315 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-05-09T06:37:24.315 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:24.315 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.315 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.315 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-09T06:37:24.315 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.315 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.315 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.315 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.315 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.315 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.315 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.315 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.316 INFO:teuthology.orchestra.run.smithi003.stdout: "predict_interval": { 2024-05-09T06:37:24.316 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "predict_interval", 2024-05-09T06:37:24.316 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:24.316 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.316 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.316 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "86400", 2024-05-09T06:37:24.316 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.316 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.316 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.316 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.316 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.316 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.316 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.316 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.316 INFO:teuthology.orchestra.run.smithi003.stdout: "predictor_model": { 2024-05-09T06:37:24.316 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "predictor_model", 2024-05-09T06:37:24.316 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:24.317 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.317 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.317 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "prophetstor", 2024-05-09T06:37:24.317 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.317 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.317 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.317 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.317 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.317 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.317 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.317 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.317 INFO:teuthology.orchestra.run.smithi003.stdout: "sleep_interval": { 2024-05-09T06:37:24.317 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sleep_interval", 2024-05-09T06:37:24.317 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:24.317 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.317 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.318 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "600", 2024-05-09T06:37:24.318 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.318 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.318 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.318 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.318 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.318 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.318 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.318 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-09T06:37:24.318 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-09T06:37:24.318 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.318 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-09T06:37:24.318 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "influx", 2024-05-09T06:37:24.318 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": false, 2024-05-09T06:37:24.318 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "influxdb python module not found", 2024-05-09T06:37:24.319 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-05-09T06:37:24.319 INFO:teuthology.orchestra.run.smithi003.stdout: "batch_size": { 2024-05-09T06:37:24.319 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "batch_size", 2024-05-09T06:37:24.319 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-09T06:37:24.319 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.319 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.319 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "5000", 2024-05-09T06:37:24.319 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.319 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.319 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.319 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-05-09T06:37:24.319 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.319 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.319 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.320 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.320 INFO:teuthology.orchestra.run.smithi003.stdout: "database": { 2024-05-09T06:37:24.320 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "database", 2024-05-09T06:37:24.320 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:24.320 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.320 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.320 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "ceph", 2024-05-09T06:37:24.320 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.320 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.320 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.321 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-05-09T06:37:24.321 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.321 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.321 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.321 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.321 INFO:teuthology.orchestra.run.smithi003.stdout: "hostname": { 2024-05-09T06:37:24.321 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "hostname", 2024-05-09T06:37:24.321 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:24.321 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.321 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.321 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-09T06:37:24.321 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.321 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.321 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.321 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "InfluxDB server hostname", 2024-05-09T06:37:24.322 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.322 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.322 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.322 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.322 INFO:teuthology.orchestra.run.smithi003.stdout: "interval": { 2024-05-09T06:37:24.322 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "interval", 2024-05-09T06:37:24.322 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-05-09T06:37:24.322 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.322 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.322 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "30", 2024-05-09T06:37:24.322 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "5", 2024-05-09T06:37:24.322 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.323 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.323 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-05-09T06:37:24.323 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.323 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.323 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.323 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.323 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-05-09T06:37:24.323 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-05-09T06:37:24.323 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:24.323 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.323 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.323 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-09T06:37:24.323 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.323 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.323 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-09T06:37:24.324 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-09T06:37:24.324 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-09T06:37:24.324 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-09T06:37:24.324 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-09T06:37:24.324 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-09T06:37:24.324 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-09T06:37:24.324 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-09T06:37:24.324 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.324 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.324 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.324 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.324 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.324 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-05-09T06:37:24.324 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-05-09T06:37:24.325 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:24.325 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.325 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.325 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-09T06:37:24.325 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.325 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.325 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.325 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.325 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.325 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.325 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.325 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.325 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-05-09T06:37:24.325 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-05-09T06:37:24.326 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:24.326 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.326 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.326 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-05-09T06:37:24.326 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.326 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.326 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-09T06:37:24.326 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-09T06:37:24.326 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-09T06:37:24.326 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-09T06:37:24.326 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-09T06:37:24.326 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-09T06:37:24.326 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-09T06:37:24.326 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-09T06:37:24.326 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.327 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.327 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.327 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.327 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.327 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-05-09T06:37:24.327 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-05-09T06:37:24.327 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:24.327 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.327 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.327 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-09T06:37:24.327 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.327 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.327 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.327 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.328 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.328 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.328 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.328 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.328 INFO:teuthology.orchestra.run.smithi003.stdout: "password": { 2024-05-09T06:37:24.328 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "password", 2024-05-09T06:37:24.328 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:24.328 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.328 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.328 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-09T06:37:24.328 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.328 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.328 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.328 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "password of InfluxDB server user", 2024-05-09T06:37:24.328 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.329 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.329 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.329 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.329 INFO:teuthology.orchestra.run.smithi003.stdout: "port": { 2024-05-09T06:37:24.329 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "port", 2024-05-09T06:37:24.329 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-09T06:37:24.329 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.329 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.329 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "8086", 2024-05-09T06:37:24.329 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.329 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.329 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.329 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "InfluxDB server port", 2024-05-09T06:37:24.329 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.329 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.330 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.330 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.330 INFO:teuthology.orchestra.run.smithi003.stdout: "ssl": { 2024-05-09T06:37:24.330 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "ssl", 2024-05-09T06:37:24.330 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:24.330 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.330 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.330 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "false", 2024-05-09T06:37:24.330 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.330 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.330 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.330 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-05-09T06:37:24.330 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.330 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.331 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.331 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.331 INFO:teuthology.orchestra.run.smithi003.stdout: "threads": { 2024-05-09T06:37:24.331 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "threads", 2024-05-09T06:37:24.331 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-09T06:37:24.331 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.331 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.331 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "5", 2024-05-09T06:37:24.331 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "1", 2024-05-09T06:37:24.331 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "32", 2024-05-09T06:37:24.331 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.331 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-05-09T06:37:24.331 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.331 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.331 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.331 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.332 INFO:teuthology.orchestra.run.smithi003.stdout: "username": { 2024-05-09T06:37:24.332 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "username", 2024-05-09T06:37:24.332 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:24.332 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.332 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.332 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-09T06:37:24.332 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.332 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.332 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.332 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "username of InfluxDB server user", 2024-05-09T06:37:24.332 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.332 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.332 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.332 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.333 INFO:teuthology.orchestra.run.smithi003.stdout: "verify_ssl": { 2024-05-09T06:37:24.333 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "verify_ssl", 2024-05-09T06:37:24.333 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:24.333 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.333 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.333 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "true", 2024-05-09T06:37:24.333 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.333 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.333 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.333 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-05-09T06:37:24.333 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.333 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.333 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.334 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-09T06:37:24.334 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-09T06:37:24.334 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.334 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-09T06:37:24.334 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "insights", 2024-05-09T06:37:24.334 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-05-09T06:37:24.334 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-05-09T06:37:24.334 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-05-09T06:37:24.334 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-05-09T06:37:24.334 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-05-09T06:37:24.334 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:24.334 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.334 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.334 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-09T06:37:24.335 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.335 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.335 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-09T06:37:24.335 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-09T06:37:24.335 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-09T06:37:24.335 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-09T06:37:24.335 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-09T06:37:24.335 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-09T06:37:24.335 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-09T06:37:24.335 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-09T06:37:24.335 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.335 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.335 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.335 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.335 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.335 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-05-09T06:37:24.335 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-05-09T06:37:24.336 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:24.336 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.336 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.336 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-09T06:37:24.336 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.336 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.336 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.336 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.336 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.336 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.336 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.336 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.336 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-05-09T06:37:24.336 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-05-09T06:37:24.337 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:24.337 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.337 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.337 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-05-09T06:37:24.337 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.337 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.337 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-09T06:37:24.337 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-09T06:37:24.337 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-09T06:37:24.337 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-09T06:37:24.337 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-09T06:37:24.337 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-09T06:37:24.337 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-09T06:37:24.337 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-09T06:37:24.338 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.338 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.338 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.338 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.338 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.338 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-05-09T06:37:24.338 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-05-09T06:37:24.338 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:24.338 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.338 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.338 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-09T06:37:24.338 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.338 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.338 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.338 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.339 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.339 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.339 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.339 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-09T06:37:24.339 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-09T06:37:24.339 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.339 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-09T06:37:24.339 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "iostat", 2024-05-09T06:37:24.339 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-05-09T06:37:24.339 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-05-09T06:37:24.339 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-05-09T06:37:24.339 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-05-09T06:37:24.339 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-05-09T06:37:24.339 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:24.340 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.340 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.340 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-09T06:37:24.340 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.340 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.340 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-09T06:37:24.340 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-09T06:37:24.340 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-09T06:37:24.340 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-09T06:37:24.340 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-09T06:37:24.340 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-09T06:37:24.340 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-09T06:37:24.341 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-09T06:37:24.341 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.341 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.341 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.341 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.341 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.341 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-05-09T06:37:24.341 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-05-09T06:37:24.341 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:24.341 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.341 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.341 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-09T06:37:24.341 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.342 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.342 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.342 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.342 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.342 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.342 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.342 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.342 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-05-09T06:37:24.342 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-05-09T06:37:24.342 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:24.342 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.342 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.342 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-05-09T06:37:24.342 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.343 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.343 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-09T06:37:24.343 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-09T06:37:24.343 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-09T06:37:24.343 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-09T06:37:24.343 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-09T06:37:24.343 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-09T06:37:24.343 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-09T06:37:24.343 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-09T06:37:24.343 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.343 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.343 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.343 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.344 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.344 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-05-09T06:37:24.344 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-05-09T06:37:24.344 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:24.344 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.344 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.344 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-09T06:37:24.344 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.344 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.344 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.344 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.344 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.344 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.345 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.345 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-09T06:37:24.345 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-09T06:37:24.345 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.345 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-09T06:37:24.345 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "localpool", 2024-05-09T06:37:24.345 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-05-09T06:37:24.345 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-05-09T06:37:24.345 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-05-09T06:37:24.345 INFO:teuthology.orchestra.run.smithi003.stdout: "failure_domain": { 2024-05-09T06:37:24.345 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "failure_domain", 2024-05-09T06:37:24.345 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:24.345 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.345 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.345 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "host", 2024-05-09T06:37:24.345 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.345 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.346 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.346 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "failure domain for any created local pool", 2024-05-09T06:37:24.346 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-05-09T06:37:24.346 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.346 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.346 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.346 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-05-09T06:37:24.346 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-05-09T06:37:24.346 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:24.346 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.346 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.346 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-09T06:37:24.346 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.346 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.346 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-09T06:37:24.347 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-09T06:37:24.347 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-09T06:37:24.347 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-09T06:37:24.347 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-09T06:37:24.347 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-09T06:37:24.347 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-09T06:37:24.347 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-09T06:37:24.347 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.347 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.347 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.347 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.347 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.347 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-05-09T06:37:24.347 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-05-09T06:37:24.347 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:24.348 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.348 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.348 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-09T06:37:24.348 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.348 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.348 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.348 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.348 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.348 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.348 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.348 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.348 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-05-09T06:37:24.348 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-05-09T06:37:24.348 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:24.348 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.348 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.348 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-05-09T06:37:24.349 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.349 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.349 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-09T06:37:24.349 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-09T06:37:24.349 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-09T06:37:24.349 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-09T06:37:24.349 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-09T06:37:24.349 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-09T06:37:24.349 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-09T06:37:24.349 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-09T06:37:24.349 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.349 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.349 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.349 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.349 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.349 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-05-09T06:37:24.350 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-05-09T06:37:24.350 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:24.350 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.350 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.350 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-09T06:37:24.350 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.350 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.350 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.350 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.350 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.350 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.350 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.350 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.350 INFO:teuthology.orchestra.run.smithi003.stdout: "min_size": { 2024-05-09T06:37:24.350 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "min_size", 2024-05-09T06:37:24.351 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-09T06:37:24.351 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.351 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.351 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-09T06:37:24.351 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.351 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.351 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.351 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "default min_size for any created local pool", 2024-05-09T06:37:24.351 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-05-09T06:37:24.351 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.351 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.351 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.351 INFO:teuthology.orchestra.run.smithi003.stdout: "num_rep": { 2024-05-09T06:37:24.351 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "num_rep", 2024-05-09T06:37:24.352 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-09T06:37:24.352 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.352 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.352 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "3", 2024-05-09T06:37:24.352 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.352 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.352 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.352 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "default replica count for any created local pool", 2024-05-09T06:37:24.352 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.352 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.352 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.352 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.352 INFO:teuthology.orchestra.run.smithi003.stdout: "pg_num": { 2024-05-09T06:37:24.352 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "pg_num", 2024-05-09T06:37:24.353 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-09T06:37:24.353 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.353 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.353 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "128", 2024-05-09T06:37:24.353 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.353 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.353 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.353 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "default pg_num for any created local pool", 2024-05-09T06:37:24.353 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.353 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.353 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.353 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.353 INFO:teuthology.orchestra.run.smithi003.stdout: "prefix": { 2024-05-09T06:37:24.353 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "prefix", 2024-05-09T06:37:24.354 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:24.354 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.354 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.354 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-09T06:37:24.354 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.354 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.354 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.354 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "name prefix for any created local pool", 2024-05-09T06:37:24.354 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.354 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.354 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.354 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.355 INFO:teuthology.orchestra.run.smithi003.stdout: "subtree": { 2024-05-09T06:37:24.355 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "subtree", 2024-05-09T06:37:24.355 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:24.355 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.355 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.355 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "rack", 2024-05-09T06:37:24.355 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.355 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.355 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.355 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "CRUSH level for which to create a local pool", 2024-05-09T06:37:24.355 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-05-09T06:37:24.355 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.355 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.355 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-09T06:37:24.356 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-09T06:37:24.356 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.356 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-09T06:37:24.356 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "mds_autoscaler", 2024-05-09T06:37:24.356 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-05-09T06:37:24.356 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-05-09T06:37:24.356 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-05-09T06:37:24.356 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-05-09T06:37:24.356 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-05-09T06:37:24.356 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:24.356 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.356 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.356 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-09T06:37:24.356 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.356 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.357 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-09T06:37:24.357 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-09T06:37:24.357 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-09T06:37:24.357 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-09T06:37:24.357 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-09T06:37:24.357 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-09T06:37:24.357 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-09T06:37:24.357 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-09T06:37:24.357 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.357 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.357 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.357 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.357 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.357 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-05-09T06:37:24.358 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-05-09T06:37:24.358 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:24.358 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.358 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.358 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-09T06:37:24.358 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.358 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.358 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.358 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.358 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.358 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.358 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.358 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.358 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-05-09T06:37:24.358 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-05-09T06:37:24.359 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:24.359 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.359 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.359 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-05-09T06:37:24.359 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.359 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.359 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-09T06:37:24.359 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-09T06:37:24.359 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-09T06:37:24.359 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-09T06:37:24.359 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-09T06:37:24.359 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-09T06:37:24.360 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-09T06:37:24.360 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-09T06:37:24.360 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.360 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.360 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.360 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.360 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.360 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-05-09T06:37:24.360 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-05-09T06:37:24.360 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:24.360 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.360 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.360 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-09T06:37:24.360 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.361 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.361 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.361 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.361 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.361 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.361 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.361 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-09T06:37:24.361 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-09T06:37:24.361 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.361 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-09T06:37:24.361 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "mirroring", 2024-05-09T06:37:24.361 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-05-09T06:37:24.361 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-05-09T06:37:24.361 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-05-09T06:37:24.361 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-05-09T06:37:24.361 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-05-09T06:37:24.362 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:24.362 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.362 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.362 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-09T06:37:24.362 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.362 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.362 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-09T06:37:24.362 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-09T06:37:24.362 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-09T06:37:24.362 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-09T06:37:24.362 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-09T06:37:24.362 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-09T06:37:24.363 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-09T06:37:24.363 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-09T06:37:24.363 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.363 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.363 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.363 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.363 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.363 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-05-09T06:37:24.363 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-05-09T06:37:24.363 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:24.363 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.364 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.364 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-09T06:37:24.364 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.364 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.364 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.364 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.364 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.364 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.364 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.364 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.364 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-05-09T06:37:24.364 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-05-09T06:37:24.364 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:24.365 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.365 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.365 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-05-09T06:37:24.365 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.365 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.365 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-09T06:37:24.365 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-09T06:37:24.365 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-09T06:37:24.365 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-09T06:37:24.365 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-09T06:37:24.365 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-09T06:37:24.365 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-09T06:37:24.365 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-09T06:37:24.365 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.366 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.366 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.366 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.366 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.366 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-05-09T06:37:24.366 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-05-09T06:37:24.366 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:24.366 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.366 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.366 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-09T06:37:24.366 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.366 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.366 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.367 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.367 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.367 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.367 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.367 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-09T06:37:24.367 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-09T06:37:24.367 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.367 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-09T06:37:24.367 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "nfs", 2024-05-09T06:37:24.367 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-05-09T06:37:24.367 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-05-09T06:37:24.367 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-05-09T06:37:24.367 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-05-09T06:37:24.368 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-05-09T06:37:24.368 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:24.368 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.368 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.368 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-09T06:37:24.368 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.368 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.368 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-09T06:37:24.368 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-09T06:37:24.368 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-09T06:37:24.368 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-09T06:37:24.368 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-09T06:37:24.368 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-09T06:37:24.368 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-09T06:37:24.369 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-09T06:37:24.369 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.369 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.369 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.369 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.369 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.369 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-05-09T06:37:24.369 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-05-09T06:37:24.369 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:24.369 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.369 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.369 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-09T06:37:24.369 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.369 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.369 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.370 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.370 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.370 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.370 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.370 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.370 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-05-09T06:37:24.370 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-05-09T06:37:24.370 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:24.370 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.370 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.370 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-05-09T06:37:24.370 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.370 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.370 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-09T06:37:24.370 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-09T06:37:24.371 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-09T06:37:24.371 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-09T06:37:24.371 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-09T06:37:24.371 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-09T06:37:24.371 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-09T06:37:24.371 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-09T06:37:24.371 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.371 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.371 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.371 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.371 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.371 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-05-09T06:37:24.372 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-05-09T06:37:24.372 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:24.372 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.372 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.372 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-09T06:37:24.372 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.372 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.372 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.372 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.372 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.372 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.372 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.372 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-09T06:37:24.373 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-09T06:37:24.373 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.373 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-09T06:37:24.373 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "orchestrator", 2024-05-09T06:37:24.373 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-05-09T06:37:24.373 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-05-09T06:37:24.373 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-05-09T06:37:24.373 INFO:teuthology.orchestra.run.smithi003.stdout: "fail_fs": { 2024-05-09T06:37:24.373 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "fail_fs", 2024-05-09T06:37:24.373 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:24.373 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.373 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.373 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-09T06:37:24.373 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.374 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.374 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.374 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-05-09T06:37:24.374 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.374 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.374 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.374 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.374 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-05-09T06:37:24.374 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-05-09T06:37:24.374 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:24.374 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.374 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.374 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-09T06:37:24.374 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.375 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.375 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-09T06:37:24.375 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-09T06:37:24.375 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-09T06:37:24.375 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-09T06:37:24.375 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-09T06:37:24.375 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-09T06:37:24.375 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-09T06:37:24.375 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-09T06:37:24.375 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.375 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.375 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.375 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.375 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.375 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-05-09T06:37:24.375 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-05-09T06:37:24.376 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:24.376 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.376 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.376 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-09T06:37:24.376 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.376 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.376 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.376 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.376 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.376 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.376 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.376 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.376 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-05-09T06:37:24.376 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-05-09T06:37:24.376 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:24.377 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.377 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.377 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-05-09T06:37:24.377 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.377 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.377 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-09T06:37:24.377 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-09T06:37:24.377 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-09T06:37:24.377 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-09T06:37:24.377 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-09T06:37:24.377 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-09T06:37:24.377 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-09T06:37:24.377 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-09T06:37:24.377 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.378 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.378 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.378 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.378 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.378 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-05-09T06:37:24.378 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-05-09T06:37:24.378 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:24.378 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.378 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.378 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-09T06:37:24.378 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.378 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.378 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.378 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.378 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.379 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.379 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.379 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.379 INFO:teuthology.orchestra.run.smithi003.stdout: "orchestrator": { 2024-05-09T06:37:24.379 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "orchestrator", 2024-05-09T06:37:24.379 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:24.379 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.379 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.379 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-09T06:37:24.379 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.379 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.379 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-09T06:37:24.379 INFO:teuthology.orchestra.run.smithi003.stdout: "cephadm", 2024-05-09T06:37:24.380 INFO:teuthology.orchestra.run.smithi003.stdout: "rook", 2024-05-09T06:37:24.380 INFO:teuthology.orchestra.run.smithi003.stdout: "test_orchestrator" 2024-05-09T06:37:24.380 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-09T06:37:24.380 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Orchestrator backend", 2024-05-09T06:37:24.380 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.380 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.380 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.380 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-09T06:37:24.380 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-09T06:37:24.380 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.380 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-09T06:37:24.380 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "osd_perf_query", 2024-05-09T06:37:24.380 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-05-09T06:37:24.380 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-05-09T06:37:24.381 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-05-09T06:37:24.381 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-05-09T06:37:24.381 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-05-09T06:37:24.381 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:24.381 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.381 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.381 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-09T06:37:24.381 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.381 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.381 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-09T06:37:24.381 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-09T06:37:24.381 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-09T06:37:24.381 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-09T06:37:24.381 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-09T06:37:24.382 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-09T06:37:24.382 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-09T06:37:24.382 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-09T06:37:24.382 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.382 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.382 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.382 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.382 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.382 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-05-09T06:37:24.382 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-05-09T06:37:24.382 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:24.382 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.382 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.383 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-09T06:37:24.383 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.383 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.383 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.383 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.383 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.383 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.383 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.383 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.383 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-05-09T06:37:24.383 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-05-09T06:37:24.383 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:24.384 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.384 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.384 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-05-09T06:37:24.384 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.384 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.384 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-09T06:37:24.384 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-09T06:37:24.384 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-09T06:37:24.384 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-09T06:37:24.384 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-09T06:37:24.384 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-09T06:37:24.384 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-09T06:37:24.384 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-09T06:37:24.384 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.384 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.385 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.385 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.385 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.385 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-05-09T06:37:24.385 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-05-09T06:37:24.385 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:24.385 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.385 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.385 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-09T06:37:24.385 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.385 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.385 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.385 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.385 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.385 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.386 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.386 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-09T06:37:24.386 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-09T06:37:24.386 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.386 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-09T06:37:24.386 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "osd_support", 2024-05-09T06:37:24.386 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-05-09T06:37:24.386 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-05-09T06:37:24.386 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-05-09T06:37:24.386 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-05-09T06:37:24.386 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-05-09T06:37:24.386 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:24.386 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.386 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.387 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-09T06:37:24.387 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.387 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.387 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-09T06:37:24.387 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-09T06:37:24.387 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-09T06:37:24.387 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-09T06:37:24.387 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-09T06:37:24.387 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-09T06:37:24.387 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-09T06:37:24.387 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-09T06:37:24.387 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.387 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.387 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.387 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.388 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.388 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-05-09T06:37:24.388 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-05-09T06:37:24.388 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:24.388 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.388 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.388 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-09T06:37:24.388 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.388 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.388 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.388 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.388 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.389 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.389 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.389 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.389 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-05-09T06:37:24.389 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-05-09T06:37:24.389 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:24.389 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.389 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.389 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-05-09T06:37:24.389 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.389 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.389 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-09T06:37:24.389 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-09T06:37:24.389 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-09T06:37:24.389 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-09T06:37:24.390 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-09T06:37:24.390 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-09T06:37:24.390 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-09T06:37:24.390 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-09T06:37:24.390 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.390 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.390 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.390 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.390 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.390 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-05-09T06:37:24.390 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-05-09T06:37:24.390 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:24.390 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.391 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.391 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-09T06:37:24.391 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.391 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.391 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.391 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.391 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.391 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.391 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.391 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-09T06:37:24.391 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-09T06:37:24.391 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.391 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-09T06:37:24.391 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "pg_autoscaler", 2024-05-09T06:37:24.392 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-05-09T06:37:24.392 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-05-09T06:37:24.392 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-05-09T06:37:24.392 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-05-09T06:37:24.392 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-05-09T06:37:24.392 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:24.392 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.392 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.392 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-09T06:37:24.392 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.392 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.392 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-09T06:37:24.392 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-09T06:37:24.392 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-09T06:37:24.392 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-09T06:37:24.393 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-09T06:37:24.393 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-09T06:37:24.393 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-09T06:37:24.393 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-09T06:37:24.393 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.393 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.393 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.393 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.393 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.393 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-05-09T06:37:24.393 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-05-09T06:37:24.393 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:24.393 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.393 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.394 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-09T06:37:24.394 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.394 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.394 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.394 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.394 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.394 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.394 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.394 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.394 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-05-09T06:37:24.394 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-05-09T06:37:24.394 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:24.394 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.394 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.395 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-05-09T06:37:24.395 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.395 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.395 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-09T06:37:24.395 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-09T06:37:24.395 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-09T06:37:24.395 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-09T06:37:24.395 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-09T06:37:24.395 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-09T06:37:24.395 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-09T06:37:24.395 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-09T06:37:24.395 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.395 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.395 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.395 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.396 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.396 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-05-09T06:37:24.396 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-05-09T06:37:24.396 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:24.396 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.396 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.396 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-09T06:37:24.396 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.396 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.396 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.396 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.396 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.396 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.396 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.397 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.397 INFO:teuthology.orchestra.run.smithi003.stdout: "sleep_interval": { 2024-05-09T06:37:24.397 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sleep_interval", 2024-05-09T06:37:24.397 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-05-09T06:37:24.397 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.397 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.397 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "60", 2024-05-09T06:37:24.397 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.397 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.397 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.397 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.397 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.397 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.398 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.398 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.398 INFO:teuthology.orchestra.run.smithi003.stdout: "threshold": { 2024-05-09T06:37:24.398 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "threshold", 2024-05-09T06:37:24.398 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "float", 2024-05-09T06:37:24.398 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.398 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.398 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "3.0", 2024-05-09T06:37:24.398 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "1.0", 2024-05-09T06:37:24.398 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.398 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.398 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "scaling threshold", 2024-05-09T06:37:24.398 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-05-09T06:37:24.398 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.399 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.399 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-09T06:37:24.399 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-09T06:37:24.399 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.399 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-09T06:37:24.399 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "progress", 2024-05-09T06:37:24.399 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-05-09T06:37:24.399 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-05-09T06:37:24.399 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-05-09T06:37:24.399 INFO:teuthology.orchestra.run.smithi003.stdout: "allow_pg_recovery_event": { 2024-05-09T06:37:24.399 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "allow_pg_recovery_event", 2024-05-09T06:37:24.399 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:24.400 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.400 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.400 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-09T06:37:24.400 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.400 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.400 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.400 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "allow the module to show pg recovery progress", 2024-05-09T06:37:24.400 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.400 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.400 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.400 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.400 INFO:teuthology.orchestra.run.smithi003.stdout: "enabled": { 2024-05-09T06:37:24.400 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "enabled", 2024-05-09T06:37:24.401 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:24.401 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.401 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.401 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-05-09T06:37:24.401 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.401 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.401 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.401 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.401 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.401 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.401 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.401 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.401 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-05-09T06:37:24.401 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-05-09T06:37:24.401 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:24.401 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.401 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.402 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-09T06:37:24.402 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.402 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.402 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-09T06:37:24.402 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-09T06:37:24.402 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-09T06:37:24.402 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-09T06:37:24.402 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-09T06:37:24.402 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-09T06:37:24.402 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-09T06:37:24.402 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-09T06:37:24.402 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.402 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.402 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.402 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.403 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.403 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-05-09T06:37:24.403 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-05-09T06:37:24.403 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:24.403 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.403 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.403 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-09T06:37:24.403 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.403 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.403 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.403 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.403 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.403 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.403 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.403 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.404 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-05-09T06:37:24.404 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-05-09T06:37:24.404 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:24.404 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.404 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.404 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-05-09T06:37:24.404 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.404 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.404 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-09T06:37:24.404 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-09T06:37:24.404 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-09T06:37:24.404 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-09T06:37:24.404 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-09T06:37:24.404 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-09T06:37:24.404 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-09T06:37:24.404 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-09T06:37:24.405 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.405 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.405 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.405 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.405 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.405 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-05-09T06:37:24.405 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-05-09T06:37:24.405 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:24.405 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.405 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.405 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-09T06:37:24.405 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.405 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.405 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.405 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.406 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.406 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.406 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.406 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.406 INFO:teuthology.orchestra.run.smithi003.stdout: "max_completed_events": { 2024-05-09T06:37:24.406 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "max_completed_events", 2024-05-09T06:37:24.406 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-09T06:37:24.406 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.406 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.406 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "50", 2024-05-09T06:37:24.406 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.406 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.406 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.406 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "number of past completed events to remember", 2024-05-09T06:37:24.407 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.407 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.407 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.407 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.407 INFO:teuthology.orchestra.run.smithi003.stdout: "sleep_interval": { 2024-05-09T06:37:24.407 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sleep_interval", 2024-05-09T06:37:24.407 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-05-09T06:37:24.407 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.407 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.407 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "5", 2024-05-09T06:37:24.407 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.407 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.407 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.407 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "how long the module is going to sleep", 2024-05-09T06:37:24.408 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.408 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.408 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.408 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-09T06:37:24.408 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-09T06:37:24.408 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.408 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-09T06:37:24.408 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "prometheus", 2024-05-09T06:37:24.408 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-05-09T06:37:24.408 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-05-09T06:37:24.408 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-05-09T06:37:24.409 INFO:teuthology.orchestra.run.smithi003.stdout: "cache": { 2024-05-09T06:37:24.409 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "cache", 2024-05-09T06:37:24.409 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:24.409 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.409 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.409 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-05-09T06:37:24.409 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.409 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.409 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.409 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.409 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.409 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.409 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.409 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.410 INFO:teuthology.orchestra.run.smithi003.stdout: "exclude_perf_counters": { 2024-05-09T06:37:24.410 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "exclude_perf_counters", 2024-05-09T06:37:24.410 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:24.410 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.410 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.410 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-05-09T06:37:24.410 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.410 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.410 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.410 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-05-09T06:37:24.410 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-05-09T06:37:24.410 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.410 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.410 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.411 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-05-09T06:37:24.411 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-05-09T06:37:24.411 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:24.411 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.411 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.411 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-09T06:37:24.411 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.411 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.411 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-09T06:37:24.411 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-09T06:37:24.411 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-09T06:37:24.411 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-09T06:37:24.411 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-09T06:37:24.411 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-09T06:37:24.411 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-09T06:37:24.411 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-09T06:37:24.412 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.412 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.412 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.412 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.412 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.412 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-05-09T06:37:24.412 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-05-09T06:37:24.412 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:24.412 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.412 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.412 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-09T06:37:24.412 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.412 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.412 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.412 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.413 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.413 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.413 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.413 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.413 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-05-09T06:37:24.413 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-05-09T06:37:24.413 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:24.413 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.413 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.413 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-05-09T06:37:24.413 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.413 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.413 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-09T06:37:24.413 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-09T06:37:24.413 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-09T06:37:24.414 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-09T06:37:24.414 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-09T06:37:24.414 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-09T06:37:24.414 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-09T06:37:24.414 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-09T06:37:24.414 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.414 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.414 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.414 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.414 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.414 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-05-09T06:37:24.414 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-05-09T06:37:24.414 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:24.414 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.414 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.415 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-09T06:37:24.415 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.415 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.415 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.415 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.415 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.415 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.415 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.415 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.415 INFO:teuthology.orchestra.run.smithi003.stdout: "rbd_stats_pools": { 2024-05-09T06:37:24.415 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rbd_stats_pools", 2024-05-09T06:37:24.415 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:24.415 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.415 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.415 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-09T06:37:24.415 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.416 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.416 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.416 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.416 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.416 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.416 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.416 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.416 INFO:teuthology.orchestra.run.smithi003.stdout: "rbd_stats_pools_refresh_interval": { 2024-05-09T06:37:24.416 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-05-09T06:37:24.416 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-09T06:37:24.416 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.416 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.416 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "300", 2024-05-09T06:37:24.416 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.417 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.417 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.417 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.417 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.417 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.417 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.417 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.417 INFO:teuthology.orchestra.run.smithi003.stdout: "scrape_interval": { 2024-05-09T06:37:24.417 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "scrape_interval", 2024-05-09T06:37:24.417 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "float", 2024-05-09T06:37:24.417 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.417 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.417 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "15.0", 2024-05-09T06:37:24.417 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.417 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.417 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.417 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.418 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.418 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.418 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.418 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.418 INFO:teuthology.orchestra.run.smithi003.stdout: "server_addr": { 2024-05-09T06:37:24.418 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "server_addr", 2024-05-09T06:37:24.418 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:24.418 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.418 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.418 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "::", 2024-05-09T06:37:24.418 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.418 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.418 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.418 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-05-09T06:37:24.419 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.419 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.419 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.419 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.419 INFO:teuthology.orchestra.run.smithi003.stdout: "server_port": { 2024-05-09T06:37:24.419 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "server_port", 2024-05-09T06:37:24.419 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-09T06:37:24.419 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.419 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.419 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "9283", 2024-05-09T06:37:24.419 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.419 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.419 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.420 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-05-09T06:37:24.420 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.420 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.420 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.420 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.420 INFO:teuthology.orchestra.run.smithi003.stdout: "stale_cache_strategy": { 2024-05-09T06:37:24.420 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "stale_cache_strategy", 2024-05-09T06:37:24.420 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:24.420 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.420 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.420 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "log", 2024-05-09T06:37:24.420 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.421 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.421 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.421 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.421 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.421 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.421 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.421 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.421 INFO:teuthology.orchestra.run.smithi003.stdout: "standby_behaviour": { 2024-05-09T06:37:24.421 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "standby_behaviour", 2024-05-09T06:37:24.421 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:24.421 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.421 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.421 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "default", 2024-05-09T06:37:24.421 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.422 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.422 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-09T06:37:24.422 INFO:teuthology.orchestra.run.smithi003.stdout: "default", 2024-05-09T06:37:24.422 INFO:teuthology.orchestra.run.smithi003.stdout: "error" 2024-05-09T06:37:24.422 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-09T06:37:24.422 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.422 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.422 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.422 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.422 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.422 INFO:teuthology.orchestra.run.smithi003.stdout: "standby_error_status_code": { 2024-05-09T06:37:24.422 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "standby_error_status_code", 2024-05-09T06:37:24.422 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-09T06:37:24.422 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.423 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.423 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "500", 2024-05-09T06:37:24.423 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "400", 2024-05-09T06:37:24.423 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "599", 2024-05-09T06:37:24.423 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.423 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.423 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.423 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.423 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.423 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-09T06:37:24.423 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-09T06:37:24.424 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.424 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-09T06:37:24.424 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rbd_support", 2024-05-09T06:37:24.424 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-05-09T06:37:24.424 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-05-09T06:37:24.424 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-05-09T06:37:24.424 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-05-09T06:37:24.424 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-05-09T06:37:24.424 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:24.424 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.424 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.424 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-09T06:37:24.424 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.425 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.425 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-09T06:37:24.425 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-09T06:37:24.425 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-09T06:37:24.425 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-09T06:37:24.425 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-09T06:37:24.425 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-09T06:37:24.425 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-09T06:37:24.425 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-09T06:37:24.425 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.425 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.425 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.425 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.425 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.426 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-05-09T06:37:24.426 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-05-09T06:37:24.426 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:24.426 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.426 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.426 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-09T06:37:24.426 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.426 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.426 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.426 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.426 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.427 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.427 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.427 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.427 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-05-09T06:37:24.427 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-05-09T06:37:24.427 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:24.427 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.427 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.427 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-05-09T06:37:24.427 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.427 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.427 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-09T06:37:24.427 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-09T06:37:24.428 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-09T06:37:24.428 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-09T06:37:24.428 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-09T06:37:24.428 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-09T06:37:24.428 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-09T06:37:24.428 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-09T06:37:24.428 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.428 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.428 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.428 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.428 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.428 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-05-09T06:37:24.429 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-05-09T06:37:24.429 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:24.429 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.429 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.429 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-09T06:37:24.429 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.429 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.429 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.429 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.429 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.429 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.429 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.429 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.430 INFO:teuthology.orchestra.run.smithi003.stdout: "max_concurrent_snap_create": { 2024-05-09T06:37:24.430 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "max_concurrent_snap_create", 2024-05-09T06:37:24.430 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-09T06:37:24.430 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.430 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.430 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "10", 2024-05-09T06:37:24.430 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.430 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.430 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.430 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.430 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.430 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.430 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.431 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.431 INFO:teuthology.orchestra.run.smithi003.stdout: "mirror_snapshot_schedule": { 2024-05-09T06:37:24.431 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "mirror_snapshot_schedule", 2024-05-09T06:37:24.431 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:24.431 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.431 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.431 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-09T06:37:24.431 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.431 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.431 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.431 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.431 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.432 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.432 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.432 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.432 INFO:teuthology.orchestra.run.smithi003.stdout: "trash_purge_schedule": { 2024-05-09T06:37:24.432 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "trash_purge_schedule", 2024-05-09T06:37:24.432 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:24.432 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.432 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.432 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-09T06:37:24.432 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.432 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.432 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.433 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.433 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.433 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.433 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.433 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-09T06:37:24.433 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-09T06:37:24.433 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.433 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-09T06:37:24.433 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "restful", 2024-05-09T06:37:24.433 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-05-09T06:37:24.433 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-05-09T06:37:24.433 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-05-09T06:37:24.433 INFO:teuthology.orchestra.run.smithi003.stdout: "enable_auth": { 2024-05-09T06:37:24.433 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "enable_auth", 2024-05-09T06:37:24.434 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:24.434 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.434 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.434 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-05-09T06:37:24.434 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.434 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.434 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.434 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.434 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.434 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.434 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.434 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.434 INFO:teuthology.orchestra.run.smithi003.stdout: "key_file": { 2024-05-09T06:37:24.434 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "key_file", 2024-05-09T06:37:24.435 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:24.435 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.435 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.435 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-09T06:37:24.435 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.435 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.435 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.435 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.435 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.435 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.435 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.435 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.435 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-05-09T06:37:24.435 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-05-09T06:37:24.435 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:24.435 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.436 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.436 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-09T06:37:24.436 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.436 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.436 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-09T06:37:24.436 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-09T06:37:24.436 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-09T06:37:24.436 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-09T06:37:24.436 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-09T06:37:24.436 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-09T06:37:24.437 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-09T06:37:24.437 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-09T06:37:24.437 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.437 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.437 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.437 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.437 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.437 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-05-09T06:37:24.437 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-05-09T06:37:24.437 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:24.437 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.437 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.437 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-09T06:37:24.437 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.437 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.438 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.438 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.438 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.438 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.438 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.438 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.438 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-05-09T06:37:24.438 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-05-09T06:37:24.438 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:24.438 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.438 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.438 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-05-09T06:37:24.438 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.438 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.438 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-09T06:37:24.439 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-09T06:37:24.439 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-09T06:37:24.439 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-09T06:37:24.439 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-09T06:37:24.439 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-09T06:37:24.439 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-09T06:37:24.439 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-09T06:37:24.439 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.439 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.439 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.439 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.439 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.439 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-05-09T06:37:24.439 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-05-09T06:37:24.439 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:24.439 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.440 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.440 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-09T06:37:24.440 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.440 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.440 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.440 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.440 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.440 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.440 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.440 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.440 INFO:teuthology.orchestra.run.smithi003.stdout: "server_addr": { 2024-05-09T06:37:24.440 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "server_addr", 2024-05-09T06:37:24.440 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:24.440 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.440 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.441 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-09T06:37:24.441 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.441 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.441 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.441 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.441 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.441 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.441 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.441 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.441 INFO:teuthology.orchestra.run.smithi003.stdout: "server_port": { 2024-05-09T06:37:24.441 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "server_port", 2024-05-09T06:37:24.441 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:24.441 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.442 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.442 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-09T06:37:24.442 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.442 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.442 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.442 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.442 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.442 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.442 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.442 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-09T06:37:24.442 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-09T06:37:24.442 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.442 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-09T06:37:24.442 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rgw", 2024-05-09T06:37:24.443 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-05-09T06:37:24.443 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-05-09T06:37:24.443 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-05-09T06:37:24.443 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-05-09T06:37:24.443 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-05-09T06:37:24.443 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:24.443 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.443 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.443 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-09T06:37:24.443 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.443 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.443 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-09T06:37:24.443 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-09T06:37:24.443 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-09T06:37:24.444 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-09T06:37:24.444 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-09T06:37:24.444 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-09T06:37:24.444 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-09T06:37:24.444 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-09T06:37:24.444 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.444 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.444 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.444 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.444 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.444 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-05-09T06:37:24.444 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-05-09T06:37:24.444 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:24.444 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.444 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.445 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-09T06:37:24.445 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.445 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.445 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.445 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.445 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.445 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.445 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.445 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.445 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-05-09T06:37:24.445 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-05-09T06:37:24.445 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:24.445 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.445 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.446 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-05-09T06:37:24.446 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.446 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.446 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-09T06:37:24.446 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-09T06:37:24.446 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-09T06:37:24.446 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-09T06:37:24.446 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-09T06:37:24.446 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-09T06:37:24.446 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-09T06:37:24.446 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-09T06:37:24.446 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.446 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.446 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.447 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.447 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.447 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-05-09T06:37:24.447 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-05-09T06:37:24.447 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:24.447 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.447 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.447 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-09T06:37:24.447 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.447 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.447 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.447 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.447 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.447 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.447 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.448 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-09T06:37:24.448 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-09T06:37:24.448 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.448 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-09T06:37:24.448 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rook", 2024-05-09T06:37:24.448 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-05-09T06:37:24.448 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-05-09T06:37:24.448 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-05-09T06:37:24.448 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-05-09T06:37:24.448 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-05-09T06:37:24.448 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:24.448 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.448 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.448 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-09T06:37:24.448 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.449 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.449 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-09T06:37:24.449 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-09T06:37:24.449 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-09T06:37:24.449 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-09T06:37:24.449 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-09T06:37:24.449 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-09T06:37:24.449 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-09T06:37:24.449 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-09T06:37:24.449 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.449 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.449 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.449 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.449 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.449 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-05-09T06:37:24.450 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-05-09T06:37:24.450 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:24.450 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.450 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.450 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-09T06:37:24.450 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.450 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.450 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.450 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.450 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.450 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.450 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.450 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.451 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-05-09T06:37:24.451 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-05-09T06:37:24.451 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:24.451 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.451 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.451 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-05-09T06:37:24.451 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.451 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.451 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-09T06:37:24.451 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-09T06:37:24.451 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-09T06:37:24.451 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-09T06:37:24.451 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-09T06:37:24.451 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-09T06:37:24.451 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-09T06:37:24.452 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-09T06:37:24.452 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.452 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.452 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.452 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.452 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.452 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-05-09T06:37:24.452 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-05-09T06:37:24.452 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:24.452 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.452 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.452 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-09T06:37:24.452 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.452 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.452 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.453 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.453 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.453 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.453 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.453 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.453 INFO:teuthology.orchestra.run.smithi003.stdout: "storage_class": { 2024-05-09T06:37:24.453 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "storage_class", 2024-05-09T06:37:24.453 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:24.453 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.453 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.453 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "local", 2024-05-09T06:37:24.453 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.453 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.453 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.453 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-05-09T06:37:24.454 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.454 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.454 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.454 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-09T06:37:24.454 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-09T06:37:24.454 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.454 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-09T06:37:24.454 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "selftest", 2024-05-09T06:37:24.454 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-05-09T06:37:24.454 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-05-09T06:37:24.454 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-05-09T06:37:24.454 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-05-09T06:37:24.454 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-05-09T06:37:24.454 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:24.454 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.455 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.455 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-09T06:37:24.455 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.455 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.455 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-09T06:37:24.455 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-09T06:37:24.455 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-09T06:37:24.455 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-09T06:37:24.455 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-09T06:37:24.455 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-09T06:37:24.455 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-09T06:37:24.455 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-09T06:37:24.455 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.455 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.455 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.456 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.456 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.456 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-05-09T06:37:24.456 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-05-09T06:37:24.456 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:24.456 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.456 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.456 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-09T06:37:24.456 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.456 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.456 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.456 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.456 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.456 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.457 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.457 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.457 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-05-09T06:37:24.457 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-05-09T06:37:24.457 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:24.457 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.457 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.457 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-05-09T06:37:24.457 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.457 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.457 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-09T06:37:24.457 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-09T06:37:24.457 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-09T06:37:24.457 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-09T06:37:24.457 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-09T06:37:24.458 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-09T06:37:24.458 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-09T06:37:24.458 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-09T06:37:24.458 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.458 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.458 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.458 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.458 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.458 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-05-09T06:37:24.458 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-05-09T06:37:24.458 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:24.458 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.458 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.459 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-09T06:37:24.459 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.459 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.459 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.459 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.459 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.459 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.459 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.459 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.459 INFO:teuthology.orchestra.run.smithi003.stdout: "roption1": { 2024-05-09T06:37:24.459 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "roption1", 2024-05-09T06:37:24.459 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:24.459 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.459 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.459 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-09T06:37:24.460 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.460 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.460 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.460 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.460 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.460 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.460 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.460 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.460 INFO:teuthology.orchestra.run.smithi003.stdout: "roption2": { 2024-05-09T06:37:24.460 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "roption2", 2024-05-09T06:37:24.460 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:24.460 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.460 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.461 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "xyz", 2024-05-09T06:37:24.461 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.461 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.461 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.461 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.461 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.461 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.461 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.461 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.461 INFO:teuthology.orchestra.run.smithi003.stdout: "rwoption1": { 2024-05-09T06:37:24.461 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rwoption1", 2024-05-09T06:37:24.461 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:24.461 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.461 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.462 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-09T06:37:24.462 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.462 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.462 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.462 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.462 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.462 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.462 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.462 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.462 INFO:teuthology.orchestra.run.smithi003.stdout: "rwoption2": { 2024-05-09T06:37:24.462 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rwoption2", 2024-05-09T06:37:24.462 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-09T06:37:24.462 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.462 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.462 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-09T06:37:24.463 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.463 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.463 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.463 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.463 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.463 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.463 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.463 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.463 INFO:teuthology.orchestra.run.smithi003.stdout: "rwoption3": { 2024-05-09T06:37:24.463 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rwoption3", 2024-05-09T06:37:24.463 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "float", 2024-05-09T06:37:24.463 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.463 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.463 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-09T06:37:24.463 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.464 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.464 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.464 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.464 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.464 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.464 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.464 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.464 INFO:teuthology.orchestra.run.smithi003.stdout: "rwoption4": { 2024-05-09T06:37:24.464 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rwoption4", 2024-05-09T06:37:24.464 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:24.464 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.464 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.464 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-09T06:37:24.464 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.464 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.464 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.465 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.465 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.465 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.465 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.465 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.465 INFO:teuthology.orchestra.run.smithi003.stdout: "rwoption5": { 2024-05-09T06:37:24.465 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rwoption5", 2024-05-09T06:37:24.465 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:24.465 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.465 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.465 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-09T06:37:24.465 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.466 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.466 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.466 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.466 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.466 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.466 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.466 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.466 INFO:teuthology.orchestra.run.smithi003.stdout: "rwoption6": { 2024-05-09T06:37:24.466 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rwoption6", 2024-05-09T06:37:24.466 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:24.466 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.466 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.466 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-05-09T06:37:24.466 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.467 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.467 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.467 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.467 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.467 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.467 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.467 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.467 INFO:teuthology.orchestra.run.smithi003.stdout: "rwoption7": { 2024-05-09T06:37:24.467 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rwoption7", 2024-05-09T06:37:24.467 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-09T06:37:24.467 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.467 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.467 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-09T06:37:24.467 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "1", 2024-05-09T06:37:24.467 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "42", 2024-05-09T06:37:24.468 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.468 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.468 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.468 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.468 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.468 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.468 INFO:teuthology.orchestra.run.smithi003.stdout: "testkey": { 2024-05-09T06:37:24.468 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "testkey", 2024-05-09T06:37:24.468 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:24.468 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.468 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.468 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-09T06:37:24.468 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.468 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.468 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.469 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.469 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.469 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.469 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.469 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.469 INFO:teuthology.orchestra.run.smithi003.stdout: "testlkey": { 2024-05-09T06:37:24.469 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "testlkey", 2024-05-09T06:37:24.469 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:24.469 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.469 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.469 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-09T06:37:24.469 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.469 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.469 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.469 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.470 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.470 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.470 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.470 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.470 INFO:teuthology.orchestra.run.smithi003.stdout: "testnewline": { 2024-05-09T06:37:24.470 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "testnewline", 2024-05-09T06:37:24.470 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:24.470 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.470 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.470 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-09T06:37:24.470 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.470 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.470 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.470 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.470 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.471 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.471 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.471 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-09T06:37:24.471 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-09T06:37:24.471 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.471 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-09T06:37:24.471 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "snap_schedule", 2024-05-09T06:37:24.471 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-05-09T06:37:24.471 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-05-09T06:37:24.471 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-05-09T06:37:24.471 INFO:teuthology.orchestra.run.smithi003.stdout: "allow_m_granularity": { 2024-05-09T06:37:24.471 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "allow_m_granularity", 2024-05-09T06:37:24.471 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:24.471 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.471 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.472 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-09T06:37:24.472 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.472 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.472 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.472 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "allow minute scheduled snapshots", 2024-05-09T06:37:24.472 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.472 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.472 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.472 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.472 INFO:teuthology.orchestra.run.smithi003.stdout: "dump_on_update": { 2024-05-09T06:37:24.472 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "dump_on_update", 2024-05-09T06:37:24.472 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:24.472 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.472 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.472 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-09T06:37:24.473 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.473 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.473 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.473 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "dump database to debug log on update", 2024-05-09T06:37:24.473 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.473 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.473 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.473 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.473 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-05-09T06:37:24.473 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-05-09T06:37:24.473 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:24.473 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.473 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.473 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-09T06:37:24.474 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.474 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.474 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-09T06:37:24.474 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-09T06:37:24.474 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-09T06:37:24.474 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-09T06:37:24.474 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-09T06:37:24.474 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-09T06:37:24.474 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-09T06:37:24.474 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-09T06:37:24.474 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.474 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.474 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.474 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.474 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.475 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-05-09T06:37:24.475 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-05-09T06:37:24.475 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:24.475 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.475 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.475 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-09T06:37:24.475 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.475 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.475 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.475 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.475 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.475 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.475 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.475 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.476 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-05-09T06:37:24.476 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-05-09T06:37:24.476 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:24.476 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.476 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.476 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-05-09T06:37:24.476 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.476 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.476 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-09T06:37:24.476 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-09T06:37:24.476 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-09T06:37:24.476 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-09T06:37:24.476 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-09T06:37:24.476 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-09T06:37:24.476 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-09T06:37:24.477 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-09T06:37:24.477 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.477 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.477 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.477 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.477 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.477 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-05-09T06:37:24.477 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-05-09T06:37:24.477 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:24.477 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.477 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.477 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-09T06:37:24.477 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.477 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.477 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.477 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.478 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.478 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.478 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.478 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-09T06:37:24.478 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-09T06:37:24.478 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.478 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-09T06:37:24.478 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "stats", 2024-05-09T06:37:24.478 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-05-09T06:37:24.478 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-05-09T06:37:24.478 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-05-09T06:37:24.478 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-05-09T06:37:24.478 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-05-09T06:37:24.478 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:24.479 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.479 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.479 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-09T06:37:24.479 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.479 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.479 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-09T06:37:24.479 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-09T06:37:24.479 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-09T06:37:24.479 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-09T06:37:24.479 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-09T06:37:24.479 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-09T06:37:24.479 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-09T06:37:24.479 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-09T06:37:24.479 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.479 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.480 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.480 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.480 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.480 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-05-09T06:37:24.480 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-05-09T06:37:24.480 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:24.480 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.480 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.480 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-09T06:37:24.480 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.480 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.480 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.480 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.480 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.480 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.480 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.481 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.481 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-05-09T06:37:24.481 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-05-09T06:37:24.481 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:24.481 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.481 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.481 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-05-09T06:37:24.481 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.481 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.481 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-09T06:37:24.481 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-09T06:37:24.481 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-09T06:37:24.481 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-09T06:37:24.481 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-09T06:37:24.481 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-09T06:37:24.481 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-09T06:37:24.482 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-09T06:37:24.482 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.482 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.482 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.482 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.482 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.482 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-05-09T06:37:24.482 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-05-09T06:37:24.482 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:24.482 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.482 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.482 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-09T06:37:24.482 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.483 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.483 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.483 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.483 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.483 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.483 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.483 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-09T06:37:24.483 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-09T06:37:24.483 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.483 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-09T06:37:24.483 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "status", 2024-05-09T06:37:24.483 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-05-09T06:37:24.483 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-05-09T06:37:24.483 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-05-09T06:37:24.483 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-05-09T06:37:24.483 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-05-09T06:37:24.484 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:24.484 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.484 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.484 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-09T06:37:24.484 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.484 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.484 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-09T06:37:24.484 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-09T06:37:24.484 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-09T06:37:24.484 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-09T06:37:24.484 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-09T06:37:24.484 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-09T06:37:24.484 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-09T06:37:24.484 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-09T06:37:24.485 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.485 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.485 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.485 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.485 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.485 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-05-09T06:37:24.485 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-05-09T06:37:24.485 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:24.485 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.485 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.485 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-09T06:37:24.485 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.485 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.485 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.485 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.486 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.486 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.486 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.486 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.486 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-05-09T06:37:24.486 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-05-09T06:37:24.486 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:24.486 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.486 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.486 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-05-09T06:37:24.486 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.486 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.486 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-09T06:37:24.486 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-09T06:37:24.487 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-09T06:37:24.487 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-09T06:37:24.487 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-09T06:37:24.487 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-09T06:37:24.487 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-09T06:37:24.487 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-09T06:37:24.487 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.487 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.487 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.487 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.487 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.487 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-05-09T06:37:24.487 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-05-09T06:37:24.487 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:24.488 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.488 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.488 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-09T06:37:24.488 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.488 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.488 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.488 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.488 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.488 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.488 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.488 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-09T06:37:24.488 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-09T06:37:24.488 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.488 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-09T06:37:24.488 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "telegraf", 2024-05-09T06:37:24.489 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-05-09T06:37:24.489 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-05-09T06:37:24.489 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-05-09T06:37:24.489 INFO:teuthology.orchestra.run.smithi003.stdout: "address": { 2024-05-09T06:37:24.489 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "address", 2024-05-09T06:37:24.489 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:24.489 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.489 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.489 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-05-09T06:37:24.489 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.489 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.489 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.489 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.489 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.489 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.490 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.490 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.490 INFO:teuthology.orchestra.run.smithi003.stdout: "interval": { 2024-05-09T06:37:24.490 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "interval", 2024-05-09T06:37:24.490 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-05-09T06:37:24.490 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.490 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.490 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "15", 2024-05-09T06:37:24.490 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.490 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.490 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.490 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.490 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.490 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.491 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.491 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.491 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-05-09T06:37:24.491 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-05-09T06:37:24.491 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:24.491 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.491 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.491 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-09T06:37:24.491 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.491 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.491 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-09T06:37:24.491 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-09T06:37:24.491 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-09T06:37:24.491 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-09T06:37:24.491 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-09T06:37:24.492 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-09T06:37:24.492 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-09T06:37:24.492 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-09T06:37:24.492 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.492 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.492 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.492 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.492 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.492 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-05-09T06:37:24.492 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-05-09T06:37:24.492 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:24.492 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.492 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.492 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-09T06:37:24.493 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.493 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.493 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.493 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.493 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.493 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.493 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.493 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.493 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-05-09T06:37:24.493 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-05-09T06:37:24.493 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:24.493 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.493 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.493 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-05-09T06:37:24.494 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.494 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.494 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-09T06:37:24.494 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-09T06:37:24.494 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-09T06:37:24.494 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-09T06:37:24.494 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-09T06:37:24.494 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-09T06:37:24.494 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-09T06:37:24.494 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-09T06:37:24.494 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.494 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.494 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.494 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.494 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.495 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-05-09T06:37:24.495 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-05-09T06:37:24.495 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:24.495 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.495 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.495 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-09T06:37:24.495 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.495 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.495 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.495 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.495 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.495 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.495 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.496 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-09T06:37:24.496 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-09T06:37:24.496 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.496 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-09T06:37:24.496 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "telemetry", 2024-05-09T06:37:24.496 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-05-09T06:37:24.496 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-05-09T06:37:24.496 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-05-09T06:37:24.496 INFO:teuthology.orchestra.run.smithi003.stdout: "channel_basic": { 2024-05-09T06:37:24.496 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "channel_basic", 2024-05-09T06:37:24.496 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:24.496 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.496 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.496 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-05-09T06:37:24.496 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.496 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.497 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.497 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Share basic cluster information (size, version)", 2024-05-09T06:37:24.497 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.497 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.497 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.497 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.497 INFO:teuthology.orchestra.run.smithi003.stdout: "channel_crash": { 2024-05-09T06:37:24.497 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "channel_crash", 2024-05-09T06:37:24.497 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:24.497 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.497 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.497 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-05-09T06:37:24.497 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.498 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.498 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.498 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-05-09T06:37:24.498 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.498 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.498 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.498 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.498 INFO:teuthology.orchestra.run.smithi003.stdout: "channel_device": { 2024-05-09T06:37:24.498 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "channel_device", 2024-05-09T06:37:24.498 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:24.501 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.501 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.501 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-05-09T06:37:24.501 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.502 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.502 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.502 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-05-09T06:37:24.502 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.502 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.502 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.502 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.502 INFO:teuthology.orchestra.run.smithi003.stdout: "channel_ident": { 2024-05-09T06:37:24.502 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "channel_ident", 2024-05-09T06:37:24.502 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:24.502 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.502 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.502 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-09T06:37:24.502 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.502 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.503 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.503 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-05-09T06:37:24.503 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.503 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.503 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.503 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.503 INFO:teuthology.orchestra.run.smithi003.stdout: "channel_perf": { 2024-05-09T06:37:24.503 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "channel_perf", 2024-05-09T06:37:24.503 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:24.503 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.503 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.503 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-09T06:37:24.503 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.503 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.503 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.504 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Share various performance metrics of a cluster", 2024-05-09T06:37:24.504 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.504 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.504 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.504 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.504 INFO:teuthology.orchestra.run.smithi003.stdout: "contact": { 2024-05-09T06:37:24.504 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "contact", 2024-05-09T06:37:24.504 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:24.504 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.504 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.504 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-09T06:37:24.504 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.504 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.504 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.504 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.504 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.505 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.505 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.505 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.505 INFO:teuthology.orchestra.run.smithi003.stdout: "description": { 2024-05-09T06:37:24.505 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "description", 2024-05-09T06:37:24.505 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:24.505 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.505 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.505 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-09T06:37:24.505 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.505 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.505 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.505 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.505 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.505 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.506 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.506 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.506 INFO:teuthology.orchestra.run.smithi003.stdout: "device_url": { 2024-05-09T06:37:24.506 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "device_url", 2024-05-09T06:37:24.506 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:24.506 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.506 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.506 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-05-09T06:37:24.506 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.506 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.506 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.506 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.506 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.506 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.506 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.507 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.507 INFO:teuthology.orchestra.run.smithi003.stdout: "enabled": { 2024-05-09T06:37:24.507 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "enabled", 2024-05-09T06:37:24.507 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:24.507 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.507 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.507 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-09T06:37:24.507 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.507 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.507 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.507 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.507 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.507 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.507 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.508 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.508 INFO:teuthology.orchestra.run.smithi003.stdout: "interval": { 2024-05-09T06:37:24.508 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "interval", 2024-05-09T06:37:24.508 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-09T06:37:24.508 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.508 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.508 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "24", 2024-05-09T06:37:24.508 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "8", 2024-05-09T06:37:24.508 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.508 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.508 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.508 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.508 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.508 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.508 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.509 INFO:teuthology.orchestra.run.smithi003.stdout: "last_opt_revision": { 2024-05-09T06:37:24.509 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "last_opt_revision", 2024-05-09T06:37:24.509 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-09T06:37:24.509 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.509 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.509 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "1", 2024-05-09T06:37:24.509 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.509 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.509 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.509 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.509 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.509 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.509 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.510 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.510 INFO:teuthology.orchestra.run.smithi003.stdout: "leaderboard": { 2024-05-09T06:37:24.510 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "leaderboard", 2024-05-09T06:37:24.510 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:24.510 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.510 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.510 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-09T06:37:24.510 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.510 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.510 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.510 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.510 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.511 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.511 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.511 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.511 INFO:teuthology.orchestra.run.smithi003.stdout: "leaderboard_description": { 2024-05-09T06:37:24.511 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "leaderboard_description", 2024-05-09T06:37:24.511 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:24.511 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.511 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.511 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-09T06:37:24.511 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.511 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.511 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.511 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.511 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.511 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.512 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.512 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.512 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-05-09T06:37:24.512 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-05-09T06:37:24.512 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:24.512 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.512 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.512 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-09T06:37:24.512 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.512 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.512 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-09T06:37:24.512 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-09T06:37:24.512 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-09T06:37:24.512 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-09T06:37:24.512 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-09T06:37:24.512 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-09T06:37:24.513 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-09T06:37:24.513 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-09T06:37:24.513 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.513 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.513 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.513 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.513 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.513 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-05-09T06:37:24.513 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-05-09T06:37:24.513 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:24.513 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.513 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.513 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-09T06:37:24.513 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.514 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.514 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.514 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.514 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.514 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.514 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.514 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.514 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-05-09T06:37:24.514 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-05-09T06:37:24.514 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:24.514 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.514 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.514 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-05-09T06:37:24.514 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.515 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.515 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-09T06:37:24.515 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-09T06:37:24.515 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-09T06:37:24.515 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-09T06:37:24.515 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-09T06:37:24.515 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-09T06:37:24.515 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-09T06:37:24.515 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-09T06:37:24.515 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.515 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.515 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.515 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.515 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.515 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-05-09T06:37:24.515 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-05-09T06:37:24.516 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:24.516 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.516 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.516 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-09T06:37:24.516 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.516 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.516 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.516 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.516 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.516 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.516 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.516 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.516 INFO:teuthology.orchestra.run.smithi003.stdout: "organization": { 2024-05-09T06:37:24.517 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "organization", 2024-05-09T06:37:24.517 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:24.517 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.517 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.517 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-09T06:37:24.517 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.517 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.517 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.517 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.517 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.517 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.517 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.517 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.517 INFO:teuthology.orchestra.run.smithi003.stdout: "proxy": { 2024-05-09T06:37:24.517 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "proxy", 2024-05-09T06:37:24.518 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:24.518 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.518 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.518 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-09T06:37:24.518 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.518 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.518 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.518 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.518 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.518 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.518 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.518 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.518 INFO:teuthology.orchestra.run.smithi003.stdout: "url": { 2024-05-09T06:37:24.518 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "url", 2024-05-09T06:37:24.518 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:24.519 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.519 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.519 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-05-09T06:37:24.519 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.519 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.519 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.519 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.519 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.519 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.519 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.519 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-09T06:37:24.519 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-09T06:37:24.519 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.520 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-09T06:37:24.520 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "test_orchestrator", 2024-05-09T06:37:24.520 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-05-09T06:37:24.520 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-05-09T06:37:24.520 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-05-09T06:37:24.520 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-05-09T06:37:24.520 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-05-09T06:37:24.520 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:24.520 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.520 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.520 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-09T06:37:24.520 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.520 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.520 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-09T06:37:24.520 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-09T06:37:24.521 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-09T06:37:24.521 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-09T06:37:24.521 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-09T06:37:24.521 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-09T06:37:24.521 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-09T06:37:24.521 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-09T06:37:24.521 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.521 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.521 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.521 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.521 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.521 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-05-09T06:37:24.521 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-05-09T06:37:24.521 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:24.521 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.522 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.522 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-09T06:37:24.522 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.522 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.522 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.522 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.522 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.522 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.522 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.522 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.522 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-05-09T06:37:24.522 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-05-09T06:37:24.522 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:24.522 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.522 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.523 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-05-09T06:37:24.523 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.523 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.523 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-09T06:37:24.523 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-09T06:37:24.523 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-09T06:37:24.523 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-09T06:37:24.523 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-09T06:37:24.523 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-09T06:37:24.523 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-09T06:37:24.523 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-09T06:37:24.523 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.523 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.523 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.523 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.524 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.524 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-05-09T06:37:24.524 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-05-09T06:37:24.524 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:24.524 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.524 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.524 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-09T06:37:24.524 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.524 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.524 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.524 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.524 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.524 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.524 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.524 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-09T06:37:24.525 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-09T06:37:24.525 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.525 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-09T06:37:24.525 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "volumes", 2024-05-09T06:37:24.525 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-05-09T06:37:24.525 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-05-09T06:37:24.525 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-05-09T06:37:24.525 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-05-09T06:37:24.525 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-05-09T06:37:24.525 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:24.525 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.525 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.525 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-09T06:37:24.525 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.526 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.526 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-09T06:37:24.526 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-09T06:37:24.526 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-09T06:37:24.526 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-09T06:37:24.526 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-09T06:37:24.526 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-09T06:37:24.526 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-09T06:37:24.526 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-09T06:37:24.526 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.526 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.526 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.526 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.526 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.527 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-05-09T06:37:24.527 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-05-09T06:37:24.527 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:24.527 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.527 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.527 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-09T06:37:24.527 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.527 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.527 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.527 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.527 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.527 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.527 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.527 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.527 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-05-09T06:37:24.528 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-05-09T06:37:24.528 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:24.528 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.528 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.528 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-05-09T06:37:24.528 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.528 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.528 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-09T06:37:24.528 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-09T06:37:24.528 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-09T06:37:24.528 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-09T06:37:24.528 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-09T06:37:24.528 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-09T06:37:24.528 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-09T06:37:24.529 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-09T06:37:24.529 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.529 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.529 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.529 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.529 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.529 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-05-09T06:37:24.529 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-05-09T06:37:24.529 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:24.529 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.529 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.529 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-09T06:37:24.529 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.529 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.530 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.530 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.530 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.530 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.530 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.530 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.530 INFO:teuthology.orchestra.run.smithi003.stdout: "max_concurrent_clones": { 2024-05-09T06:37:24.530 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "max_concurrent_clones", 2024-05-09T06:37:24.530 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-09T06:37:24.530 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.530 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.530 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "4", 2024-05-09T06:37:24.530 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.531 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.531 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.531 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Number of asynchronous cloner threads", 2024-05-09T06:37:24.531 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.531 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.531 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.531 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.531 INFO:teuthology.orchestra.run.smithi003.stdout: "periodic_async_work": { 2024-05-09T06:37:24.531 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "periodic_async_work", 2024-05-09T06:37:24.531 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:24.531 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.531 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.532 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-09T06:37:24.532 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.532 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.532 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.532 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Periodically check for async work", 2024-05-09T06:37:24.532 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.532 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.532 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.532 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.532 INFO:teuthology.orchestra.run.smithi003.stdout: "snapshot_clone_delay": { 2024-05-09T06:37:24.532 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "snapshot_clone_delay", 2024-05-09T06:37:24.532 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-09T06:37:24.532 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.532 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.532 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-05-09T06:37:24.533 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.533 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.533 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.533 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-05-09T06:37:24.533 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.533 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.533 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.533 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.533 INFO:teuthology.orchestra.run.smithi003.stdout: "snapshot_clone_no_wait": { 2024-05-09T06:37:24.533 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "snapshot_clone_no_wait", 2024-05-09T06:37:24.533 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:24.533 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.533 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.533 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-05-09T06:37:24.534 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.534 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.534 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.534 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-05-09T06:37:24.534 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.534 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.534 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.534 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-09T06:37:24.534 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-09T06:37:24.534 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.534 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-09T06:37:24.534 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "zabbix", 2024-05-09T06:37:24.534 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-05-09T06:37:24.534 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-05-09T06:37:24.535 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-05-09T06:37:24.535 INFO:teuthology.orchestra.run.smithi003.stdout: "discovery_interval": { 2024-05-09T06:37:24.535 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "discovery_interval", 2024-05-09T06:37:24.535 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "uint", 2024-05-09T06:37:24.535 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.535 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.535 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "100", 2024-05-09T06:37:24.535 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.535 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.535 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.535 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.535 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.535 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.535 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.535 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.535 INFO:teuthology.orchestra.run.smithi003.stdout: "identifier": { 2024-05-09T06:37:24.536 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "identifier", 2024-05-09T06:37:24.536 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:24.536 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.536 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.536 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-09T06:37:24.536 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.536 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.536 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.536 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.536 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.536 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.536 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.536 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.536 INFO:teuthology.orchestra.run.smithi003.stdout: "interval": { 2024-05-09T06:37:24.536 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "interval", 2024-05-09T06:37:24.536 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-05-09T06:37:24.537 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.537 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.537 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "60", 2024-05-09T06:37:24.537 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.537 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.537 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.537 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.537 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.537 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.537 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.537 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.537 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-05-09T06:37:24.537 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-05-09T06:37:24.537 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:24.537 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.537 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.538 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-09T06:37:24.538 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.538 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.538 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-09T06:37:24.538 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-09T06:37:24.538 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-09T06:37:24.538 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-09T06:37:24.538 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-09T06:37:24.538 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-09T06:37:24.538 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-09T06:37:24.538 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-09T06:37:24.538 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.538 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.539 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.539 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.539 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.539 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-05-09T06:37:24.539 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-05-09T06:37:24.539 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:24.539 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.539 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.539 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-09T06:37:24.539 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.539 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.539 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.539 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.539 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.539 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.539 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.539 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.540 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-05-09T06:37:24.540 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-05-09T06:37:24.540 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:24.540 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.540 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.540 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-05-09T06:37:24.540 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.540 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.540 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-09T06:37:24.540 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-09T06:37:24.540 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-09T06:37:24.540 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-09T06:37:24.540 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-09T06:37:24.540 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-09T06:37:24.541 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-09T06:37:24.541 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-09T06:37:24.541 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.541 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.541 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.541 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.541 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.541 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-05-09T06:37:24.541 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-05-09T06:37:24.541 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-09T06:37:24.541 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.541 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-09T06:37:24.541 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-09T06:37:24.541 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.541 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.541 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.542 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.542 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.542 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.542 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.542 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.542 INFO:teuthology.orchestra.run.smithi003.stdout: "zabbix_host": { 2024-05-09T06:37:24.542 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "zabbix_host", 2024-05-09T06:37:24.542 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:24.542 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.542 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.542 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-09T06:37:24.542 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.542 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.542 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.542 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.543 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.543 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.543 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.543 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.543 INFO:teuthology.orchestra.run.smithi003.stdout: "zabbix_port": { 2024-05-09T06:37:24.543 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "zabbix_port", 2024-05-09T06:37:24.543 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-09T06:37:24.543 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.543 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.543 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "10051", 2024-05-09T06:37:24.543 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.543 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.543 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.543 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.544 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.544 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.544 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.544 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.544 INFO:teuthology.orchestra.run.smithi003.stdout: "zabbix_sender": { 2024-05-09T06:37:24.544 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "zabbix_sender", 2024-05-09T06:37:24.544 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-09T06:37:24.544 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-09T06:37:24.544 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-09T06:37:24.544 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-05-09T06:37:24.544 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-09T06:37:24.544 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-09T06:37:24.544 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-09T06:37:24.544 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-09T06:37:24.544 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-09T06:37:24.545 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-09T06:37:24.545 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-09T06:37:24.545 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-09T06:37:24.545 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-09T06:37:24.545 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-09T06:37:24.545 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-09T06:37:24.545 INFO:teuthology.orchestra.run.smithi003.stdout: "services": {}, 2024-05-09T06:37:24.545 INFO:teuthology.orchestra.run.smithi003.stdout: "always_on_modules": { 2024-05-09T06:37:24.545 INFO:teuthology.orchestra.run.smithi003.stdout: "octopus": [ 2024-05-09T06:37:24.545 INFO:teuthology.orchestra.run.smithi003.stdout: "balancer", 2024-05-09T06:37:24.545 INFO:teuthology.orchestra.run.smithi003.stdout: "crash", 2024-05-09T06:37:24.545 INFO:teuthology.orchestra.run.smithi003.stdout: "devicehealth", 2024-05-09T06:37:24.545 INFO:teuthology.orchestra.run.smithi003.stdout: "orchestrator", 2024-05-09T06:37:24.545 INFO:teuthology.orchestra.run.smithi003.stdout: "pg_autoscaler", 2024-05-09T06:37:24.545 INFO:teuthology.orchestra.run.smithi003.stdout: "progress", 2024-05-09T06:37:24.546 INFO:teuthology.orchestra.run.smithi003.stdout: "rbd_support", 2024-05-09T06:37:24.546 INFO:teuthology.orchestra.run.smithi003.stdout: "status", 2024-05-09T06:37:24.546 INFO:teuthology.orchestra.run.smithi003.stdout: "telemetry", 2024-05-09T06:37:24.546 INFO:teuthology.orchestra.run.smithi003.stdout: "volumes" 2024-05-09T06:37:24.546 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-09T06:37:24.546 INFO:teuthology.orchestra.run.smithi003.stdout: "pacific": [ 2024-05-09T06:37:24.546 INFO:teuthology.orchestra.run.smithi003.stdout: "balancer", 2024-05-09T06:37:24.546 INFO:teuthology.orchestra.run.smithi003.stdout: "crash", 2024-05-09T06:37:24.546 INFO:teuthology.orchestra.run.smithi003.stdout: "devicehealth", 2024-05-09T06:37:24.546 INFO:teuthology.orchestra.run.smithi003.stdout: "orchestrator", 2024-05-09T06:37:24.546 INFO:teuthology.orchestra.run.smithi003.stdout: "pg_autoscaler", 2024-05-09T06:37:24.546 INFO:teuthology.orchestra.run.smithi003.stdout: "progress", 2024-05-09T06:37:24.546 INFO:teuthology.orchestra.run.smithi003.stdout: "rbd_support", 2024-05-09T06:37:24.546 INFO:teuthology.orchestra.run.smithi003.stdout: "status", 2024-05-09T06:37:24.546 INFO:teuthology.orchestra.run.smithi003.stdout: "telemetry", 2024-05-09T06:37:24.547 INFO:teuthology.orchestra.run.smithi003.stdout: "volumes" 2024-05-09T06:37:24.547 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-09T06:37:24.547 INFO:teuthology.orchestra.run.smithi003.stdout: "quincy": [ 2024-05-09T06:37:24.547 INFO:teuthology.orchestra.run.smithi003.stdout: "balancer", 2024-05-09T06:37:24.547 INFO:teuthology.orchestra.run.smithi003.stdout: "crash", 2024-05-09T06:37:24.547 INFO:teuthology.orchestra.run.smithi003.stdout: "devicehealth", 2024-05-09T06:37:24.547 INFO:teuthology.orchestra.run.smithi003.stdout: "orchestrator", 2024-05-09T06:37:24.547 INFO:teuthology.orchestra.run.smithi003.stdout: "pg_autoscaler", 2024-05-09T06:37:24.547 INFO:teuthology.orchestra.run.smithi003.stdout: "progress", 2024-05-09T06:37:24.547 INFO:teuthology.orchestra.run.smithi003.stdout: "rbd_support", 2024-05-09T06:37:24.547 INFO:teuthology.orchestra.run.smithi003.stdout: "status", 2024-05-09T06:37:24.547 INFO:teuthology.orchestra.run.smithi003.stdout: "telemetry", 2024-05-09T06:37:24.547 INFO:teuthology.orchestra.run.smithi003.stdout: "volumes" 2024-05-09T06:37:24.547 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-09T06:37:24.547 INFO:teuthology.orchestra.run.smithi003.stdout: "reef": [ 2024-05-09T06:37:24.548 INFO:teuthology.orchestra.run.smithi003.stdout: "balancer", 2024-05-09T06:37:24.548 INFO:teuthology.orchestra.run.smithi003.stdout: "crash", 2024-05-09T06:37:24.548 INFO:teuthology.orchestra.run.smithi003.stdout: "devicehealth", 2024-05-09T06:37:24.548 INFO:teuthology.orchestra.run.smithi003.stdout: "orchestrator", 2024-05-09T06:37:24.548 INFO:teuthology.orchestra.run.smithi003.stdout: "pg_autoscaler", 2024-05-09T06:37:24.548 INFO:teuthology.orchestra.run.smithi003.stdout: "progress", 2024-05-09T06:37:24.548 INFO:teuthology.orchestra.run.smithi003.stdout: "rbd_support", 2024-05-09T06:37:24.548 INFO:teuthology.orchestra.run.smithi003.stdout: "status", 2024-05-09T06:37:24.548 INFO:teuthology.orchestra.run.smithi003.stdout: "telemetry", 2024-05-09T06:37:24.548 INFO:teuthology.orchestra.run.smithi003.stdout: "volumes" 2024-05-09T06:37:24.548 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-09T06:37:24.548 INFO:teuthology.orchestra.run.smithi003.stdout: "squid": [ 2024-05-09T06:37:24.548 INFO:teuthology.orchestra.run.smithi003.stdout: "balancer", 2024-05-09T06:37:24.548 INFO:teuthology.orchestra.run.smithi003.stdout: "crash", 2024-05-09T06:37:24.548 INFO:teuthology.orchestra.run.smithi003.stdout: "devicehealth", 2024-05-09T06:37:24.548 INFO:teuthology.orchestra.run.smithi003.stdout: "orchestrator", 2024-05-09T06:37:24.549 INFO:teuthology.orchestra.run.smithi003.stdout: "pg_autoscaler", 2024-05-09T06:37:24.549 INFO:teuthology.orchestra.run.smithi003.stdout: "progress", 2024-05-09T06:37:24.549 INFO:teuthology.orchestra.run.smithi003.stdout: "rbd_support", 2024-05-09T06:37:24.549 INFO:teuthology.orchestra.run.smithi003.stdout: "status", 2024-05-09T06:37:24.549 INFO:teuthology.orchestra.run.smithi003.stdout: "telemetry", 2024-05-09T06:37:24.549 INFO:teuthology.orchestra.run.smithi003.stdout: "volumes" 2024-05-09T06:37:24.549 INFO:teuthology.orchestra.run.smithi003.stdout: ] 2024-05-09T06:37:24.549 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.549 INFO:teuthology.orchestra.run.smithi003.stdout: "last_failure_osd_epoch": 0, 2024-05-09T06:37:24.549 INFO:teuthology.orchestra.run.smithi003.stdout: "active_clients": [ 2024-05-09T06:37:24.549 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-09T06:37:24.549 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "devicehealth", 2024-05-09T06:37:24.549 INFO:teuthology.orchestra.run.smithi003.stdout: "addrvec": [ 2024-05-09T06:37:24.549 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-09T06:37:24.549 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "v2", 2024-05-09T06:37:24.549 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": "172.21.15.190:0", 2024-05-09T06:37:24.550 INFO:teuthology.orchestra.run.smithi003.stdout: "nonce": 658212118 2024-05-09T06:37:24.550 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-09T06:37:24.550 INFO:teuthology.orchestra.run.smithi003.stdout: ] 2024-05-09T06:37:24.550 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.550 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-09T06:37:24.550 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "libcephsqlite", 2024-05-09T06:37:24.550 INFO:teuthology.orchestra.run.smithi003.stdout: "addrvec": [ 2024-05-09T06:37:24.550 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-09T06:37:24.550 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "v2", 2024-05-09T06:37:24.550 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": "172.21.15.190:0", 2024-05-09T06:37:24.550 INFO:teuthology.orchestra.run.smithi003.stdout: "nonce": 2567150848 2024-05-09T06:37:24.550 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-09T06:37:24.550 INFO:teuthology.orchestra.run.smithi003.stdout: ] 2024-05-09T06:37:24.550 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.550 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-09T06:37:24.551 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rbd_support", 2024-05-09T06:37:24.551 INFO:teuthology.orchestra.run.smithi003.stdout: "addrvec": [ 2024-05-09T06:37:24.551 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-09T06:37:24.551 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "v2", 2024-05-09T06:37:24.551 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": "172.21.15.190:0", 2024-05-09T06:37:24.551 INFO:teuthology.orchestra.run.smithi003.stdout: "nonce": 2582713269 2024-05-09T06:37:24.551 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-09T06:37:24.551 INFO:teuthology.orchestra.run.smithi003.stdout: ] 2024-05-09T06:37:24.551 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.551 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-09T06:37:24.551 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "volumes", 2024-05-09T06:37:24.551 INFO:teuthology.orchestra.run.smithi003.stdout: "addrvec": [ 2024-05-09T06:37:24.551 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-09T06:37:24.551 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "v2", 2024-05-09T06:37:24.552 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": "172.21.15.190:0", 2024-05-09T06:37:24.552 INFO:teuthology.orchestra.run.smithi003.stdout: "nonce": 3766699314 2024-05-09T06:37:24.552 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-09T06:37:24.552 INFO:teuthology.orchestra.run.smithi003.stdout: ] 2024-05-09T06:37:24.552 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.552 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-09T06:37:24.552 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "volumes", 2024-05-09T06:37:24.552 INFO:teuthology.orchestra.run.smithi003.stdout: "addrvec": [ 2024-05-09T06:37:24.552 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-09T06:37:24.552 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "v2", 2024-05-09T06:37:24.552 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": "172.21.15.190:0", 2024-05-09T06:37:24.552 INFO:teuthology.orchestra.run.smithi003.stdout: "nonce": 3301650661 2024-05-09T06:37:24.552 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-09T06:37:24.552 INFO:teuthology.orchestra.run.smithi003.stdout: ] 2024-05-09T06:37:24.553 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.553 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-09T06:37:24.553 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "volumes", 2024-05-09T06:37:24.553 INFO:teuthology.orchestra.run.smithi003.stdout: "addrvec": [ 2024-05-09T06:37:24.553 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-09T06:37:24.553 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "v2", 2024-05-09T06:37:24.553 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": "172.21.15.190:0", 2024-05-09T06:37:24.553 INFO:teuthology.orchestra.run.smithi003.stdout: "nonce": 557617456 2024-05-09T06:37:24.553 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-09T06:37:24.553 INFO:teuthology.orchestra.run.smithi003.stdout: ] 2024-05-09T06:37:24.553 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-09T06:37:24.553 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-09T06:37:24.553 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "volumes", 2024-05-09T06:37:24.553 INFO:teuthology.orchestra.run.smithi003.stdout: "addrvec": [ 2024-05-09T06:37:24.553 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-09T06:37:24.553 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "v2", 2024-05-09T06:37:24.554 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": "172.21.15.190:0", 2024-05-09T06:37:24.554 INFO:teuthology.orchestra.run.smithi003.stdout: "nonce": 1644555879 2024-05-09T06:37:24.554 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-09T06:37:24.554 INFO:teuthology.orchestra.run.smithi003.stdout: ] 2024-05-09T06:37:24.554 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-09T06:37:24.554 INFO:teuthology.orchestra.run.smithi003.stdout: ] 2024-05-09T06:37:24.554 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-05-09T06:37:24.556 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:23.727+0000 7f83a228c640 1 -- 172.21.15.3:0/2899903044 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f838806bfe0 msgr2=0x7f838806e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:24.556 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:23.727+0000 7f83a228c640 1 --2- 172.21.15.3:0/2899903044 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f838806bfe0 0x7f838806e4a0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f839c156220 tx=0x7f8390008040 comp rx=0 tx=0).stop 2024-05-09T06:37:24.556 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:23.727+0000 7f83a228c640 1 -- 172.21.15.3:0/2899903044 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f83840039b0 msgr2=0x7f839c13f300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:24.556 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:23.727+0000 7f83a228c640 1 --2- 172.21.15.3:0/2899903044 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f83840039b0 0x7f839c13f300 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f838c00bb80 tx=0x7f838c004730 comp rx=0 tx=0).stop 2024-05-09T06:37:24.556 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:23.727+0000 7f83a228c640 1 -- 172.21.15.3:0/2899903044 shutdown_connections 2024-05-09T06:37:24.556 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:23.727+0000 7f83a228c640 1 --2- 172.21.15.3:0/2899903044 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f838806bfe0 0x7f838806e4a0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:24.556 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:23.727+0000 7f83a228c640 1 --2- 172.21.15.3:0/2899903044 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f839c1414e0 0x7f839c142ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:24.556 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:23.727+0000 7f83a228c640 1 --2- 172.21.15.3:0/2899903044 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f839c13f9b0 0x7f839c140fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:24.556 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:23.727+0000 7f83a228c640 1 --2- 172.21.15.3:0/2899903044 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f83840039b0 0x7f839c13f300 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:24.556 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:23.727+0000 7f83a228c640 1 -- 172.21.15.3:0/2899903044 >> 172.21.15.3:0/2899903044 conn(0x7f839c06b430 msgr2=0x7f839c06b810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:24.556 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:23.728+0000 7f83a228c640 1 -- 172.21.15.3:0/2899903044 shutdown_connections 2024-05-09T06:37:24.556 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:23.728+0000 7f83a228c640 1 -- 172.21.15.3:0/2899903044 wait complete. 2024-05-09T06:37:24.562 DEBUG:tasks.check_counter:Getting stats from x 2024-05-09T06:37:24.562 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mgr.x.asok perf dump 2024-05-09T06:37:24.674 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-05-09T06:37:24.675 INFO:teuthology.orchestra.run.smithi190.stdout: "AsyncMessenger::Worker-0": { 2024-05-09T06:37:24.675 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_messages": 265, 2024-05-09T06:37:24.675 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_messages": 70, 2024-05-09T06:37:24.675 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_bytes": 3520370, 2024-05-09T06:37:24.675 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_bytes": 54477, 2024-05-09T06:37:24.675 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_created_connections": 92, 2024-05-09T06:37:24.675 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_active_connections": 14, 2024-05-09T06:37:24.675 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_total_time": 0.065144000, 2024-05-09T06:37:24.675 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_send_time": 0.001971816, 2024-05-09T06:37:24.675 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_recv_time": 0.061960776, 2024-05-09T06:37:24.675 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_fast_dispatch_time": 0.000036715, 2024-05-09T06:37:24.675 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T06:37:24.675 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 54, 2024-05-09T06:37:24.675 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.208557042, 2024-05-09T06:37:24.676 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.003862167 2024-05-09T06:37:24.676 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-05-09T06:37:24.676 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_handle_ack_lat": { 2024-05-09T06:37:24.676 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-05-09T06:37:24.676 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-05-09T06:37:24.676 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-05-09T06:37:24.676 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-05-09T06:37:24.676 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_encrypted_bytes": 3519488, 2024-05-09T06:37:24.676 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_encrypted_bytes": 53968 2024-05-09T06:37:24.676 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-05-09T06:37:24.676 INFO:teuthology.orchestra.run.smithi190.stdout: "AsyncMessenger::Worker-1": { 2024-05-09T06:37:24.676 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_messages": 382, 2024-05-09T06:37:24.676 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_messages": 213, 2024-05-09T06:37:24.676 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_bytes": 2110704, 2024-05-09T06:37:24.677 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_bytes": 331632, 2024-05-09T06:37:24.677 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_created_connections": 96, 2024-05-09T06:37:24.677 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_active_connections": 18, 2024-05-09T06:37:24.677 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_total_time": 0.060750423, 2024-05-09T06:37:24.677 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_send_time": 0.008399118, 2024-05-09T06:37:24.677 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_recv_time": 0.058101367, 2024-05-09T06:37:24.677 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_fast_dispatch_time": 0.000089921, 2024-05-09T06:37:24.677 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T06:37:24.677 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 199, 2024-05-09T06:37:24.677 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.015277583, 2024-05-09T06:37:24.677 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000076771 2024-05-09T06:37:24.677 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-05-09T06:37:24.677 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_handle_ack_lat": { 2024-05-09T06:37:24.677 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 281, 2024-05-09T06:37:24.677 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000036512, 2024-05-09T06:37:24.678 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000129 2024-05-09T06:37:24.678 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-05-09T06:37:24.678 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_encrypted_bytes": 2110704, 2024-05-09T06:37:24.678 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_encrypted_bytes": 331632 2024-05-09T06:37:24.678 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-05-09T06:37:24.678 INFO:teuthology.orchestra.run.smithi190.stdout: "AsyncMessenger::Worker-2": { 2024-05-09T06:37:24.678 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_messages": 797, 2024-05-09T06:37:24.678 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_messages": 613, 2024-05-09T06:37:24.678 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_bytes": 3290880, 2024-05-09T06:37:24.678 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_bytes": 5519344, 2024-05-09T06:37:24.678 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_created_connections": 148, 2024-05-09T06:37:24.678 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_active_connections": 24, 2024-05-09T06:37:24.678 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_total_time": 0.142072654, 2024-05-09T06:37:24.679 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_send_time": 0.032612165, 2024-05-09T06:37:24.679 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_recv_time": 0.129902205, 2024-05-09T06:37:24.679 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_fast_dispatch_time": 0.002790909, 2024-05-09T06:37:24.679 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T06:37:24.679 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 600, 2024-05-09T06:37:24.679 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.044220239, 2024-05-09T06:37:24.679 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000073700 2024-05-09T06:37:24.679 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-05-09T06:37:24.679 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_handle_ack_lat": { 2024-05-09T06:37:24.680 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 209, 2024-05-09T06:37:24.680 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000070076, 2024-05-09T06:37:24.680 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000335 2024-05-09T06:37:24.680 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-05-09T06:37:24.680 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_encrypted_bytes": 3290880, 2024-05-09T06:37:24.680 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_encrypted_bytes": 5519344 2024-05-09T06:37:24.680 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-05-09T06:37:24.680 INFO:teuthology.orchestra.run.smithi190.stdout: "cct": { 2024-05-09T06:37:24.680 INFO:teuthology.orchestra.run.smithi190.stdout: "total_workers": 0, 2024-05-09T06:37:24.680 INFO:teuthology.orchestra.run.smithi190.stdout: "unhealthy_workers": 0 2024-05-09T06:37:24.680 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-05-09T06:37:24.680 INFO:teuthology.orchestra.run.smithi190.stdout: "client": { 2024-05-09T06:37:24.680 INFO:teuthology.orchestra.run.smithi190.stdout: "reply": { 2024-05-09T06:37:24.680 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-05-09T06:37:24.681 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-05-09T06:37:24.681 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-05-09T06:37:24.681 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-05-09T06:37:24.681 INFO:teuthology.orchestra.run.smithi190.stdout: "lat": { 2024-05-09T06:37:24.681 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-05-09T06:37:24.681 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-05-09T06:37:24.681 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-05-09T06:37:24.681 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-05-09T06:37:24.681 INFO:teuthology.orchestra.run.smithi190.stdout: "wrlat": { 2024-05-09T06:37:24.681 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-05-09T06:37:24.681 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-05-09T06:37:24.681 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-05-09T06:37:24.681 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-05-09T06:37:24.681 INFO:teuthology.orchestra.run.smithi190.stdout: "rdlat": { 2024-05-09T06:37:24.682 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-05-09T06:37:24.682 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-05-09T06:37:24.682 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-05-09T06:37:24.682 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-05-09T06:37:24.682 INFO:teuthology.orchestra.run.smithi190.stdout: "fsync": { 2024-05-09T06:37:24.682 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-05-09T06:37:24.682 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-05-09T06:37:24.682 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-05-09T06:37:24.682 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-05-09T06:37:24.682 INFO:teuthology.orchestra.run.smithi190.stdout: "mdavg": 0.000000000, 2024-05-09T06:37:24.682 INFO:teuthology.orchestra.run.smithi190.stdout: "mdsqsum": 0, 2024-05-09T06:37:24.682 INFO:teuthology.orchestra.run.smithi190.stdout: "mdops": 0, 2024-05-09T06:37:24.682 INFO:teuthology.orchestra.run.smithi190.stdout: "readavg": 0.000000000, 2024-05-09T06:37:24.682 INFO:teuthology.orchestra.run.smithi190.stdout: "readsqsum": 0, 2024-05-09T06:37:24.682 INFO:teuthology.orchestra.run.smithi190.stdout: "rdops": 0, 2024-05-09T06:37:24.683 INFO:teuthology.orchestra.run.smithi190.stdout: "writeavg": 0.000000000, 2024-05-09T06:37:24.683 INFO:teuthology.orchestra.run.smithi190.stdout: "writesqsum": 0, 2024-05-09T06:37:24.683 INFO:teuthology.orchestra.run.smithi190.stdout: "rdops": 0 2024-05-09T06:37:24.683 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-05-09T06:37:24.683 INFO:teuthology.orchestra.run.smithi190.stdout: "client-0x55b09f6c2460": { 2024-05-09T06:37:24.683 INFO:teuthology.orchestra.run.smithi190.stdout: "reply": { 2024-05-09T06:37:24.683 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 57, 2024-05-09T06:37:24.683 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.072492119, 2024-05-09T06:37:24.683 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.001271791 2024-05-09T06:37:24.683 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-05-09T06:37:24.683 INFO:teuthology.orchestra.run.smithi190.stdout: "lat": { 2024-05-09T06:37:24.683 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 57, 2024-05-09T06:37:24.683 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.072492119, 2024-05-09T06:37:24.683 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.001271791 2024-05-09T06:37:24.684 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-05-09T06:37:24.684 INFO:teuthology.orchestra.run.smithi190.stdout: "wrlat": { 2024-05-09T06:37:24.684 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-05-09T06:37:24.684 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-05-09T06:37:24.684 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-05-09T06:37:24.684 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-05-09T06:37:24.684 INFO:teuthology.orchestra.run.smithi190.stdout: "rdlat": { 2024-05-09T06:37:24.684 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-05-09T06:37:24.684 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-05-09T06:37:24.684 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-05-09T06:37:24.684 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-05-09T06:37:24.684 INFO:teuthology.orchestra.run.smithi190.stdout: "fsync": { 2024-05-09T06:37:24.685 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-05-09T06:37:24.685 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-05-09T06:37:24.685 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-05-09T06:37:24.685 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-05-09T06:37:24.685 INFO:teuthology.orchestra.run.smithi190.stdout: "mdavg": 0.001271776, 2024-05-09T06:37:24.685 INFO:teuthology.orchestra.run.smithi190.stdout: "mdsqsum": 165168477475723, 2024-05-09T06:37:24.685 INFO:teuthology.orchestra.run.smithi190.stdout: "mdops": 57, 2024-05-09T06:37:24.685 INFO:teuthology.orchestra.run.smithi190.stdout: "readavg": 0.000000000, 2024-05-09T06:37:24.685 INFO:teuthology.orchestra.run.smithi190.stdout: "readsqsum": 0, 2024-05-09T06:37:24.685 INFO:teuthology.orchestra.run.smithi190.stdout: "rdops": 0, 2024-05-09T06:37:24.685 INFO:teuthology.orchestra.run.smithi190.stdout: "writeavg": 0.000000000, 2024-05-09T06:37:24.685 INFO:teuthology.orchestra.run.smithi190.stdout: "writesqsum": 0, 2024-05-09T06:37:24.685 INFO:teuthology.orchestra.run.smithi190.stdout: "rdops": 0 2024-05-09T06:37:24.686 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-05-09T06:37:24.686 INFO:teuthology.orchestra.run.smithi190.stdout: "client-0x55b09f6c2aa0": { 2024-05-09T06:37:24.686 INFO:teuthology.orchestra.run.smithi190.stdout: "reply": { 2024-05-09T06:37:24.686 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 68, 2024-05-09T06:37:24.686 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.088834254, 2024-05-09T06:37:24.686 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.001306386 2024-05-09T06:37:24.686 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-05-09T06:37:24.686 INFO:teuthology.orchestra.run.smithi190.stdout: "lat": { 2024-05-09T06:37:24.687 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 68, 2024-05-09T06:37:24.687 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.088834254, 2024-05-09T06:37:24.687 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.001306386 2024-05-09T06:37:24.687 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-05-09T06:37:24.687 INFO:teuthology.orchestra.run.smithi190.stdout: "wrlat": { 2024-05-09T06:37:24.687 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 18, 2024-05-09T06:37:24.687 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000371198, 2024-05-09T06:37:24.687 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000020622 2024-05-09T06:37:24.687 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-05-09T06:37:24.687 INFO:teuthology.orchestra.run.smithi190.stdout: "rdlat": { 2024-05-09T06:37:24.687 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 18, 2024-05-09T06:37:24.687 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000348565, 2024-05-09T06:37:24.687 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000019364 2024-05-09T06:37:24.687 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-05-09T06:37:24.688 INFO:teuthology.orchestra.run.smithi190.stdout: "fsync": { 2024-05-09T06:37:24.688 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 3, 2024-05-09T06:37:24.688 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.012415030, 2024-05-09T06:37:24.688 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.004138343 2024-05-09T06:37:24.688 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-05-09T06:37:24.688 INFO:teuthology.orchestra.run.smithi190.stdout: "mdavg": 0.001306370, 2024-05-09T06:37:24.688 INFO:teuthology.orchestra.run.smithi190.stdout: "mdsqsum": 104407478782889, 2024-05-09T06:37:24.688 INFO:teuthology.orchestra.run.smithi190.stdout: "mdops": 68, 2024-05-09T06:37:24.688 INFO:teuthology.orchestra.run.smithi190.stdout: "readavg": 0.000019361, 2024-05-09T06:37:24.688 INFO:teuthology.orchestra.run.smithi190.stdout: "readsqsum": 1638438348, 2024-05-09T06:37:24.689 INFO:teuthology.orchestra.run.smithi190.stdout: "rdops": 18, 2024-05-09T06:37:24.689 INFO:teuthology.orchestra.run.smithi190.stdout: "writeavg": 0.000020618, 2024-05-09T06:37:24.689 INFO:teuthology.orchestra.run.smithi190.stdout: "writesqsum": 1710497965, 2024-05-09T06:37:24.689 INFO:teuthology.orchestra.run.smithi190.stdout: "rdops": 18 2024-05-09T06:37:24.689 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-05-09T06:37:24.689 INFO:teuthology.orchestra.run.smithi190.stdout: "client-0x55b09f6c3180": { 2024-05-09T06:37:24.689 INFO:teuthology.orchestra.run.smithi190.stdout: "reply": { 2024-05-09T06:37:24.689 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 20, 2024-05-09T06:37:24.689 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.342545281, 2024-05-09T06:37:24.689 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.017127264 2024-05-09T06:37:24.689 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-05-09T06:37:24.689 INFO:teuthology.orchestra.run.smithi190.stdout: "lat": { 2024-05-09T06:37:24.689 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 20, 2024-05-09T06:37:24.689 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.342545281, 2024-05-09T06:37:24.690 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.017127264 2024-05-09T06:37:24.690 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-05-09T06:37:24.690 INFO:teuthology.orchestra.run.smithi190.stdout: "wrlat": { 2024-05-09T06:37:24.690 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-05-09T06:37:24.690 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-05-09T06:37:24.690 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-05-09T06:37:24.690 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-05-09T06:37:24.690 INFO:teuthology.orchestra.run.smithi190.stdout: "rdlat": { 2024-05-09T06:37:24.690 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-05-09T06:37:24.690 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-05-09T06:37:24.690 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-05-09T06:37:24.690 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-05-09T06:37:24.690 INFO:teuthology.orchestra.run.smithi190.stdout: "fsync": { 2024-05-09T06:37:24.691 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-05-09T06:37:24.691 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-05-09T06:37:24.691 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-05-09T06:37:24.691 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-05-09T06:37:24.691 INFO:teuthology.orchestra.run.smithi190.stdout: "mdavg": 0.017127259, 2024-05-09T06:37:24.691 INFO:teuthology.orchestra.run.smithi190.stdout: "mdsqsum": 103010011325986688, 2024-05-09T06:37:24.691 INFO:teuthology.orchestra.run.smithi190.stdout: "mdops": 20, 2024-05-09T06:37:24.691 INFO:teuthology.orchestra.run.smithi190.stdout: "readavg": 0.000000000, 2024-05-09T06:37:24.691 INFO:teuthology.orchestra.run.smithi190.stdout: "readsqsum": 0, 2024-05-09T06:37:24.691 INFO:teuthology.orchestra.run.smithi190.stdout: "rdops": 0, 2024-05-09T06:37:24.691 INFO:teuthology.orchestra.run.smithi190.stdout: "writeavg": 0.000000000, 2024-05-09T06:37:24.691 INFO:teuthology.orchestra.run.smithi190.stdout: "writesqsum": 0, 2024-05-09T06:37:24.691 INFO:teuthology.orchestra.run.smithi190.stdout: "rdops": 0 2024-05-09T06:37:24.691 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-05-09T06:37:24.692 INFO:teuthology.orchestra.run.smithi190.stdout: "finisher-Mgr": { 2024-05-09T06:37:24.692 INFO:teuthology.orchestra.run.smithi190.stdout: "queue_len": 0, 2024-05-09T06:37:24.692 INFO:teuthology.orchestra.run.smithi190.stdout: "complete_latency": { 2024-05-09T06:37:24.692 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 2, 2024-05-09T06:37:24.692 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.032582177, 2024-05-09T06:37:24.692 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.016291088 2024-05-09T06:37:24.692 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-05-09T06:37:24.692 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-05-09T06:37:24.692 INFO:teuthology.orchestra.run.smithi190.stdout: "finisher-MgrStandby": { 2024-05-09T06:37:24.692 INFO:teuthology.orchestra.run.smithi190.stdout: "queue_len": 0, 2024-05-09T06:37:24.692 INFO:teuthology.orchestra.run.smithi190.stdout: "complete_latency": { 2024-05-09T06:37:24.692 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-05-09T06:37:24.692 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-05-09T06:37:24.692 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-05-09T06:37:24.692 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-05-09T06:37:24.693 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-05-09T06:37:24.693 INFO:teuthology.orchestra.run.smithi190.stdout: "finisher-balancer": { 2024-05-09T06:37:24.693 INFO:teuthology.orchestra.run.smithi190.stdout: "queue_len": 0, 2024-05-09T06:37:24.693 INFO:teuthology.orchestra.run.smithi190.stdout: "complete_latency": { 2024-05-09T06:37:24.693 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-05-09T06:37:24.693 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-05-09T06:37:24.693 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-05-09T06:37:24.693 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-05-09T06:37:24.693 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-05-09T06:37:24.693 INFO:teuthology.orchestra.run.smithi190.stdout: "finisher-cmd_finisher": { 2024-05-09T06:37:24.693 INFO:teuthology.orchestra.run.smithi190.stdout: "queue_len": 0, 2024-05-09T06:37:24.694 INFO:teuthology.orchestra.run.smithi190.stdout: "complete_latency": { 2024-05-09T06:37:24.694 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 15, 2024-05-09T06:37:24.694 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000931035, 2024-05-09T06:37:24.694 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000062069 2024-05-09T06:37:24.694 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-05-09T06:37:24.694 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-05-09T06:37:24.694 INFO:teuthology.orchestra.run.smithi190.stdout: "finisher-crash": { 2024-05-09T06:37:24.694 INFO:teuthology.orchestra.run.smithi190.stdout: "queue_len": 0, 2024-05-09T06:37:24.694 INFO:teuthology.orchestra.run.smithi190.stdout: "complete_latency": { 2024-05-09T06:37:24.694 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-05-09T06:37:24.694 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-05-09T06:37:24.694 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-05-09T06:37:24.695 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-05-09T06:37:24.695 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-05-09T06:37:24.695 INFO:teuthology.orchestra.run.smithi190.stdout: "finisher-devicehealth": { 2024-05-09T06:37:24.695 INFO:teuthology.orchestra.run.smithi190.stdout: "queue_len": 0, 2024-05-09T06:37:24.695 INFO:teuthology.orchestra.run.smithi190.stdout: "complete_latency": { 2024-05-09T06:37:24.695 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-05-09T06:37:24.695 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-05-09T06:37:24.695 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-05-09T06:37:24.695 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-05-09T06:37:24.695 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-05-09T06:37:24.695 INFO:teuthology.orchestra.run.smithi190.stdout: "finisher-iostat": { 2024-05-09T06:37:24.695 INFO:teuthology.orchestra.run.smithi190.stdout: "queue_len": 0, 2024-05-09T06:37:24.695 INFO:teuthology.orchestra.run.smithi190.stdout: "complete_latency": { 2024-05-09T06:37:24.695 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-05-09T06:37:24.696 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-05-09T06:37:24.696 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-05-09T06:37:24.696 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-05-09T06:37:24.696 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-05-09T06:37:24.696 INFO:teuthology.orchestra.run.smithi190.stdout: "finisher-nfs": { 2024-05-09T06:37:24.696 INFO:teuthology.orchestra.run.smithi190.stdout: "queue_len": 0, 2024-05-09T06:37:24.696 INFO:teuthology.orchestra.run.smithi190.stdout: "complete_latency": { 2024-05-09T06:37:24.696 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-05-09T06:37:24.696 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-05-09T06:37:24.696 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-05-09T06:37:24.696 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-05-09T06:37:24.696 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-05-09T06:37:24.696 INFO:teuthology.orchestra.run.smithi190.stdout: "finisher-orchestrator": { 2024-05-09T06:37:24.697 INFO:teuthology.orchestra.run.smithi190.stdout: "queue_len": 0, 2024-05-09T06:37:24.697 INFO:teuthology.orchestra.run.smithi190.stdout: "complete_latency": { 2024-05-09T06:37:24.697 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-05-09T06:37:24.697 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-05-09T06:37:24.697 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-05-09T06:37:24.697 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-05-09T06:37:24.697 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-05-09T06:37:24.697 INFO:teuthology.orchestra.run.smithi190.stdout: "finisher-pg_autoscaler": { 2024-05-09T06:37:24.697 INFO:teuthology.orchestra.run.smithi190.stdout: "queue_len": 0, 2024-05-09T06:37:24.697 INFO:teuthology.orchestra.run.smithi190.stdout: "complete_latency": { 2024-05-09T06:37:24.697 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-05-09T06:37:24.697 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-05-09T06:37:24.697 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-05-09T06:37:24.697 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-05-09T06:37:24.697 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-05-09T06:37:24.698 INFO:teuthology.orchestra.run.smithi190.stdout: "finisher-progress": { 2024-05-09T06:37:24.698 INFO:teuthology.orchestra.run.smithi190.stdout: "queue_len": 0, 2024-05-09T06:37:24.698 INFO:teuthology.orchestra.run.smithi190.stdout: "complete_latency": { 2024-05-09T06:37:24.698 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-05-09T06:37:24.698 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-05-09T06:37:24.698 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-05-09T06:37:24.698 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-05-09T06:37:24.698 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-05-09T06:37:24.698 INFO:teuthology.orchestra.run.smithi190.stdout: "finisher-rbd_support": { 2024-05-09T06:37:24.698 INFO:teuthology.orchestra.run.smithi190.stdout: "queue_len": 0, 2024-05-09T06:37:24.698 INFO:teuthology.orchestra.run.smithi190.stdout: "complete_latency": { 2024-05-09T06:37:24.698 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-05-09T06:37:24.698 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-05-09T06:37:24.699 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-05-09T06:37:24.699 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-05-09T06:37:24.699 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-05-09T06:37:24.699 INFO:teuthology.orchestra.run.smithi190.stdout: "finisher-restful": { 2024-05-09T06:37:24.699 INFO:teuthology.orchestra.run.smithi190.stdout: "queue_len": 0, 2024-05-09T06:37:24.699 INFO:teuthology.orchestra.run.smithi190.stdout: "complete_latency": { 2024-05-09T06:37:24.699 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 16, 2024-05-09T06:37:24.699 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.003055983, 2024-05-09T06:37:24.699 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000190998 2024-05-09T06:37:24.699 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-05-09T06:37:24.699 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-05-09T06:37:24.699 INFO:teuthology.orchestra.run.smithi190.stdout: "finisher-status": { 2024-05-09T06:37:24.699 INFO:teuthology.orchestra.run.smithi190.stdout: "queue_len": 0, 2024-05-09T06:37:24.699 INFO:teuthology.orchestra.run.smithi190.stdout: "complete_latency": { 2024-05-09T06:37:24.699 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-05-09T06:37:24.700 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-05-09T06:37:24.700 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-05-09T06:37:24.700 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-05-09T06:37:24.700 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-05-09T06:37:24.700 INFO:teuthology.orchestra.run.smithi190.stdout: "finisher-telemetry": { 2024-05-09T06:37:24.700 INFO:teuthology.orchestra.run.smithi190.stdout: "queue_len": 0, 2024-05-09T06:37:24.700 INFO:teuthology.orchestra.run.smithi190.stdout: "complete_latency": { 2024-05-09T06:37:24.700 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-05-09T06:37:24.700 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-05-09T06:37:24.700 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-05-09T06:37:24.700 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-05-09T06:37:24.700 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-05-09T06:37:24.700 INFO:teuthology.orchestra.run.smithi190.stdout: "finisher-volumes": { 2024-05-09T06:37:24.701 INFO:teuthology.orchestra.run.smithi190.stdout: "queue_len": 0, 2024-05-09T06:37:24.701 INFO:teuthology.orchestra.run.smithi190.stdout: "complete_latency": { 2024-05-09T06:37:24.701 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 9, 2024-05-09T06:37:24.701 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.147773992, 2024-05-09T06:37:24.701 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.016419332 2024-05-09T06:37:24.701 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-05-09T06:37:24.701 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-05-09T06:37:24.701 INFO:teuthology.orchestra.run.smithi190.stdout: "libcephsqlite_striper": { 2024-05-09T06:37:24.701 INFO:teuthology.orchestra.run.smithi190.stdout: "update_metadata": 8, 2024-05-09T06:37:24.701 INFO:teuthology.orchestra.run.smithi190.stdout: "update_allocated": 2, 2024-05-09T06:37:24.701 INFO:teuthology.orchestra.run.smithi190.stdout: "update_size": 6, 2024-05-09T06:37:24.701 INFO:teuthology.orchestra.run.smithi190.stdout: "update_version": 8, 2024-05-09T06:37:24.701 INFO:teuthology.orchestra.run.smithi190.stdout: "shrink": 0, 2024-05-09T06:37:24.701 INFO:teuthology.orchestra.run.smithi190.stdout: "shrink_bytes": 0, 2024-05-09T06:37:24.702 INFO:teuthology.orchestra.run.smithi190.stdout: "lock": 11, 2024-05-09T06:37:24.702 INFO:teuthology.orchestra.run.smithi190.stdout: "unlock": 11 2024-05-09T06:37:24.702 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-05-09T06:37:24.702 INFO:teuthology.orchestra.run.smithi190.stdout: "libcephsqlite_vfs": { 2024-05-09T06:37:24.702 INFO:teuthology.orchestra.run.smithi190.stdout: "op_open": { 2024-05-09T06:37:24.702 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 2, 2024-05-09T06:37:24.702 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.069997970, 2024-05-09T06:37:24.702 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.034998985 2024-05-09T06:37:24.702 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-05-09T06:37:24.702 INFO:teuthology.orchestra.run.smithi190.stdout: "op_delete": { 2024-05-09T06:37:24.702 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-05-09T06:37:24.702 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-05-09T06:37:24.702 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-05-09T06:37:24.703 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-05-09T06:37:24.703 INFO:teuthology.orchestra.run.smithi190.stdout: "op_access": { 2024-05-09T06:37:24.703 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-05-09T06:37:24.703 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-05-09T06:37:24.703 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-05-09T06:37:24.703 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-05-09T06:37:24.703 INFO:teuthology.orchestra.run.smithi190.stdout: "op_fullpathname": { 2024-05-09T06:37:24.703 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 1, 2024-05-09T06:37:24.703 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000999971, 2024-05-09T06:37:24.703 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000999971 2024-05-09T06:37:24.703 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-05-09T06:37:24.703 INFO:teuthology.orchestra.run.smithi190.stdout: "op_currenttime": { 2024-05-09T06:37:24.703 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 4, 2024-05-09T06:37:24.703 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-05-09T06:37:24.703 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-05-09T06:37:24.704 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-05-09T06:37:24.704 INFO:teuthology.orchestra.run.smithi190.stdout: "opf_close": { 2024-05-09T06:37:24.704 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-05-09T06:37:24.704 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-05-09T06:37:24.704 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-05-09T06:37:24.704 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-05-09T06:37:24.704 INFO:teuthology.orchestra.run.smithi190.stdout: "opf_read": { 2024-05-09T06:37:24.704 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 20, 2024-05-09T06:37:24.704 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.006999797, 2024-05-09T06:37:24.704 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000349989 2024-05-09T06:37:24.704 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-05-09T06:37:24.705 INFO:teuthology.orchestra.run.smithi190.stdout: "opf_write": { 2024-05-09T06:37:24.705 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 85, 2024-05-09T06:37:24.705 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.004999856, 2024-05-09T06:37:24.705 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000058821 2024-05-09T06:37:24.705 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-05-09T06:37:24.705 INFO:teuthology.orchestra.run.smithi190.stdout: "opf_truncate": { 2024-05-09T06:37:24.705 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-05-09T06:37:24.705 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-05-09T06:37:24.705 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-05-09T06:37:24.705 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-05-09T06:37:24.705 INFO:teuthology.orchestra.run.smithi190.stdout: "opf_sync": { 2024-05-09T06:37:24.705 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 24, 2024-05-09T06:37:24.706 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.080997652, 2024-05-09T06:37:24.706 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.003374902 2024-05-09T06:37:24.706 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-05-09T06:37:24.706 INFO:teuthology.orchestra.run.smithi190.stdout: "opf_filesize": { 2024-05-09T06:37:24.706 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 20, 2024-05-09T06:37:24.706 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-05-09T06:37:24.706 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-05-09T06:37:24.706 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-05-09T06:37:24.706 INFO:teuthology.orchestra.run.smithi190.stdout: "opf_lock": { 2024-05-09T06:37:24.706 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 27, 2024-05-09T06:37:24.706 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.019999419, 2024-05-09T06:37:24.706 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000740719 2024-05-09T06:37:24.706 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-05-09T06:37:24.706 INFO:teuthology.orchestra.run.smithi190.stdout: "opf_unlock": { 2024-05-09T06:37:24.707 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 19, 2024-05-09T06:37:24.707 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.013999594, 2024-05-09T06:37:24.707 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000736820 2024-05-09T06:37:24.707 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-05-09T06:37:24.707 INFO:teuthology.orchestra.run.smithi190.stdout: "opf_checkreservedlock": { 2024-05-09T06:37:24.707 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 9, 2024-05-09T06:37:24.707 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-05-09T06:37:24.707 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-05-09T06:37:24.707 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-05-09T06:37:24.707 INFO:teuthology.orchestra.run.smithi190.stdout: "opf_filecontrol": { 2024-05-09T06:37:24.707 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 26, 2024-05-09T06:37:24.707 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-05-09T06:37:24.708 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-05-09T06:37:24.708 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-05-09T06:37:24.708 INFO:teuthology.orchestra.run.smithi190.stdout: "opf_sectorsize": { 2024-05-09T06:37:24.708 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-05-09T06:37:24.708 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-05-09T06:37:24.708 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-05-09T06:37:24.708 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-05-09T06:37:24.708 INFO:teuthology.orchestra.run.smithi190.stdout: "opf_devicecharacteristics": { 2024-05-09T06:37:24.708 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 30, 2024-05-09T06:37:24.708 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-05-09T06:37:24.708 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-05-09T06:37:24.708 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-05-09T06:37:24.708 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-05-09T06:37:24.708 INFO:teuthology.orchestra.run.smithi190.stdout: "mempool": { 2024-05-09T06:37:24.709 INFO:teuthology.orchestra.run.smithi190.stdout: "bloom_filter_bytes": 0, 2024-05-09T06:37:24.709 INFO:teuthology.orchestra.run.smithi190.stdout: "bloom_filter_items": 0, 2024-05-09T06:37:24.709 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_alloc_bytes": 0, 2024-05-09T06:37:24.709 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_alloc_items": 0, 2024-05-09T06:37:24.709 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_data_bytes": 0, 2024-05-09T06:37:24.709 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_data_items": 0, 2024-05-09T06:37:24.709 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-09T06:37:24.709 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_onode_items": 0, 2024-05-09T06:37:24.709 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-09T06:37:24.709 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_meta_items": 0, 2024-05-09T06:37:24.709 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_other_bytes": 0, 2024-05-09T06:37:24.709 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_other_items": 0, 2024-05-09T06:37:24.709 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-09T06:37:24.709 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_buffer_items": 0, 2024-05-09T06:37:24.710 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_extent_bytes": 0, 2024-05-09T06:37:24.710 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_extent_items": 0, 2024-05-09T06:37:24.710 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_blob_bytes": 0, 2024-05-09T06:37:24.710 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_blob_items": 0, 2024-05-09T06:37:24.710 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-09T06:37:24.710 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_shared_blob_items": 0, 2024-05-09T06:37:24.710 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-09T06:37:24.710 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_inline_bl_items": 0, 2024-05-09T06:37:24.710 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_fsck_bytes": 0, 2024-05-09T06:37:24.710 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_fsck_items": 0, 2024-05-09T06:37:24.710 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_txc_bytes": 0, 2024-05-09T06:37:24.710 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_txc_items": 0, 2024-05-09T06:37:24.711 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-09T06:37:24.711 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_writing_deferred_items": 0, 2024-05-09T06:37:24.711 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_writing_bytes": 0, 2024-05-09T06:37:24.711 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_writing_items": 0, 2024-05-09T06:37:24.711 INFO:teuthology.orchestra.run.smithi190.stdout: "bluefs_bytes": 0, 2024-05-09T06:37:24.711 INFO:teuthology.orchestra.run.smithi190.stdout: "bluefs_items": 0, 2024-05-09T06:37:24.711 INFO:teuthology.orchestra.run.smithi190.stdout: "bluefs_file_reader_bytes": 0, 2024-05-09T06:37:24.711 INFO:teuthology.orchestra.run.smithi190.stdout: "bluefs_file_reader_items": 0, 2024-05-09T06:37:24.711 INFO:teuthology.orchestra.run.smithi190.stdout: "bluefs_file_writer_bytes": 0, 2024-05-09T06:37:24.711 INFO:teuthology.orchestra.run.smithi190.stdout: "bluefs_file_writer_items": 0, 2024-05-09T06:37:24.711 INFO:teuthology.orchestra.run.smithi190.stdout: "buffer_anon_bytes": 1648901, 2024-05-09T06:37:24.711 INFO:teuthology.orchestra.run.smithi190.stdout: "buffer_anon_items": 325, 2024-05-09T06:37:24.711 INFO:teuthology.orchestra.run.smithi190.stdout: "buffer_meta_bytes": 1496, 2024-05-09T06:37:24.711 INFO:teuthology.orchestra.run.smithi190.stdout: "buffer_meta_items": 17, 2024-05-09T06:37:24.711 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_bytes": 0, 2024-05-09T06:37:24.712 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_items": 0, 2024-05-09T06:37:24.712 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_mapbl_bytes": 0, 2024-05-09T06:37:24.712 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_mapbl_items": 0, 2024-05-09T06:37:24.712 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_pglog_bytes": 0, 2024-05-09T06:37:24.712 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_pglog_items": 0, 2024-05-09T06:37:24.712 INFO:teuthology.orchestra.run.smithi190.stdout: "osdmap_bytes": 48360, 2024-05-09T06:37:24.712 INFO:teuthology.orchestra.run.smithi190.stdout: "osdmap_items": 804, 2024-05-09T06:37:24.712 INFO:teuthology.orchestra.run.smithi190.stdout: "osdmap_mapping_bytes": 0, 2024-05-09T06:37:24.712 INFO:teuthology.orchestra.run.smithi190.stdout: "osdmap_mapping_items": 0, 2024-05-09T06:37:24.712 INFO:teuthology.orchestra.run.smithi190.stdout: "pgmap_bytes": 12976, 2024-05-09T06:37:24.712 INFO:teuthology.orchestra.run.smithi190.stdout: "pgmap_items": 309, 2024-05-09T06:37:24.712 INFO:teuthology.orchestra.run.smithi190.stdout: "mds_co_bytes": 0, 2024-05-09T06:37:24.712 INFO:teuthology.orchestra.run.smithi190.stdout: "mds_co_items": 0, 2024-05-09T06:37:24.712 INFO:teuthology.orchestra.run.smithi190.stdout: "unittest_1_bytes": 0, 2024-05-09T06:37:24.713 INFO:teuthology.orchestra.run.smithi190.stdout: "unittest_1_items": 0, 2024-05-09T06:37:24.713 INFO:teuthology.orchestra.run.smithi190.stdout: "unittest_2_bytes": 0, 2024-05-09T06:37:24.713 INFO:teuthology.orchestra.run.smithi190.stdout: "unittest_2_items": 0 2024-05-09T06:37:24.713 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-05-09T06:37:24.713 INFO:teuthology.orchestra.run.smithi190.stdout: "mgr": { 2024-05-09T06:37:24.713 INFO:teuthology.orchestra.run.smithi190.stdout: "cache_hit": 0, 2024-05-09T06:37:24.713 INFO:teuthology.orchestra.run.smithi190.stdout: "cache_miss": 0 2024-05-09T06:37:24.713 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-05-09T06:37:24.713 INFO:teuthology.orchestra.run.smithi190.stdout: "objectcacher-libcephfs": { 2024-05-09T06:37:24.713 INFO:teuthology.orchestra.run.smithi190.stdout: "cache_ops_hit": 0, 2024-05-09T06:37:24.713 INFO:teuthology.orchestra.run.smithi190.stdout: "cache_ops_miss": 0, 2024-05-09T06:37:24.713 INFO:teuthology.orchestra.run.smithi190.stdout: "cache_bytes_hit": 0, 2024-05-09T06:37:24.713 INFO:teuthology.orchestra.run.smithi190.stdout: "cache_bytes_miss": 0, 2024-05-09T06:37:24.713 INFO:teuthology.orchestra.run.smithi190.stdout: "data_read": 0, 2024-05-09T06:37:24.714 INFO:teuthology.orchestra.run.smithi190.stdout: "data_written": 0, 2024-05-09T06:37:24.714 INFO:teuthology.orchestra.run.smithi190.stdout: "data_flushed": 0, 2024-05-09T06:37:24.714 INFO:teuthology.orchestra.run.smithi190.stdout: "data_overwritten_while_flushing": 0, 2024-05-09T06:37:24.714 INFO:teuthology.orchestra.run.smithi190.stdout: "write_ops_blocked": 0, 2024-05-09T06:37:24.714 INFO:teuthology.orchestra.run.smithi190.stdout: "write_bytes_blocked": 0, 2024-05-09T06:37:24.714 INFO:teuthology.orchestra.run.smithi190.stdout: "write_time_blocked": 0.000000000 2024-05-09T06:37:24.714 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-05-09T06:37:24.714 INFO:teuthology.orchestra.run.smithi190.stdout: "objectcacher-libcephfs-0x55b09e94bcc0": { 2024-05-09T06:37:24.714 INFO:teuthology.orchestra.run.smithi190.stdout: "cache_ops_hit": 0, 2024-05-09T06:37:24.714 INFO:teuthology.orchestra.run.smithi190.stdout: "cache_ops_miss": 0, 2024-05-09T06:37:24.714 INFO:teuthology.orchestra.run.smithi190.stdout: "cache_bytes_hit": 0, 2024-05-09T06:37:24.714 INFO:teuthology.orchestra.run.smithi190.stdout: "cache_bytes_miss": 0, 2024-05-09T06:37:24.714 INFO:teuthology.orchestra.run.smithi190.stdout: "data_read": 0, 2024-05-09T06:37:24.714 INFO:teuthology.orchestra.run.smithi190.stdout: "data_written": 0, 2024-05-09T06:37:24.715 INFO:teuthology.orchestra.run.smithi190.stdout: "data_flushed": 0, 2024-05-09T06:37:24.715 INFO:teuthology.orchestra.run.smithi190.stdout: "data_overwritten_while_flushing": 0, 2024-05-09T06:37:24.715 INFO:teuthology.orchestra.run.smithi190.stdout: "write_ops_blocked": 0, 2024-05-09T06:37:24.715 INFO:teuthology.orchestra.run.smithi190.stdout: "write_bytes_blocked": 0, 2024-05-09T06:37:24.715 INFO:teuthology.orchestra.run.smithi190.stdout: "write_time_blocked": 0.000000000 2024-05-09T06:37:24.715 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-05-09T06:37:24.715 INFO:teuthology.orchestra.run.smithi190.stdout: "objectcacher-libcephfs-0x55b09f6c2960": { 2024-05-09T06:37:24.715 INFO:teuthology.orchestra.run.smithi190.stdout: "cache_ops_hit": 9, 2024-05-09T06:37:24.715 INFO:teuthology.orchestra.run.smithi190.stdout: "cache_ops_miss": 0, 2024-05-09T06:37:24.715 INFO:teuthology.orchestra.run.smithi190.stdout: "cache_bytes_hit": 1230, 2024-05-09T06:37:24.715 INFO:teuthology.orchestra.run.smithi190.stdout: "cache_bytes_miss": 0, 2024-05-09T06:37:24.715 INFO:teuthology.orchestra.run.smithi190.stdout: "data_read": 1230, 2024-05-09T06:37:24.715 INFO:teuthology.orchestra.run.smithi190.stdout: "data_written": 410, 2024-05-09T06:37:24.716 INFO:teuthology.orchestra.run.smithi190.stdout: "data_flushed": 410, 2024-05-09T06:37:24.716 INFO:teuthology.orchestra.run.smithi190.stdout: "data_overwritten_while_flushing": 0, 2024-05-09T06:37:24.716 INFO:teuthology.orchestra.run.smithi190.stdout: "write_ops_blocked": 0, 2024-05-09T06:37:24.716 INFO:teuthology.orchestra.run.smithi190.stdout: "write_bytes_blocked": 0, 2024-05-09T06:37:24.716 INFO:teuthology.orchestra.run.smithi190.stdout: "write_time_blocked": 0.000000000 2024-05-09T06:37:24.716 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-05-09T06:37:24.716 INFO:teuthology.orchestra.run.smithi190.stdout: "objectcacher-libcephfs-0x55b09f6c2e60": { 2024-05-09T06:37:24.716 INFO:teuthology.orchestra.run.smithi190.stdout: "cache_ops_hit": 0, 2024-05-09T06:37:24.716 INFO:teuthology.orchestra.run.smithi190.stdout: "cache_ops_miss": 0, 2024-05-09T06:37:24.716 INFO:teuthology.orchestra.run.smithi190.stdout: "cache_bytes_hit": 0, 2024-05-09T06:37:24.716 INFO:teuthology.orchestra.run.smithi190.stdout: "cache_bytes_miss": 0, 2024-05-09T06:37:24.716 INFO:teuthology.orchestra.run.smithi190.stdout: "data_read": 0, 2024-05-09T06:37:24.716 INFO:teuthology.orchestra.run.smithi190.stdout: "data_written": 0, 2024-05-09T06:37:24.717 INFO:teuthology.orchestra.run.smithi190.stdout: "data_flushed": 0, 2024-05-09T06:37:24.717 INFO:teuthology.orchestra.run.smithi190.stdout: "data_overwritten_while_flushing": 0, 2024-05-09T06:37:24.717 INFO:teuthology.orchestra.run.smithi190.stdout: "write_ops_blocked": 0, 2024-05-09T06:37:24.717 INFO:teuthology.orchestra.run.smithi190.stdout: "write_bytes_blocked": 0, 2024-05-09T06:37:24.717 INFO:teuthology.orchestra.run.smithi190.stdout: "write_time_blocked": 0.000000000 2024-05-09T06:37:24.717 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-05-09T06:37:24.717 INFO:teuthology.orchestra.run.smithi190.stdout: "objecter": { 2024-05-09T06:37:24.717 INFO:teuthology.orchestra.run.smithi190.stdout: "op_active": 0, 2024-05-09T06:37:24.717 INFO:teuthology.orchestra.run.smithi190.stdout: "op_laggy": 0, 2024-05-09T06:37:24.718 INFO:teuthology.orchestra.run.smithi190.stdout: "op_send": 0, 2024-05-09T06:37:24.718 INFO:teuthology.orchestra.run.smithi190.stdout: "op_send_bytes": 0, 2024-05-09T06:37:24.718 INFO:teuthology.orchestra.run.smithi190.stdout: "op_resend": 0, 2024-05-09T06:37:24.718 INFO:teuthology.orchestra.run.smithi190.stdout: "op_reply": 0, 2024-05-09T06:37:24.718 INFO:teuthology.orchestra.run.smithi190.stdout: "op_latency": { 2024-05-09T06:37:24.718 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-05-09T06:37:24.718 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-05-09T06:37:24.718 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-05-09T06:37:24.718 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-05-09T06:37:24.718 INFO:teuthology.orchestra.run.smithi190.stdout: "op_inflight": 0, 2024-05-09T06:37:24.718 INFO:teuthology.orchestra.run.smithi190.stdout: "oplen_avg": { 2024-05-09T06:37:24.718 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-05-09T06:37:24.719 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0 2024-05-09T06:37:24.719 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-05-09T06:37:24.719 INFO:teuthology.orchestra.run.smithi190.stdout: "op": 0, 2024-05-09T06:37:24.719 INFO:teuthology.orchestra.run.smithi190.stdout: "op_r": 0, 2024-05-09T06:37:24.719 INFO:teuthology.orchestra.run.smithi190.stdout: "op_w": 0, 2024-05-09T06:37:24.719 INFO:teuthology.orchestra.run.smithi190.stdout: "op_rmw": 0, 2024-05-09T06:37:24.719 INFO:teuthology.orchestra.run.smithi190.stdout: "op_pg": 0, 2024-05-09T06:37:24.719 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_stat": 0, 2024-05-09T06:37:24.719 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_create": 0, 2024-05-09T06:37:24.719 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_read": 0, 2024-05-09T06:37:24.719 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_write": 0, 2024-05-09T06:37:24.720 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_writefull": 0, 2024-05-09T06:37:24.720 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_writesame": 0, 2024-05-09T06:37:24.720 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_append": 0, 2024-05-09T06:37:24.720 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_zero": 0, 2024-05-09T06:37:24.720 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_truncate": 0, 2024-05-09T06:37:24.720 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_delete": 0, 2024-05-09T06:37:24.720 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_mapext": 0, 2024-05-09T06:37:24.720 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_sparse_read": 0, 2024-05-09T06:37:24.720 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_clonerange": 0, 2024-05-09T06:37:24.720 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_getxattr": 0, 2024-05-09T06:37:24.720 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_setxattr": 0, 2024-05-09T06:37:24.720 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_cmpxattr": 0, 2024-05-09T06:37:24.720 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_rmxattr": 0, 2024-05-09T06:37:24.720 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_resetxattrs": 0, 2024-05-09T06:37:24.720 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_call": 0, 2024-05-09T06:37:24.721 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_watch": 0, 2024-05-09T06:37:24.721 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_notify": 0, 2024-05-09T06:37:24.721 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_src_cmpxattr": 0, 2024-05-09T06:37:24.721 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_pgls": 0, 2024-05-09T06:37:24.721 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_pgls_filter": 0, 2024-05-09T06:37:24.721 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_other": 0, 2024-05-09T06:37:24.721 INFO:teuthology.orchestra.run.smithi190.stdout: "linger_active": 0, 2024-05-09T06:37:24.721 INFO:teuthology.orchestra.run.smithi190.stdout: "linger_send": 0, 2024-05-09T06:37:24.721 INFO:teuthology.orchestra.run.smithi190.stdout: "linger_resend": 0, 2024-05-09T06:37:24.721 INFO:teuthology.orchestra.run.smithi190.stdout: "linger_ping": 0, 2024-05-09T06:37:24.721 INFO:teuthology.orchestra.run.smithi190.stdout: "poolop_active": 0, 2024-05-09T06:37:24.721 INFO:teuthology.orchestra.run.smithi190.stdout: "poolop_send": 0, 2024-05-09T06:37:24.721 INFO:teuthology.orchestra.run.smithi190.stdout: "poolop_resend": 0, 2024-05-09T06:37:24.721 INFO:teuthology.orchestra.run.smithi190.stdout: "poolstat_active": 0, 2024-05-09T06:37:24.722 INFO:teuthology.orchestra.run.smithi190.stdout: "poolstat_send": 0, 2024-05-09T06:37:24.722 INFO:teuthology.orchestra.run.smithi190.stdout: "poolstat_resend": 0, 2024-05-09T06:37:24.722 INFO:teuthology.orchestra.run.smithi190.stdout: "statfs_active": 0, 2024-05-09T06:37:24.722 INFO:teuthology.orchestra.run.smithi190.stdout: "statfs_send": 0, 2024-05-09T06:37:24.722 INFO:teuthology.orchestra.run.smithi190.stdout: "statfs_resend": 0, 2024-05-09T06:37:24.722 INFO:teuthology.orchestra.run.smithi190.stdout: "command_active": 0, 2024-05-09T06:37:24.722 INFO:teuthology.orchestra.run.smithi190.stdout: "command_send": 8, 2024-05-09T06:37:24.722 INFO:teuthology.orchestra.run.smithi190.stdout: "command_resend": 0, 2024-05-09T06:37:24.722 INFO:teuthology.orchestra.run.smithi190.stdout: "map_epoch": 36, 2024-05-09T06:37:24.722 INFO:teuthology.orchestra.run.smithi190.stdout: "map_full": 0, 2024-05-09T06:37:24.722 INFO:teuthology.orchestra.run.smithi190.stdout: "map_inc": 35, 2024-05-09T06:37:24.722 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_sessions": 8, 2024-05-09T06:37:24.722 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_session_open": 8, 2024-05-09T06:37:24.723 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_session_close": 0, 2024-05-09T06:37:24.723 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_laggy": 0, 2024-05-09T06:37:24.723 INFO:teuthology.orchestra.run.smithi190.stdout: "omap_wr": 0, 2024-05-09T06:37:24.723 INFO:teuthology.orchestra.run.smithi190.stdout: "omap_rd": 0, 2024-05-09T06:37:24.723 INFO:teuthology.orchestra.run.smithi190.stdout: "omap_del": 0 2024-05-09T06:37:24.723 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-05-09T06:37:24.723 INFO:teuthology.orchestra.run.smithi190.stdout: "objecter-0x55b09e6fed20": { 2024-05-09T06:37:24.723 INFO:teuthology.orchestra.run.smithi190.stdout: "op_active": 0, 2024-05-09T06:37:24.723 INFO:teuthology.orchestra.run.smithi190.stdout: "op_laggy": 0, 2024-05-09T06:37:24.723 INFO:teuthology.orchestra.run.smithi190.stdout: "op_send": 163, 2024-05-09T06:37:24.723 INFO:teuthology.orchestra.run.smithi190.stdout: "op_send_bytes": 2434738, 2024-05-09T06:37:24.723 INFO:teuthology.orchestra.run.smithi190.stdout: "op_resend": 0, 2024-05-09T06:37:24.723 INFO:teuthology.orchestra.run.smithi190.stdout: "op_reply": 163, 2024-05-09T06:37:24.724 INFO:teuthology.orchestra.run.smithi190.stdout: "op_latency": { 2024-05-09T06:37:24.724 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 163, 2024-05-09T06:37:24.724 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.478986116, 2024-05-09T06:37:24.724 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.002938565 2024-05-09T06:37:24.724 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-05-09T06:37:24.724 INFO:teuthology.orchestra.run.smithi190.stdout: "op_inflight": 0, 2024-05-09T06:37:24.724 INFO:teuthology.orchestra.run.smithi190.stdout: "oplen_avg": { 2024-05-09T06:37:24.724 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 163, 2024-05-09T06:37:24.724 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 307 2024-05-09T06:37:24.724 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-05-09T06:37:24.724 INFO:teuthology.orchestra.run.smithi190.stdout: "op": 163, 2024-05-09T06:37:24.724 INFO:teuthology.orchestra.run.smithi190.stdout: "op_r": 46, 2024-05-09T06:37:24.724 INFO:teuthology.orchestra.run.smithi190.stdout: "op_w": 117, 2024-05-09T06:37:24.725 INFO:teuthology.orchestra.run.smithi190.stdout: "op_rmw": 0, 2024-05-09T06:37:24.725 INFO:teuthology.orchestra.run.smithi190.stdout: "op_pg": 0, 2024-05-09T06:37:24.725 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_stat": 0, 2024-05-09T06:37:24.725 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_create": 2, 2024-05-09T06:37:24.725 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_read": 20, 2024-05-09T06:37:24.725 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_write": 85, 2024-05-09T06:37:24.725 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_writefull": 0, 2024-05-09T06:37:24.725 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_writesame": 0, 2024-05-09T06:37:24.725 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_append": 0, 2024-05-09T06:37:24.725 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_zero": 0, 2024-05-09T06:37:24.725 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_truncate": 0, 2024-05-09T06:37:24.725 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_delete": 0, 2024-05-09T06:37:24.725 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_mapext": 0, 2024-05-09T06:37:24.725 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_sparse_read": 0, 2024-05-09T06:37:24.726 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_clonerange": 0, 2024-05-09T06:37:24.726 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_getxattr": 104, 2024-05-09T06:37:24.726 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_setxattr": 52, 2024-05-09T06:37:24.726 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_cmpxattr": 22, 2024-05-09T06:37:24.726 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_rmxattr": 0, 2024-05-09T06:37:24.726 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_resetxattrs": 0, 2024-05-09T06:37:24.726 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_call": 22, 2024-05-09T06:37:24.726 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_watch": 0, 2024-05-09T06:37:24.726 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_notify": 0, 2024-05-09T06:37:24.726 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_src_cmpxattr": 0, 2024-05-09T06:37:24.726 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_pgls": 0, 2024-05-09T06:37:24.726 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_pgls_filter": 0, 2024-05-09T06:37:24.726 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_other": 0, 2024-05-09T06:37:24.727 INFO:teuthology.orchestra.run.smithi190.stdout: "linger_active": 0, 2024-05-09T06:37:24.727 INFO:teuthology.orchestra.run.smithi190.stdout: "linger_send": 0, 2024-05-09T06:37:24.727 INFO:teuthology.orchestra.run.smithi190.stdout: "linger_resend": 0, 2024-05-09T06:37:24.727 INFO:teuthology.orchestra.run.smithi190.stdout: "linger_ping": 0, 2024-05-09T06:37:24.727 INFO:teuthology.orchestra.run.smithi190.stdout: "poolop_active": 0, 2024-05-09T06:37:24.727 INFO:teuthology.orchestra.run.smithi190.stdout: "poolop_send": 0, 2024-05-09T06:37:24.727 INFO:teuthology.orchestra.run.smithi190.stdout: "poolop_resend": 0, 2024-05-09T06:37:24.727 INFO:teuthology.orchestra.run.smithi190.stdout: "poolstat_active": 0, 2024-05-09T06:37:24.727 INFO:teuthology.orchestra.run.smithi190.stdout: "poolstat_send": 0, 2024-05-09T06:37:24.727 INFO:teuthology.orchestra.run.smithi190.stdout: "poolstat_resend": 0, 2024-05-09T06:37:24.727 INFO:teuthology.orchestra.run.smithi190.stdout: "statfs_active": 0, 2024-05-09T06:37:24.727 INFO:teuthology.orchestra.run.smithi190.stdout: "statfs_send": 0, 2024-05-09T06:37:24.727 INFO:teuthology.orchestra.run.smithi190.stdout: "statfs_resend": 0, 2024-05-09T06:37:24.727 INFO:teuthology.orchestra.run.smithi190.stdout: "command_active": 0, 2024-05-09T06:37:24.728 INFO:teuthology.orchestra.run.smithi190.stdout: "command_send": 0, 2024-05-09T06:37:24.728 INFO:teuthology.orchestra.run.smithi190.stdout: "command_resend": 0, 2024-05-09T06:37:24.728 INFO:teuthology.orchestra.run.smithi190.stdout: "map_epoch": 16, 2024-05-09T06:37:24.728 INFO:teuthology.orchestra.run.smithi190.stdout: "map_full": 0, 2024-05-09T06:37:24.728 INFO:teuthology.orchestra.run.smithi190.stdout: "map_inc": 6, 2024-05-09T06:37:24.728 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_sessions": 1, 2024-05-09T06:37:24.728 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_session_open": 1, 2024-05-09T06:37:24.728 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_session_close": 0, 2024-05-09T06:37:24.728 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_laggy": 0, 2024-05-09T06:37:24.728 INFO:teuthology.orchestra.run.smithi190.stdout: "omap_wr": 0, 2024-05-09T06:37:24.728 INFO:teuthology.orchestra.run.smithi190.stdout: "omap_rd": 0, 2024-05-09T06:37:24.728 INFO:teuthology.orchestra.run.smithi190.stdout: "omap_del": 0 2024-05-09T06:37:24.728 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-05-09T06:37:24.728 INFO:teuthology.orchestra.run.smithi190.stdout: "objecter-0x55b09e6ff720": { 2024-05-09T06:37:24.728 INFO:teuthology.orchestra.run.smithi190.stdout: "op_active": 0, 2024-05-09T06:37:24.729 INFO:teuthology.orchestra.run.smithi190.stdout: "op_laggy": 0, 2024-05-09T06:37:24.729 INFO:teuthology.orchestra.run.smithi190.stdout: "op_send": 0, 2024-05-09T06:37:24.729 INFO:teuthology.orchestra.run.smithi190.stdout: "op_send_bytes": 0, 2024-05-09T06:37:24.729 INFO:teuthology.orchestra.run.smithi190.stdout: "op_resend": 0, 2024-05-09T06:37:24.729 INFO:teuthology.orchestra.run.smithi190.stdout: "op_reply": 0, 2024-05-09T06:37:24.729 INFO:teuthology.orchestra.run.smithi190.stdout: "op_latency": { 2024-05-09T06:37:24.729 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-05-09T06:37:24.729 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-05-09T06:37:24.729 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-05-09T06:37:24.729 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-05-09T06:37:24.729 INFO:teuthology.orchestra.run.smithi190.stdout: "op_inflight": 0, 2024-05-09T06:37:24.729 INFO:teuthology.orchestra.run.smithi190.stdout: "oplen_avg": { 2024-05-09T06:37:24.729 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-05-09T06:37:24.729 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0 2024-05-09T06:37:24.729 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-05-09T06:37:24.730 INFO:teuthology.orchestra.run.smithi190.stdout: "op": 0, 2024-05-09T06:37:24.730 INFO:teuthology.orchestra.run.smithi190.stdout: "op_r": 0, 2024-05-09T06:37:24.730 INFO:teuthology.orchestra.run.smithi190.stdout: "op_w": 0, 2024-05-09T06:37:24.730 INFO:teuthology.orchestra.run.smithi190.stdout: "op_rmw": 0, 2024-05-09T06:37:24.730 INFO:teuthology.orchestra.run.smithi190.stdout: "op_pg": 0, 2024-05-09T06:37:24.730 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_stat": 0, 2024-05-09T06:37:24.730 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_create": 0, 2024-05-09T06:37:24.730 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_read": 0, 2024-05-09T06:37:24.730 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_write": 0, 2024-05-09T06:37:24.730 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_writefull": 0, 2024-05-09T06:37:24.730 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_writesame": 0, 2024-05-09T06:37:24.730 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_append": 0, 2024-05-09T06:37:24.730 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_zero": 0, 2024-05-09T06:37:24.730 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_truncate": 0, 2024-05-09T06:37:24.730 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_delete": 0, 2024-05-09T06:37:24.731 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_mapext": 0, 2024-05-09T06:37:24.731 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_sparse_read": 0, 2024-05-09T06:37:24.731 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_clonerange": 0, 2024-05-09T06:37:24.731 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_getxattr": 0, 2024-05-09T06:37:24.731 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_setxattr": 0, 2024-05-09T06:37:24.731 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_cmpxattr": 0, 2024-05-09T06:37:24.731 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_rmxattr": 0, 2024-05-09T06:37:24.731 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_resetxattrs": 0, 2024-05-09T06:37:24.731 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_call": 0, 2024-05-09T06:37:24.731 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_watch": 0, 2024-05-09T06:37:24.731 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_notify": 0, 2024-05-09T06:37:24.731 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_src_cmpxattr": 0, 2024-05-09T06:37:24.731 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_pgls": 0, 2024-05-09T06:37:24.731 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_pgls_filter": 0, 2024-05-09T06:37:24.731 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_other": 0, 2024-05-09T06:37:24.731 INFO:teuthology.orchestra.run.smithi190.stdout: "linger_active": 0, 2024-05-09T06:37:24.732 INFO:teuthology.orchestra.run.smithi190.stdout: "linger_send": 0, 2024-05-09T06:37:24.732 INFO:teuthology.orchestra.run.smithi190.stdout: "linger_resend": 0, 2024-05-09T06:37:24.732 INFO:teuthology.orchestra.run.smithi190.stdout: "linger_ping": 0, 2024-05-09T06:37:24.732 INFO:teuthology.orchestra.run.smithi190.stdout: "poolop_active": 0, 2024-05-09T06:37:24.732 INFO:teuthology.orchestra.run.smithi190.stdout: "poolop_send": 0, 2024-05-09T06:37:24.732 INFO:teuthology.orchestra.run.smithi190.stdout: "poolop_resend": 0, 2024-05-09T06:37:24.732 INFO:teuthology.orchestra.run.smithi190.stdout: "poolstat_active": 0, 2024-05-09T06:37:24.732 INFO:teuthology.orchestra.run.smithi190.stdout: "poolstat_send": 0, 2024-05-09T06:37:24.732 INFO:teuthology.orchestra.run.smithi190.stdout: "poolstat_resend": 0, 2024-05-09T06:37:24.732 INFO:teuthology.orchestra.run.smithi190.stdout: "statfs_active": 0, 2024-05-09T06:37:24.732 INFO:teuthology.orchestra.run.smithi190.stdout: "statfs_send": 0, 2024-05-09T06:37:24.732 INFO:teuthology.orchestra.run.smithi190.stdout: "statfs_resend": 0, 2024-05-09T06:37:24.732 INFO:teuthology.orchestra.run.smithi190.stdout: "command_active": 0, 2024-05-09T06:37:24.732 INFO:teuthology.orchestra.run.smithi190.stdout: "command_send": 0, 2024-05-09T06:37:24.733 INFO:teuthology.orchestra.run.smithi190.stdout: "command_resend": 0, 2024-05-09T06:37:24.733 INFO:teuthology.orchestra.run.smithi190.stdout: "map_epoch": 0, 2024-05-09T06:37:24.733 INFO:teuthology.orchestra.run.smithi190.stdout: "map_full": 0, 2024-05-09T06:37:24.733 INFO:teuthology.orchestra.run.smithi190.stdout: "map_inc": 0, 2024-05-09T06:37:24.733 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_sessions": 0, 2024-05-09T06:37:24.733 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_session_open": 0, 2024-05-09T06:37:24.733 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_session_close": 0, 2024-05-09T06:37:24.733 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_laggy": 0, 2024-05-09T06:37:24.733 INFO:teuthology.orchestra.run.smithi190.stdout: "omap_wr": 0, 2024-05-09T06:37:24.733 INFO:teuthology.orchestra.run.smithi190.stdout: "omap_rd": 0, 2024-05-09T06:37:24.733 INFO:teuthology.orchestra.run.smithi190.stdout: "omap_del": 0 2024-05-09T06:37:24.733 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-05-09T06:37:24.733 INFO:teuthology.orchestra.run.smithi190.stdout: "objecter-0x55b09e6ff9a0": { 2024-05-09T06:37:24.733 INFO:teuthology.orchestra.run.smithi190.stdout: "op_active": 0, 2024-05-09T06:37:24.733 INFO:teuthology.orchestra.run.smithi190.stdout: "op_laggy": 0, 2024-05-09T06:37:24.734 INFO:teuthology.orchestra.run.smithi190.stdout: "op_send": 0, 2024-05-09T06:37:24.734 INFO:teuthology.orchestra.run.smithi190.stdout: "op_send_bytes": 0, 2024-05-09T06:37:24.734 INFO:teuthology.orchestra.run.smithi190.stdout: "op_resend": 0, 2024-05-09T06:37:24.734 INFO:teuthology.orchestra.run.smithi190.stdout: "op_reply": 0, 2024-05-09T06:37:24.734 INFO:teuthology.orchestra.run.smithi190.stdout: "op_latency": { 2024-05-09T06:37:24.734 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-05-09T06:37:24.734 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-05-09T06:37:24.734 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-05-09T06:37:24.734 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-05-09T06:37:24.734 INFO:teuthology.orchestra.run.smithi190.stdout: "op_inflight": 0, 2024-05-09T06:37:24.734 INFO:teuthology.orchestra.run.smithi190.stdout: "oplen_avg": { 2024-05-09T06:37:24.734 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-05-09T06:37:24.734 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0 2024-05-09T06:37:24.735 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-05-09T06:37:24.735 INFO:teuthology.orchestra.run.smithi190.stdout: "op": 0, 2024-05-09T06:37:24.735 INFO:teuthology.orchestra.run.smithi190.stdout: "op_r": 0, 2024-05-09T06:37:24.735 INFO:teuthology.orchestra.run.smithi190.stdout: "op_w": 0, 2024-05-09T06:37:24.735 INFO:teuthology.orchestra.run.smithi190.stdout: "op_rmw": 0, 2024-05-09T06:37:24.735 INFO:teuthology.orchestra.run.smithi190.stdout: "op_pg": 0, 2024-05-09T06:37:24.735 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_stat": 0, 2024-05-09T06:37:24.735 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_create": 0, 2024-05-09T06:37:24.735 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_read": 0, 2024-05-09T06:37:24.735 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_write": 0, 2024-05-09T06:37:24.735 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_writefull": 0, 2024-05-09T06:37:24.735 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_writesame": 0, 2024-05-09T06:37:24.735 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_append": 0, 2024-05-09T06:37:24.735 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_zero": 0, 2024-05-09T06:37:24.735 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_truncate": 0, 2024-05-09T06:37:24.736 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_delete": 0, 2024-05-09T06:37:24.736 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_mapext": 0, 2024-05-09T06:37:24.736 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_sparse_read": 0, 2024-05-09T06:37:24.736 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_clonerange": 0, 2024-05-09T06:37:24.736 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_getxattr": 0, 2024-05-09T06:37:24.736 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_setxattr": 0, 2024-05-09T06:37:24.736 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_cmpxattr": 0, 2024-05-09T06:37:24.736 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_rmxattr": 0, 2024-05-09T06:37:24.736 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_resetxattrs": 0, 2024-05-09T06:37:24.736 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_call": 0, 2024-05-09T06:37:24.736 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_watch": 0, 2024-05-09T06:37:24.736 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_notify": 0, 2024-05-09T06:37:24.736 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_src_cmpxattr": 0, 2024-05-09T06:37:24.736 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_pgls": 0, 2024-05-09T06:37:24.736 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_pgls_filter": 0, 2024-05-09T06:37:24.737 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_other": 0, 2024-05-09T06:37:24.737 INFO:teuthology.orchestra.run.smithi190.stdout: "linger_active": 0, 2024-05-09T06:37:24.737 INFO:teuthology.orchestra.run.smithi190.stdout: "linger_send": 0, 2024-05-09T06:37:24.737 INFO:teuthology.orchestra.run.smithi190.stdout: "linger_resend": 0, 2024-05-09T06:37:24.737 INFO:teuthology.orchestra.run.smithi190.stdout: "linger_ping": 0, 2024-05-09T06:37:24.737 INFO:teuthology.orchestra.run.smithi190.stdout: "poolop_active": 0, 2024-05-09T06:37:24.737 INFO:teuthology.orchestra.run.smithi190.stdout: "poolop_send": 0, 2024-05-09T06:37:24.737 INFO:teuthology.orchestra.run.smithi190.stdout: "poolop_resend": 0, 2024-05-09T06:37:24.737 INFO:teuthology.orchestra.run.smithi190.stdout: "poolstat_active": 0, 2024-05-09T06:37:24.737 INFO:teuthology.orchestra.run.smithi190.stdout: "poolstat_send": 0, 2024-05-09T06:37:24.737 INFO:teuthology.orchestra.run.smithi190.stdout: "poolstat_resend": 0, 2024-05-09T06:37:24.737 INFO:teuthology.orchestra.run.smithi190.stdout: "statfs_active": 0, 2024-05-09T06:37:24.737 INFO:teuthology.orchestra.run.smithi190.stdout: "statfs_send": 0, 2024-05-09T06:37:24.737 INFO:teuthology.orchestra.run.smithi190.stdout: "statfs_resend": 0, 2024-05-09T06:37:24.737 INFO:teuthology.orchestra.run.smithi190.stdout: "command_active": 0, 2024-05-09T06:37:24.738 INFO:teuthology.orchestra.run.smithi190.stdout: "command_send": 0, 2024-05-09T06:37:24.738 INFO:teuthology.orchestra.run.smithi190.stdout: "command_resend": 0, 2024-05-09T06:37:24.738 INFO:teuthology.orchestra.run.smithi190.stdout: "map_epoch": 0, 2024-05-09T06:37:24.738 INFO:teuthology.orchestra.run.smithi190.stdout: "map_full": 0, 2024-05-09T06:37:24.738 INFO:teuthology.orchestra.run.smithi190.stdout: "map_inc": 0, 2024-05-09T06:37:24.738 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_sessions": 0, 2024-05-09T06:37:24.738 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_session_open": 0, 2024-05-09T06:37:24.738 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_session_close": 0, 2024-05-09T06:37:24.738 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_laggy": 0, 2024-05-09T06:37:24.738 INFO:teuthology.orchestra.run.smithi190.stdout: "omap_wr": 0, 2024-05-09T06:37:24.738 INFO:teuthology.orchestra.run.smithi190.stdout: "omap_rd": 0, 2024-05-09T06:37:24.738 INFO:teuthology.orchestra.run.smithi190.stdout: "omap_del": 0 2024-05-09T06:37:24.738 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-05-09T06:37:24.739 INFO:teuthology.orchestra.run.smithi190.stdout: "objecter-0x55b09e94a3c0": { 2024-05-09T06:37:24.739 INFO:teuthology.orchestra.run.smithi190.stdout: "op_active": 0, 2024-05-09T06:37:24.739 INFO:teuthology.orchestra.run.smithi190.stdout: "op_laggy": 0, 2024-05-09T06:37:24.739 INFO:teuthology.orchestra.run.smithi190.stdout: "op_send": 1, 2024-05-09T06:37:24.739 INFO:teuthology.orchestra.run.smithi190.stdout: "op_send_bytes": 39, 2024-05-09T06:37:24.739 INFO:teuthology.orchestra.run.smithi190.stdout: "op_resend": 0, 2024-05-09T06:37:24.739 INFO:teuthology.orchestra.run.smithi190.stdout: "op_reply": 1, 2024-05-09T06:37:24.739 INFO:teuthology.orchestra.run.smithi190.stdout: "op_latency": { 2024-05-09T06:37:24.739 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 1, 2024-05-09T06:37:24.739 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.001999943, 2024-05-09T06:37:24.739 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.001999943 2024-05-09T06:37:24.739 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-05-09T06:37:24.739 INFO:teuthology.orchestra.run.smithi190.stdout: "op_inflight": 0, 2024-05-09T06:37:24.739 INFO:teuthology.orchestra.run.smithi190.stdout: "oplen_avg": { 2024-05-09T06:37:24.739 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 1, 2024-05-09T06:37:24.740 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 1 2024-05-09T06:37:24.740 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-05-09T06:37:24.740 INFO:teuthology.orchestra.run.smithi190.stdout: "op": 1, 2024-05-09T06:37:24.740 INFO:teuthology.orchestra.run.smithi190.stdout: "op_r": 1, 2024-05-09T06:37:24.740 INFO:teuthology.orchestra.run.smithi190.stdout: "op_w": 0, 2024-05-09T06:37:24.740 INFO:teuthology.orchestra.run.smithi190.stdout: "op_rmw": 0, 2024-05-09T06:37:24.740 INFO:teuthology.orchestra.run.smithi190.stdout: "op_pg": 0, 2024-05-09T06:37:24.740 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_stat": 0, 2024-05-09T06:37:24.740 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_create": 0, 2024-05-09T06:37:24.740 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_read": 0, 2024-05-09T06:37:24.740 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_write": 0, 2024-05-09T06:37:24.740 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_writefull": 0, 2024-05-09T06:37:24.740 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_writesame": 0, 2024-05-09T06:37:24.740 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_append": 0, 2024-05-09T06:37:24.740 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_zero": 0, 2024-05-09T06:37:24.741 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_truncate": 0, 2024-05-09T06:37:24.741 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_delete": 0, 2024-05-09T06:37:24.741 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_mapext": 0, 2024-05-09T06:37:24.741 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_sparse_read": 0, 2024-05-09T06:37:24.741 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_clonerange": 0, 2024-05-09T06:37:24.741 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_getxattr": 0, 2024-05-09T06:37:24.741 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_setxattr": 0, 2024-05-09T06:37:24.741 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_cmpxattr": 0, 2024-05-09T06:37:24.741 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_rmxattr": 0, 2024-05-09T06:37:24.741 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_resetxattrs": 0, 2024-05-09T06:37:24.741 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_call": 0, 2024-05-09T06:37:24.741 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_watch": 0, 2024-05-09T06:37:24.741 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_notify": 0, 2024-05-09T06:37:24.741 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_src_cmpxattr": 0, 2024-05-09T06:37:24.741 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_pgls": 0, 2024-05-09T06:37:24.742 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_pgls_filter": 0, 2024-05-09T06:37:24.742 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_other": 1, 2024-05-09T06:37:24.742 INFO:teuthology.orchestra.run.smithi190.stdout: "linger_active": 0, 2024-05-09T06:37:24.742 INFO:teuthology.orchestra.run.smithi190.stdout: "linger_send": 0, 2024-05-09T06:37:24.742 INFO:teuthology.orchestra.run.smithi190.stdout: "linger_resend": 0, 2024-05-09T06:37:24.742 INFO:teuthology.orchestra.run.smithi190.stdout: "linger_ping": 0, 2024-05-09T06:37:24.742 INFO:teuthology.orchestra.run.smithi190.stdout: "poolop_active": 0, 2024-05-09T06:37:24.742 INFO:teuthology.orchestra.run.smithi190.stdout: "poolop_send": 0, 2024-05-09T06:37:24.742 INFO:teuthology.orchestra.run.smithi190.stdout: "poolop_resend": 0, 2024-05-09T06:37:24.742 INFO:teuthology.orchestra.run.smithi190.stdout: "poolstat_active": 0, 2024-05-09T06:37:24.742 INFO:teuthology.orchestra.run.smithi190.stdout: "poolstat_send": 0, 2024-05-09T06:37:24.742 INFO:teuthology.orchestra.run.smithi190.stdout: "poolstat_resend": 0, 2024-05-09T06:37:24.743 INFO:teuthology.orchestra.run.smithi190.stdout: "statfs_active": 0, 2024-05-09T06:37:24.743 INFO:teuthology.orchestra.run.smithi190.stdout: "statfs_send": 0, 2024-05-09T06:37:24.743 INFO:teuthology.orchestra.run.smithi190.stdout: "statfs_resend": 0, 2024-05-09T06:37:24.743 INFO:teuthology.orchestra.run.smithi190.stdout: "command_active": 0, 2024-05-09T06:37:24.743 INFO:teuthology.orchestra.run.smithi190.stdout: "command_send": 0, 2024-05-09T06:37:24.743 INFO:teuthology.orchestra.run.smithi190.stdout: "command_resend": 0, 2024-05-09T06:37:24.743 INFO:teuthology.orchestra.run.smithi190.stdout: "map_epoch": 0, 2024-05-09T06:37:24.743 INFO:teuthology.orchestra.run.smithi190.stdout: "map_full": 0, 2024-05-09T06:37:24.743 INFO:teuthology.orchestra.run.smithi190.stdout: "map_inc": 0, 2024-05-09T06:37:24.743 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_sessions": 1, 2024-05-09T06:37:24.743 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_session_open": 1, 2024-05-09T06:37:24.743 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_session_close": 0, 2024-05-09T06:37:24.743 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_laggy": 0, 2024-05-09T06:37:24.743 INFO:teuthology.orchestra.run.smithi190.stdout: "omap_wr": 0, 2024-05-09T06:37:24.743 INFO:teuthology.orchestra.run.smithi190.stdout: "omap_rd": 0, 2024-05-09T06:37:24.744 INFO:teuthology.orchestra.run.smithi190.stdout: "omap_del": 0 2024-05-09T06:37:24.744 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-05-09T06:37:24.744 INFO:teuthology.orchestra.run.smithi190.stdout: "objecter-0x55b09e94bd60": { 2024-05-09T06:37:24.744 INFO:teuthology.orchestra.run.smithi190.stdout: "op_active": 0, 2024-05-09T06:37:24.744 INFO:teuthology.orchestra.run.smithi190.stdout: "op_laggy": 0, 2024-05-09T06:37:24.744 INFO:teuthology.orchestra.run.smithi190.stdout: "op_send": 0, 2024-05-09T06:37:24.744 INFO:teuthology.orchestra.run.smithi190.stdout: "op_send_bytes": 0, 2024-05-09T06:37:24.744 INFO:teuthology.orchestra.run.smithi190.stdout: "op_resend": 0, 2024-05-09T06:37:24.744 INFO:teuthology.orchestra.run.smithi190.stdout: "op_reply": 0, 2024-05-09T06:37:24.744 INFO:teuthology.orchestra.run.smithi190.stdout: "op_latency": { 2024-05-09T06:37:24.744 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-05-09T06:37:24.744 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-05-09T06:37:24.744 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-05-09T06:37:24.744 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-05-09T06:37:24.744 INFO:teuthology.orchestra.run.smithi190.stdout: "op_inflight": 0, 2024-05-09T06:37:24.745 INFO:teuthology.orchestra.run.smithi190.stdout: "oplen_avg": { 2024-05-09T06:37:24.745 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-05-09T06:37:24.745 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0 2024-05-09T06:37:24.745 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-05-09T06:37:24.745 INFO:teuthology.orchestra.run.smithi190.stdout: "op": 0, 2024-05-09T06:37:24.745 INFO:teuthology.orchestra.run.smithi190.stdout: "op_r": 0, 2024-05-09T06:37:24.745 INFO:teuthology.orchestra.run.smithi190.stdout: "op_w": 0, 2024-05-09T06:37:24.745 INFO:teuthology.orchestra.run.smithi190.stdout: "op_rmw": 0, 2024-05-09T06:37:24.745 INFO:teuthology.orchestra.run.smithi190.stdout: "op_pg": 0, 2024-05-09T06:37:24.745 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_stat": 0, 2024-05-09T06:37:24.745 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_create": 0, 2024-05-09T06:37:24.745 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_read": 0, 2024-05-09T06:37:24.745 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_write": 0, 2024-05-09T06:37:24.745 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_writefull": 0, 2024-05-09T06:37:24.746 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_writesame": 0, 2024-05-09T06:37:24.746 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_append": 0, 2024-05-09T06:37:24.746 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_zero": 0, 2024-05-09T06:37:24.746 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_truncate": 0, 2024-05-09T06:37:24.746 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_delete": 0, 2024-05-09T06:37:24.746 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_mapext": 0, 2024-05-09T06:37:24.746 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_sparse_read": 0, 2024-05-09T06:37:24.746 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_clonerange": 0, 2024-05-09T06:37:24.746 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_getxattr": 0, 2024-05-09T06:37:24.746 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_setxattr": 0, 2024-05-09T06:37:24.746 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_cmpxattr": 0, 2024-05-09T06:37:24.746 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_rmxattr": 0, 2024-05-09T06:37:24.746 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_resetxattrs": 0, 2024-05-09T06:37:24.746 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_call": 0, 2024-05-09T06:37:24.746 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_watch": 0, 2024-05-09T06:37:24.747 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_notify": 0, 2024-05-09T06:37:24.747 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_src_cmpxattr": 0, 2024-05-09T06:37:24.747 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_pgls": 0, 2024-05-09T06:37:24.747 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_pgls_filter": 0, 2024-05-09T06:37:24.747 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_other": 0, 2024-05-09T06:37:24.747 INFO:teuthology.orchestra.run.smithi190.stdout: "linger_active": 0, 2024-05-09T06:37:24.747 INFO:teuthology.orchestra.run.smithi190.stdout: "linger_send": 0, 2024-05-09T06:37:24.747 INFO:teuthology.orchestra.run.smithi190.stdout: "linger_resend": 0, 2024-05-09T06:37:24.747 INFO:teuthology.orchestra.run.smithi190.stdout: "linger_ping": 0, 2024-05-09T06:37:24.747 INFO:teuthology.orchestra.run.smithi190.stdout: "poolop_active": 0, 2024-05-09T06:37:24.747 INFO:teuthology.orchestra.run.smithi190.stdout: "poolop_send": 0, 2024-05-09T06:37:24.747 INFO:teuthology.orchestra.run.smithi190.stdout: "poolop_resend": 0, 2024-05-09T06:37:24.747 INFO:teuthology.orchestra.run.smithi190.stdout: "poolstat_active": 0, 2024-05-09T06:37:24.747 INFO:teuthology.orchestra.run.smithi190.stdout: "poolstat_send": 0, 2024-05-09T06:37:24.748 INFO:teuthology.orchestra.run.smithi190.stdout: "poolstat_resend": 0, 2024-05-09T06:37:24.748 INFO:teuthology.orchestra.run.smithi190.stdout: "statfs_active": 0, 2024-05-09T06:37:24.748 INFO:teuthology.orchestra.run.smithi190.stdout: "statfs_send": 0, 2024-05-09T06:37:24.748 INFO:teuthology.orchestra.run.smithi190.stdout: "statfs_resend": 0, 2024-05-09T06:37:24.748 INFO:teuthology.orchestra.run.smithi190.stdout: "command_active": 0, 2024-05-09T06:37:24.748 INFO:teuthology.orchestra.run.smithi190.stdout: "command_send": 0, 2024-05-09T06:37:24.748 INFO:teuthology.orchestra.run.smithi190.stdout: "command_resend": 0, 2024-05-09T06:37:24.748 INFO:teuthology.orchestra.run.smithi190.stdout: "map_epoch": 0, 2024-05-09T06:37:24.748 INFO:teuthology.orchestra.run.smithi190.stdout: "map_full": 0, 2024-05-09T06:37:24.748 INFO:teuthology.orchestra.run.smithi190.stdout: "map_inc": 0, 2024-05-09T06:37:24.748 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_sessions": 0, 2024-05-09T06:37:24.748 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_session_open": 0, 2024-05-09T06:37:24.748 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_session_close": 0, 2024-05-09T06:37:24.749 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_laggy": 0, 2024-05-09T06:37:24.749 INFO:teuthology.orchestra.run.smithi190.stdout: "omap_wr": 0, 2024-05-09T06:37:24.749 INFO:teuthology.orchestra.run.smithi190.stdout: "omap_rd": 0, 2024-05-09T06:37:24.749 INFO:teuthology.orchestra.run.smithi190.stdout: "omap_del": 0 2024-05-09T06:37:24.749 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-05-09T06:37:24.749 INFO:teuthology.orchestra.run.smithi190.stdout: "objecter-0x55b09f6c2a00": { 2024-05-09T06:37:24.749 INFO:teuthology.orchestra.run.smithi190.stdout: "op_active": 0, 2024-05-09T06:37:24.749 INFO:teuthology.orchestra.run.smithi190.stdout: "op_laggy": 0, 2024-05-09T06:37:24.749 INFO:teuthology.orchestra.run.smithi190.stdout: "op_send": 3, 2024-05-09T06:37:24.749 INFO:teuthology.orchestra.run.smithi190.stdout: "op_send_bytes": 410, 2024-05-09T06:37:24.749 INFO:teuthology.orchestra.run.smithi190.stdout: "op_resend": 0, 2024-05-09T06:37:24.749 INFO:teuthology.orchestra.run.smithi190.stdout: "op_reply": 3, 2024-05-09T06:37:24.749 INFO:teuthology.orchestra.run.smithi190.stdout: "op_latency": { 2024-05-09T06:37:24.750 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 3, 2024-05-09T06:37:24.750 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.008999739, 2024-05-09T06:37:24.750 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.002999913 2024-05-09T06:37:24.750 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-05-09T06:37:24.750 INFO:teuthology.orchestra.run.smithi190.stdout: "op_inflight": 0, 2024-05-09T06:37:24.750 INFO:teuthology.orchestra.run.smithi190.stdout: "oplen_avg": { 2024-05-09T06:37:24.750 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 3, 2024-05-09T06:37:24.750 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 3 2024-05-09T06:37:24.750 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-05-09T06:37:24.750 INFO:teuthology.orchestra.run.smithi190.stdout: "op": 3, 2024-05-09T06:37:24.750 INFO:teuthology.orchestra.run.smithi190.stdout: "op_r": 0, 2024-05-09T06:37:24.750 INFO:teuthology.orchestra.run.smithi190.stdout: "op_w": 3, 2024-05-09T06:37:24.750 INFO:teuthology.orchestra.run.smithi190.stdout: "op_rmw": 0, 2024-05-09T06:37:24.751 INFO:teuthology.orchestra.run.smithi190.stdout: "op_pg": 0, 2024-05-09T06:37:24.751 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_stat": 0, 2024-05-09T06:37:24.751 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_create": 0, 2024-05-09T06:37:24.751 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_read": 0, 2024-05-09T06:37:24.751 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_write": 3, 2024-05-09T06:37:24.751 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_writefull": 0, 2024-05-09T06:37:24.751 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_writesame": 0, 2024-05-09T06:37:24.751 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_append": 0, 2024-05-09T06:37:24.751 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_zero": 0, 2024-05-09T06:37:24.751 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_truncate": 0, 2024-05-09T06:37:24.751 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_delete": 0, 2024-05-09T06:37:24.751 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_mapext": 0, 2024-05-09T06:37:24.751 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_sparse_read": 0, 2024-05-09T06:37:24.752 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_clonerange": 0, 2024-05-09T06:37:24.752 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_getxattr": 0, 2024-05-09T06:37:24.752 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_setxattr": 0, 2024-05-09T06:37:24.752 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_cmpxattr": 0, 2024-05-09T06:37:24.752 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_rmxattr": 0, 2024-05-09T06:37:24.752 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_resetxattrs": 0, 2024-05-09T06:37:24.752 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_call": 0, 2024-05-09T06:37:24.752 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_watch": 0, 2024-05-09T06:37:24.752 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_notify": 0, 2024-05-09T06:37:24.752 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_src_cmpxattr": 0, 2024-05-09T06:37:24.752 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_pgls": 0, 2024-05-09T06:37:24.752 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_pgls_filter": 0, 2024-05-09T06:37:24.752 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_other": 0, 2024-05-09T06:37:24.753 INFO:teuthology.orchestra.run.smithi190.stdout: "linger_active": 0, 2024-05-09T06:37:24.753 INFO:teuthology.orchestra.run.smithi190.stdout: "linger_send": 0, 2024-05-09T06:37:24.753 INFO:teuthology.orchestra.run.smithi190.stdout: "linger_resend": 0, 2024-05-09T06:37:24.753 INFO:teuthology.orchestra.run.smithi190.stdout: "linger_ping": 0, 2024-05-09T06:37:24.753 INFO:teuthology.orchestra.run.smithi190.stdout: "poolop_active": 0, 2024-05-09T06:37:24.753 INFO:teuthology.orchestra.run.smithi190.stdout: "poolop_send": 0, 2024-05-09T06:37:24.753 INFO:teuthology.orchestra.run.smithi190.stdout: "poolop_resend": 0, 2024-05-09T06:37:24.753 INFO:teuthology.orchestra.run.smithi190.stdout: "poolstat_active": 0, 2024-05-09T06:37:24.753 INFO:teuthology.orchestra.run.smithi190.stdout: "poolstat_send": 0, 2024-05-09T06:37:24.753 INFO:teuthology.orchestra.run.smithi190.stdout: "poolstat_resend": 0, 2024-05-09T06:37:24.753 INFO:teuthology.orchestra.run.smithi190.stdout: "statfs_active": 0, 2024-05-09T06:37:24.753 INFO:teuthology.orchestra.run.smithi190.stdout: "statfs_send": 0, 2024-05-09T06:37:24.753 INFO:teuthology.orchestra.run.smithi190.stdout: "statfs_resend": 0, 2024-05-09T06:37:24.754 INFO:teuthology.orchestra.run.smithi190.stdout: "command_active": 0, 2024-05-09T06:37:24.754 INFO:teuthology.orchestra.run.smithi190.stdout: "command_send": 0, 2024-05-09T06:37:24.754 INFO:teuthology.orchestra.run.smithi190.stdout: "command_resend": 0, 2024-05-09T06:37:24.754 INFO:teuthology.orchestra.run.smithi190.stdout: "map_epoch": 0, 2024-05-09T06:37:24.754 INFO:teuthology.orchestra.run.smithi190.stdout: "map_full": 0, 2024-05-09T06:37:24.754 INFO:teuthology.orchestra.run.smithi190.stdout: "map_inc": 0, 2024-05-09T06:37:24.754 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_sessions": 2, 2024-05-09T06:37:24.754 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_session_open": 2, 2024-05-09T06:37:24.754 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_session_close": 0, 2024-05-09T06:37:24.754 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_laggy": 0, 2024-05-09T06:37:24.754 INFO:teuthology.orchestra.run.smithi190.stdout: "omap_wr": 0, 2024-05-09T06:37:24.754 INFO:teuthology.orchestra.run.smithi190.stdout: "omap_rd": 0, 2024-05-09T06:37:24.754 INFO:teuthology.orchestra.run.smithi190.stdout: "omap_del": 0 2024-05-09T06:37:24.755 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-05-09T06:37:24.755 INFO:teuthology.orchestra.run.smithi190.stdout: "objecter-0x55b09f6c3040": { 2024-05-09T06:37:24.755 INFO:teuthology.orchestra.run.smithi190.stdout: "op_active": 0, 2024-05-09T06:37:24.755 INFO:teuthology.orchestra.run.smithi190.stdout: "op_laggy": 0, 2024-05-09T06:37:24.755 INFO:teuthology.orchestra.run.smithi190.stdout: "op_send": 0, 2024-05-09T06:37:24.755 INFO:teuthology.orchestra.run.smithi190.stdout: "op_send_bytes": 0, 2024-05-09T06:37:24.755 INFO:teuthology.orchestra.run.smithi190.stdout: "op_resend": 0, 2024-05-09T06:37:24.755 INFO:teuthology.orchestra.run.smithi190.stdout: "op_reply": 0, 2024-05-09T06:37:24.755 INFO:teuthology.orchestra.run.smithi190.stdout: "op_latency": { 2024-05-09T06:37:24.755 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-05-09T06:37:24.755 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-05-09T06:37:24.755 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-05-09T06:37:24.755 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-05-09T06:37:24.756 INFO:teuthology.orchestra.run.smithi190.stdout: "op_inflight": 0, 2024-05-09T06:37:24.756 INFO:teuthology.orchestra.run.smithi190.stdout: "oplen_avg": { 2024-05-09T06:37:24.756 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-05-09T06:37:24.756 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0 2024-05-09T06:37:24.756 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-05-09T06:37:24.756 INFO:teuthology.orchestra.run.smithi190.stdout: "op": 0, 2024-05-09T06:37:24.756 INFO:teuthology.orchestra.run.smithi190.stdout: "op_r": 0, 2024-05-09T06:37:24.756 INFO:teuthology.orchestra.run.smithi190.stdout: "op_w": 0, 2024-05-09T06:37:24.756 INFO:teuthology.orchestra.run.smithi190.stdout: "op_rmw": 0, 2024-05-09T06:37:24.756 INFO:teuthology.orchestra.run.smithi190.stdout: "op_pg": 0, 2024-05-09T06:37:24.756 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_stat": 0, 2024-05-09T06:37:24.756 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_create": 0, 2024-05-09T06:37:24.756 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_read": 0, 2024-05-09T06:37:24.757 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_write": 0, 2024-05-09T06:37:24.757 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_writefull": 0, 2024-05-09T06:37:24.757 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_writesame": 0, 2024-05-09T06:37:24.757 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_append": 0, 2024-05-09T06:37:24.757 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_zero": 0, 2024-05-09T06:37:24.757 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_truncate": 0, 2024-05-09T06:37:24.757 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_delete": 0, 2024-05-09T06:37:24.757 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_mapext": 0, 2024-05-09T06:37:24.757 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_sparse_read": 0, 2024-05-09T06:37:24.757 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_clonerange": 0, 2024-05-09T06:37:24.757 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_getxattr": 0, 2024-05-09T06:37:24.757 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_setxattr": 0, 2024-05-09T06:37:24.757 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_cmpxattr": 0, 2024-05-09T06:37:24.758 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_rmxattr": 0, 2024-05-09T06:37:24.758 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_resetxattrs": 0, 2024-05-09T06:37:24.758 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_call": 0, 2024-05-09T06:37:24.758 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_watch": 0, 2024-05-09T06:37:24.758 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_notify": 0, 2024-05-09T06:37:24.758 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_src_cmpxattr": 0, 2024-05-09T06:37:24.758 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_pgls": 0, 2024-05-09T06:37:24.758 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_pgls_filter": 0, 2024-05-09T06:37:24.758 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_other": 0, 2024-05-09T06:37:24.758 INFO:teuthology.orchestra.run.smithi190.stdout: "linger_active": 0, 2024-05-09T06:37:24.758 INFO:teuthology.orchestra.run.smithi190.stdout: "linger_send": 0, 2024-05-09T06:37:24.758 INFO:teuthology.orchestra.run.smithi190.stdout: "linger_resend": 0, 2024-05-09T06:37:24.758 INFO:teuthology.orchestra.run.smithi190.stdout: "linger_ping": 0, 2024-05-09T06:37:24.758 INFO:teuthology.orchestra.run.smithi190.stdout: "poolop_active": 0, 2024-05-09T06:37:24.759 INFO:teuthology.orchestra.run.smithi190.stdout: "poolop_send": 0, 2024-05-09T06:37:24.759 INFO:teuthology.orchestra.run.smithi190.stdout: "poolop_resend": 0, 2024-05-09T06:37:24.759 INFO:teuthology.orchestra.run.smithi190.stdout: "poolstat_active": 0, 2024-05-09T06:37:24.759 INFO:teuthology.orchestra.run.smithi190.stdout: "poolstat_send": 0, 2024-05-09T06:37:24.759 INFO:teuthology.orchestra.run.smithi190.stdout: "poolstat_resend": 0, 2024-05-09T06:37:24.759 INFO:teuthology.orchestra.run.smithi190.stdout: "statfs_active": 0, 2024-05-09T06:37:24.759 INFO:teuthology.orchestra.run.smithi190.stdout: "statfs_send": 0, 2024-05-09T06:37:24.759 INFO:teuthology.orchestra.run.smithi190.stdout: "statfs_resend": 0, 2024-05-09T06:37:24.759 INFO:teuthology.orchestra.run.smithi190.stdout: "command_active": 0, 2024-05-09T06:37:24.759 INFO:teuthology.orchestra.run.smithi190.stdout: "command_send": 0, 2024-05-09T06:37:24.759 INFO:teuthology.orchestra.run.smithi190.stdout: "command_resend": 0, 2024-05-09T06:37:24.759 INFO:teuthology.orchestra.run.smithi190.stdout: "map_epoch": 0, 2024-05-09T06:37:24.759 INFO:teuthology.orchestra.run.smithi190.stdout: "map_full": 0, 2024-05-09T06:37:24.759 INFO:teuthology.orchestra.run.smithi190.stdout: "map_inc": 0, 2024-05-09T06:37:24.760 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_sessions": 0, 2024-05-09T06:37:24.760 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_session_open": 0, 2024-05-09T06:37:24.760 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_session_close": 0, 2024-05-09T06:37:24.760 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_laggy": 0, 2024-05-09T06:37:24.760 INFO:teuthology.orchestra.run.smithi190.stdout: "omap_wr": 0, 2024-05-09T06:37:24.760 INFO:teuthology.orchestra.run.smithi190.stdout: "omap_rd": 0, 2024-05-09T06:37:24.760 INFO:teuthology.orchestra.run.smithi190.stdout: "omap_del": 0 2024-05-09T06:37:24.760 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-05-09T06:37:24.760 INFO:teuthology.orchestra.run.smithi190.stdout: "throttle-mgr_client_bytes": { 2024-05-09T06:37:24.760 INFO:teuthology.orchestra.run.smithi190.stdout: "val": 0, 2024-05-09T06:37:24.760 INFO:teuthology.orchestra.run.smithi190.stdout: "max": 134217728, 2024-05-09T06:37:24.760 INFO:teuthology.orchestra.run.smithi190.stdout: "get_started": 0, 2024-05-09T06:37:24.760 INFO:teuthology.orchestra.run.smithi190.stdout: "get": 58, 2024-05-09T06:37:24.760 INFO:teuthology.orchestra.run.smithi190.stdout: "get_sum": 247729, 2024-05-09T06:37:24.760 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_fail": 0, 2024-05-09T06:37:24.761 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_success": 58, 2024-05-09T06:37:24.761 INFO:teuthology.orchestra.run.smithi190.stdout: "take": 0, 2024-05-09T06:37:24.761 INFO:teuthology.orchestra.run.smithi190.stdout: "take_sum": 0, 2024-05-09T06:37:24.761 INFO:teuthology.orchestra.run.smithi190.stdout: "put": 58, 2024-05-09T06:37:24.761 INFO:teuthology.orchestra.run.smithi190.stdout: "put_sum": 247729, 2024-05-09T06:37:24.761 INFO:teuthology.orchestra.run.smithi190.stdout: "wait": { 2024-05-09T06:37:24.761 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-05-09T06:37:24.761 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-05-09T06:37:24.761 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-05-09T06:37:24.761 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-05-09T06:37:24.761 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-05-09T06:37:24.761 INFO:teuthology.orchestra.run.smithi190.stdout: "throttle-mgr_client_messages": { 2024-05-09T06:37:24.761 INFO:teuthology.orchestra.run.smithi190.stdout: "val": 0, 2024-05-09T06:37:24.761 INFO:teuthology.orchestra.run.smithi190.stdout: "max": 512, 2024-05-09T06:37:24.761 INFO:teuthology.orchestra.run.smithi190.stdout: "get_started": 0, 2024-05-09T06:37:24.762 INFO:teuthology.orchestra.run.smithi190.stdout: "get": 58, 2024-05-09T06:37:24.762 INFO:teuthology.orchestra.run.smithi190.stdout: "get_sum": 58, 2024-05-09T06:37:24.762 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_fail": 0, 2024-05-09T06:37:24.762 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_success": 58, 2024-05-09T06:37:24.762 INFO:teuthology.orchestra.run.smithi190.stdout: "take": 0, 2024-05-09T06:37:24.762 INFO:teuthology.orchestra.run.smithi190.stdout: "take_sum": 0, 2024-05-09T06:37:24.762 INFO:teuthology.orchestra.run.smithi190.stdout: "put": 58, 2024-05-09T06:37:24.762 INFO:teuthology.orchestra.run.smithi190.stdout: "put_sum": 58, 2024-05-09T06:37:24.762 INFO:teuthology.orchestra.run.smithi190.stdout: "wait": { 2024-05-09T06:37:24.762 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-05-09T06:37:24.762 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-05-09T06:37:24.762 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-05-09T06:37:24.762 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-05-09T06:37:24.762 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-05-09T06:37:24.763 INFO:teuthology.orchestra.run.smithi190.stdout: "throttle-mgr_mds_bytes": { 2024-05-09T06:37:24.763 INFO:teuthology.orchestra.run.smithi190.stdout: "val": 0, 2024-05-09T06:37:24.763 INFO:teuthology.orchestra.run.smithi190.stdout: "max": 134217728, 2024-05-09T06:37:24.763 INFO:teuthology.orchestra.run.smithi190.stdout: "get_started": 0, 2024-05-09T06:37:24.763 INFO:teuthology.orchestra.run.smithi190.stdout: "get": 83, 2024-05-09T06:37:24.763 INFO:teuthology.orchestra.run.smithi190.stdout: "get_sum": 818922, 2024-05-09T06:37:24.763 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_fail": 0, 2024-05-09T06:37:24.763 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_success": 83, 2024-05-09T06:37:24.763 INFO:teuthology.orchestra.run.smithi190.stdout: "take": 0, 2024-05-09T06:37:24.763 INFO:teuthology.orchestra.run.smithi190.stdout: "take_sum": 0, 2024-05-09T06:37:24.763 INFO:teuthology.orchestra.run.smithi190.stdout: "put": 83, 2024-05-09T06:37:24.763 INFO:teuthology.orchestra.run.smithi190.stdout: "put_sum": 818922, 2024-05-09T06:37:24.763 INFO:teuthology.orchestra.run.smithi190.stdout: "wait": { 2024-05-09T06:37:24.763 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-05-09T06:37:24.764 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-05-09T06:37:24.764 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-05-09T06:37:24.764 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-05-09T06:37:24.764 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-05-09T06:37:24.764 INFO:teuthology.orchestra.run.smithi190.stdout: "throttle-mgr_mds_messsages": { 2024-05-09T06:37:24.764 INFO:teuthology.orchestra.run.smithi190.stdout: "val": 0, 2024-05-09T06:37:24.764 INFO:teuthology.orchestra.run.smithi190.stdout: "max": 128, 2024-05-09T06:37:24.764 INFO:teuthology.orchestra.run.smithi190.stdout: "get_started": 0, 2024-05-09T06:37:24.764 INFO:teuthology.orchestra.run.smithi190.stdout: "get": 83, 2024-05-09T06:37:24.764 INFO:teuthology.orchestra.run.smithi190.stdout: "get_sum": 83, 2024-05-09T06:37:24.764 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_fail": 0, 2024-05-09T06:37:24.764 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_success": 83, 2024-05-09T06:37:24.764 INFO:teuthology.orchestra.run.smithi190.stdout: "take": 0, 2024-05-09T06:37:24.764 INFO:teuthology.orchestra.run.smithi190.stdout: "take_sum": 0, 2024-05-09T06:37:24.764 INFO:teuthology.orchestra.run.smithi190.stdout: "put": 83, 2024-05-09T06:37:24.764 INFO:teuthology.orchestra.run.smithi190.stdout: "put_sum": 83, 2024-05-09T06:37:24.765 INFO:teuthology.orchestra.run.smithi190.stdout: "wait": { 2024-05-09T06:37:24.765 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-05-09T06:37:24.765 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-05-09T06:37:24.765 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-05-09T06:37:24.765 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-05-09T06:37:24.765 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-05-09T06:37:24.765 INFO:teuthology.orchestra.run.smithi190.stdout: "throttle-mgr_mon_bytes": { 2024-05-09T06:37:24.765 INFO:teuthology.orchestra.run.smithi190.stdout: "val": 0, 2024-05-09T06:37:24.765 INFO:teuthology.orchestra.run.smithi190.stdout: "max": 134217728, 2024-05-09T06:37:24.765 INFO:teuthology.orchestra.run.smithi190.stdout: "get_started": 0, 2024-05-09T06:37:24.765 INFO:teuthology.orchestra.run.smithi190.stdout: "get": 63, 2024-05-09T06:37:24.765 INFO:teuthology.orchestra.run.smithi190.stdout: "get_sum": 456374, 2024-05-09T06:37:24.765 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_fail": 0, 2024-05-09T06:37:24.765 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_success": 63, 2024-05-09T06:37:24.766 INFO:teuthology.orchestra.run.smithi190.stdout: "take": 0, 2024-05-09T06:37:24.766 INFO:teuthology.orchestra.run.smithi190.stdout: "take_sum": 0, 2024-05-09T06:37:24.766 INFO:teuthology.orchestra.run.smithi190.stdout: "put": 63, 2024-05-09T06:37:24.766 INFO:teuthology.orchestra.run.smithi190.stdout: "put_sum": 456374, 2024-05-09T06:37:24.766 INFO:teuthology.orchestra.run.smithi190.stdout: "wait": { 2024-05-09T06:37:24.766 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-05-09T06:37:24.766 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-05-09T06:37:24.766 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-05-09T06:37:24.766 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-05-09T06:37:24.766 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-05-09T06:37:24.766 INFO:teuthology.orchestra.run.smithi190.stdout: "throttle-mgr_mon_messsages": { 2024-05-09T06:37:24.766 INFO:teuthology.orchestra.run.smithi190.stdout: "val": 0, 2024-05-09T06:37:24.766 INFO:teuthology.orchestra.run.smithi190.stdout: "max": 128, 2024-05-09T06:37:24.766 INFO:teuthology.orchestra.run.smithi190.stdout: "get_started": 0, 2024-05-09T06:37:24.767 INFO:teuthology.orchestra.run.smithi190.stdout: "get": 63, 2024-05-09T06:37:24.767 INFO:teuthology.orchestra.run.smithi190.stdout: "get_sum": 63, 2024-05-09T06:37:24.767 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_fail": 0, 2024-05-09T06:37:24.767 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_success": 63, 2024-05-09T06:37:24.767 INFO:teuthology.orchestra.run.smithi190.stdout: "take": 0, 2024-05-09T06:37:24.767 INFO:teuthology.orchestra.run.smithi190.stdout: "take_sum": 0, 2024-05-09T06:37:24.767 INFO:teuthology.orchestra.run.smithi190.stdout: "put": 63, 2024-05-09T06:37:24.767 INFO:teuthology.orchestra.run.smithi190.stdout: "put_sum": 63, 2024-05-09T06:37:24.767 INFO:teuthology.orchestra.run.smithi190.stdout: "wait": { 2024-05-09T06:37:24.767 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-05-09T06:37:24.767 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-05-09T06:37:24.767 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-05-09T06:37:24.767 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-05-09T06:37:24.767 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-05-09T06:37:24.767 INFO:teuthology.orchestra.run.smithi190.stdout: "throttle-mgr_osd_bytes": { 2024-05-09T06:37:24.768 INFO:teuthology.orchestra.run.smithi190.stdout: "val": 0, 2024-05-09T06:37:24.768 INFO:teuthology.orchestra.run.smithi190.stdout: "max": 536870912, 2024-05-09T06:37:24.768 INFO:teuthology.orchestra.run.smithi190.stdout: "get_started": 0, 2024-05-09T06:37:24.768 INFO:teuthology.orchestra.run.smithi190.stdout: "get": 344, 2024-05-09T06:37:24.768 INFO:teuthology.orchestra.run.smithi190.stdout: "get_sum": 4284563, 2024-05-09T06:37:24.768 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_fail": 0, 2024-05-09T06:37:24.768 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_success": 344, 2024-05-09T06:37:24.768 INFO:teuthology.orchestra.run.smithi190.stdout: "take": 0, 2024-05-09T06:37:24.768 INFO:teuthology.orchestra.run.smithi190.stdout: "take_sum": 0, 2024-05-09T06:37:24.768 INFO:teuthology.orchestra.run.smithi190.stdout: "put": 344, 2024-05-09T06:37:24.768 INFO:teuthology.orchestra.run.smithi190.stdout: "put_sum": 4284563, 2024-05-09T06:37:24.768 INFO:teuthology.orchestra.run.smithi190.stdout: "wait": { 2024-05-09T06:37:24.768 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-05-09T06:37:24.768 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-05-09T06:37:24.768 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-05-09T06:37:24.768 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-05-09T06:37:24.769 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-05-09T06:37:24.769 INFO:teuthology.orchestra.run.smithi190.stdout: "throttle-mgr_osd_messsages": { 2024-05-09T06:37:24.769 INFO:teuthology.orchestra.run.smithi190.stdout: "val": 0, 2024-05-09T06:37:24.769 INFO:teuthology.orchestra.run.smithi190.stdout: "max": 8192, 2024-05-09T06:37:24.769 INFO:teuthology.orchestra.run.smithi190.stdout: "get_started": 0, 2024-05-09T06:37:24.769 INFO:teuthology.orchestra.run.smithi190.stdout: "get": 344, 2024-05-09T06:37:24.769 INFO:teuthology.orchestra.run.smithi190.stdout: "get_sum": 344, 2024-05-09T06:37:24.769 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_fail": 0, 2024-05-09T06:37:24.769 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_success": 344, 2024-05-09T06:37:24.769 INFO:teuthology.orchestra.run.smithi190.stdout: "take": 0, 2024-05-09T06:37:24.769 INFO:teuthology.orchestra.run.smithi190.stdout: "take_sum": 0, 2024-05-09T06:37:24.769 INFO:teuthology.orchestra.run.smithi190.stdout: "put": 344, 2024-05-09T06:37:24.769 INFO:teuthology.orchestra.run.smithi190.stdout: "put_sum": 344, 2024-05-09T06:37:24.769 INFO:teuthology.orchestra.run.smithi190.stdout: "wait": { 2024-05-09T06:37:24.769 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-05-09T06:37:24.769 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-05-09T06:37:24.770 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-05-09T06:37:24.770 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-05-09T06:37:24.770 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-05-09T06:37:24.770 INFO:teuthology.orchestra.run.smithi190.stdout: "throttle-msgr_dispatch_throttler-client": { 2024-05-09T06:37:24.770 INFO:teuthology.orchestra.run.smithi190.stdout: "val": 0, 2024-05-09T06:37:24.770 INFO:teuthology.orchestra.run.smithi190.stdout: "max": 104857600, 2024-05-09T06:37:24.770 INFO:teuthology.orchestra.run.smithi190.stdout: "get_started": 0, 2024-05-09T06:37:24.770 INFO:teuthology.orchestra.run.smithi190.stdout: "get": 172, 2024-05-09T06:37:24.770 INFO:teuthology.orchestra.run.smithi190.stdout: "get_sum": 66997, 2024-05-09T06:37:24.770 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_fail": 0, 2024-05-09T06:37:24.770 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_success": 172, 2024-05-09T06:37:24.770 INFO:teuthology.orchestra.run.smithi190.stdout: "take": 0, 2024-05-09T06:37:24.770 INFO:teuthology.orchestra.run.smithi190.stdout: "take_sum": 0, 2024-05-09T06:37:24.770 INFO:teuthology.orchestra.run.smithi190.stdout: "put": 172, 2024-05-09T06:37:24.771 INFO:teuthology.orchestra.run.smithi190.stdout: "put_sum": 66997, 2024-05-09T06:37:24.771 INFO:teuthology.orchestra.run.smithi190.stdout: "wait": { 2024-05-09T06:37:24.771 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-05-09T06:37:24.771 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-05-09T06:37:24.771 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-05-09T06:37:24.771 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-05-09T06:37:24.771 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-05-09T06:37:24.771 INFO:teuthology.orchestra.run.smithi190.stdout: "throttle-msgr_dispatch_throttler-client-0x55b09e94b7c0": { 2024-05-09T06:37:24.771 INFO:teuthology.orchestra.run.smithi190.stdout: "val": 0, 2024-05-09T06:37:24.771 INFO:teuthology.orchestra.run.smithi190.stdout: "max": 104857600, 2024-05-09T06:37:24.771 INFO:teuthology.orchestra.run.smithi190.stdout: "get_started": 0, 2024-05-09T06:37:24.771 INFO:teuthology.orchestra.run.smithi190.stdout: "get": 87, 2024-05-09T06:37:24.771 INFO:teuthology.orchestra.run.smithi190.stdout: "get_sum": 46276, 2024-05-09T06:37:24.771 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_fail": 0, 2024-05-09T06:37:24.772 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_success": 87, 2024-05-09T06:37:24.772 INFO:teuthology.orchestra.run.smithi190.stdout: "take": 0, 2024-05-09T06:37:24.772 INFO:teuthology.orchestra.run.smithi190.stdout: "take_sum": 0, 2024-05-09T06:37:24.772 INFO:teuthology.orchestra.run.smithi190.stdout: "put": 87, 2024-05-09T06:37:24.772 INFO:teuthology.orchestra.run.smithi190.stdout: "put_sum": 46276, 2024-05-09T06:37:24.772 INFO:teuthology.orchestra.run.smithi190.stdout: "wait": { 2024-05-09T06:37:24.772 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-05-09T06:37:24.772 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-05-09T06:37:24.772 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-05-09T06:37:24.772 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-05-09T06:37:24.772 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-05-09T06:37:24.772 INFO:teuthology.orchestra.run.smithi190.stdout: "throttle-msgr_dispatch_throttler-client-0x55b09f6c2c80": { 2024-05-09T06:37:24.772 INFO:teuthology.orchestra.run.smithi190.stdout: "val": 0, 2024-05-09T06:37:24.772 INFO:teuthology.orchestra.run.smithi190.stdout: "max": 104857600, 2024-05-09T06:37:24.773 INFO:teuthology.orchestra.run.smithi190.stdout: "get_started": 0, 2024-05-09T06:37:24.773 INFO:teuthology.orchestra.run.smithi190.stdout: "get": 38, 2024-05-09T06:37:24.773 INFO:teuthology.orchestra.run.smithi190.stdout: "get_sum": 21336, 2024-05-09T06:37:24.773 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_fail": 0, 2024-05-09T06:37:24.773 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_success": 38, 2024-05-09T06:37:24.773 INFO:teuthology.orchestra.run.smithi190.stdout: "take": 0, 2024-05-09T06:37:24.773 INFO:teuthology.orchestra.run.smithi190.stdout: "take_sum": 0, 2024-05-09T06:37:24.773 INFO:teuthology.orchestra.run.smithi190.stdout: "put": 38, 2024-05-09T06:37:24.773 INFO:teuthology.orchestra.run.smithi190.stdout: "put_sum": 21336, 2024-05-09T06:37:24.773 INFO:teuthology.orchestra.run.smithi190.stdout: "wait": { 2024-05-09T06:37:24.773 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-05-09T06:37:24.773 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-05-09T06:37:24.773 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-05-09T06:37:24.773 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-05-09T06:37:24.774 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-05-09T06:37:24.774 INFO:teuthology.orchestra.run.smithi190.stdout: "throttle-msgr_dispatch_throttler-mgr": { 2024-05-09T06:37:24.774 INFO:teuthology.orchestra.run.smithi190.stdout: "val": 0, 2024-05-09T06:37:24.774 INFO:teuthology.orchestra.run.smithi190.stdout: "max": 104857600, 2024-05-09T06:37:24.774 INFO:teuthology.orchestra.run.smithi190.stdout: "get_started": 0, 2024-05-09T06:37:24.774 INFO:teuthology.orchestra.run.smithi190.stdout: "get": 355, 2024-05-09T06:37:24.774 INFO:teuthology.orchestra.run.smithi190.stdout: "get_sum": 862821, 2024-05-09T06:37:24.774 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_fail": 0, 2024-05-09T06:37:24.774 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_success": 355, 2024-05-09T06:37:24.774 INFO:teuthology.orchestra.run.smithi190.stdout: "take": 0, 2024-05-09T06:37:24.774 INFO:teuthology.orchestra.run.smithi190.stdout: "take_sum": 0, 2024-05-09T06:37:24.774 INFO:teuthology.orchestra.run.smithi190.stdout: "put": 355, 2024-05-09T06:37:24.774 INFO:teuthology.orchestra.run.smithi190.stdout: "put_sum": 862821, 2024-05-09T06:37:24.774 INFO:teuthology.orchestra.run.smithi190.stdout: "wait": { 2024-05-09T06:37:24.774 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-05-09T06:37:24.775 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-05-09T06:37:24.775 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-05-09T06:37:24.775 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-05-09T06:37:24.775 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-05-09T06:37:24.775 INFO:teuthology.orchestra.run.smithi190.stdout: "throttle-msgr_dispatch_throttler-mgr-0x55b09b8e3c20": { 2024-05-09T06:37:24.775 INFO:teuthology.orchestra.run.smithi190.stdout: "val": 0, 2024-05-09T06:37:24.775 INFO:teuthology.orchestra.run.smithi190.stdout: "max": 104857600, 2024-05-09T06:37:24.775 INFO:teuthology.orchestra.run.smithi190.stdout: "get_started": 0, 2024-05-09T06:37:24.775 INFO:teuthology.orchestra.run.smithi190.stdout: "get": 548, 2024-05-09T06:37:24.775 INFO:teuthology.orchestra.run.smithi190.stdout: "get_sum": 5807588, 2024-05-09T06:37:24.775 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_fail": 0, 2024-05-09T06:37:24.775 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_success": 548, 2024-05-09T06:37:24.775 INFO:teuthology.orchestra.run.smithi190.stdout: "take": 0, 2024-05-09T06:37:24.775 INFO:teuthology.orchestra.run.smithi190.stdout: "take_sum": 0, 2024-05-09T06:37:24.776 INFO:teuthology.orchestra.run.smithi190.stdout: "put": 548, 2024-05-09T06:37:24.776 INFO:teuthology.orchestra.run.smithi190.stdout: "put_sum": 5807588, 2024-05-09T06:37:24.776 INFO:teuthology.orchestra.run.smithi190.stdout: "wait": { 2024-05-09T06:37:24.776 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-05-09T06:37:24.776 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-05-09T06:37:24.776 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-05-09T06:37:24.776 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-05-09T06:37:24.776 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-05-09T06:37:24.776 INFO:teuthology.orchestra.run.smithi190.stdout: "throttle-msgr_dispatch_throttler-radosclient": { 2024-05-09T06:37:24.776 INFO:teuthology.orchestra.run.smithi190.stdout: "val": 0, 2024-05-09T06:37:24.776 INFO:teuthology.orchestra.run.smithi190.stdout: "max": 104857600, 2024-05-09T06:37:24.776 INFO:teuthology.orchestra.run.smithi190.stdout: "get_started": 0, 2024-05-09T06:37:24.776 INFO:teuthology.orchestra.run.smithi190.stdout: "get": 177, 2024-05-09T06:37:24.776 INFO:teuthology.orchestra.run.smithi190.stdout: "get_sum": 543273, 2024-05-09T06:37:24.776 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_fail": 0, 2024-05-09T06:37:24.777 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_success": 177, 2024-05-09T06:37:24.777 INFO:teuthology.orchestra.run.smithi190.stdout: "take": 0, 2024-05-09T06:37:24.777 INFO:teuthology.orchestra.run.smithi190.stdout: "take_sum": 0, 2024-05-09T06:37:24.777 INFO:teuthology.orchestra.run.smithi190.stdout: "put": 177, 2024-05-09T06:37:24.777 INFO:teuthology.orchestra.run.smithi190.stdout: "put_sum": 543273, 2024-05-09T06:37:24.777 INFO:teuthology.orchestra.run.smithi190.stdout: "wait": { 2024-05-09T06:37:24.777 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-05-09T06:37:24.777 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-05-09T06:37:24.777 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-05-09T06:37:24.777 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-05-09T06:37:24.777 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-05-09T06:37:24.777 INFO:teuthology.orchestra.run.smithi190.stdout: "throttle-msgr_dispatch_throttler-radosclient-0x55b09e6ff180": { 2024-05-09T06:37:24.777 INFO:teuthology.orchestra.run.smithi190.stdout: "val": 0, 2024-05-09T06:37:24.777 INFO:teuthology.orchestra.run.smithi190.stdout: "max": 104857600, 2024-05-09T06:37:24.778 INFO:teuthology.orchestra.run.smithi190.stdout: "get_started": 0, 2024-05-09T06:37:24.778 INFO:teuthology.orchestra.run.smithi190.stdout: "get": 12, 2024-05-09T06:37:24.778 INFO:teuthology.orchestra.run.smithi190.stdout: "get_sum": 500443, 2024-05-09T06:37:24.778 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_fail": 0, 2024-05-09T06:37:24.778 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_success": 12, 2024-05-09T06:37:24.778 INFO:teuthology.orchestra.run.smithi190.stdout: "take": 0, 2024-05-09T06:37:24.778 INFO:teuthology.orchestra.run.smithi190.stdout: "take_sum": 0, 2024-05-09T06:37:24.778 INFO:teuthology.orchestra.run.smithi190.stdout: "put": 12, 2024-05-09T06:37:24.778 INFO:teuthology.orchestra.run.smithi190.stdout: "put_sum": 500443, 2024-05-09T06:37:24.778 INFO:teuthology.orchestra.run.smithi190.stdout: "wait": { 2024-05-09T06:37:24.778 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-05-09T06:37:24.778 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-05-09T06:37:24.778 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-05-09T06:37:24.779 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-05-09T06:37:24.779 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-05-09T06:37:24.779 INFO:teuthology.orchestra.run.smithi190.stdout: "throttle-msgr_dispatch_throttler-radosclient-0x55b09e6ff900": { 2024-05-09T06:37:24.779 INFO:teuthology.orchestra.run.smithi190.stdout: "val": 0, 2024-05-09T06:37:24.779 INFO:teuthology.orchestra.run.smithi190.stdout: "max": 104857600, 2024-05-09T06:37:24.779 INFO:teuthology.orchestra.run.smithi190.stdout: "get_started": 0, 2024-05-09T06:37:24.779 INFO:teuthology.orchestra.run.smithi190.stdout: "get": 11, 2024-05-09T06:37:24.779 INFO:teuthology.orchestra.run.smithi190.stdout: "get_sum": 500419, 2024-05-09T06:37:24.779 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_fail": 0, 2024-05-09T06:37:24.779 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_success": 11, 2024-05-09T06:37:24.779 INFO:teuthology.orchestra.run.smithi190.stdout: "take": 0, 2024-05-09T06:37:24.779 INFO:teuthology.orchestra.run.smithi190.stdout: "take_sum": 0, 2024-05-09T06:37:24.779 INFO:teuthology.orchestra.run.smithi190.stdout: "put": 11, 2024-05-09T06:37:24.779 INFO:teuthology.orchestra.run.smithi190.stdout: "put_sum": 500419, 2024-05-09T06:37:24.779 INFO:teuthology.orchestra.run.smithi190.stdout: "wait": { 2024-05-09T06:37:24.780 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-05-09T06:37:24.780 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-05-09T06:37:24.780 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-05-09T06:37:24.780 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-05-09T06:37:24.780 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-05-09T06:37:24.780 INFO:teuthology.orchestra.run.smithi190.stdout: "throttle-msgr_dispatch_throttler-radosclient-0x55b09e94a820": { 2024-05-09T06:37:24.780 INFO:teuthology.orchestra.run.smithi190.stdout: "val": 0, 2024-05-09T06:37:24.780 INFO:teuthology.orchestra.run.smithi190.stdout: "max": 104857600, 2024-05-09T06:37:24.780 INFO:teuthology.orchestra.run.smithi190.stdout: "get_started": 0, 2024-05-09T06:37:24.780 INFO:teuthology.orchestra.run.smithi190.stdout: "get": 10, 2024-05-09T06:37:24.780 INFO:teuthology.orchestra.run.smithi190.stdout: "get_sum": 368214, 2024-05-09T06:37:24.780 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_fail": 0, 2024-05-09T06:37:24.780 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_success": 10, 2024-05-09T06:37:24.780 INFO:teuthology.orchestra.run.smithi190.stdout: "take": 0, 2024-05-09T06:37:24.780 INFO:teuthology.orchestra.run.smithi190.stdout: "take_sum": 0, 2024-05-09T06:37:24.780 INFO:teuthology.orchestra.run.smithi190.stdout: "put": 10, 2024-05-09T06:37:24.781 INFO:teuthology.orchestra.run.smithi190.stdout: "put_sum": 368214, 2024-05-09T06:37:24.781 INFO:teuthology.orchestra.run.smithi190.stdout: "wait": { 2024-05-09T06:37:24.781 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-05-09T06:37:24.781 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-05-09T06:37:24.781 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-05-09T06:37:24.781 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-05-09T06:37:24.781 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-05-09T06:37:24.781 INFO:teuthology.orchestra.run.smithi190.stdout: "throttle-objecter_bytes": { 2024-05-09T06:37:24.781 INFO:teuthology.orchestra.run.smithi190.stdout: "val": 0, 2024-05-09T06:37:24.781 INFO:teuthology.orchestra.run.smithi190.stdout: "max": 104857600, 2024-05-09T06:37:24.781 INFO:teuthology.orchestra.run.smithi190.stdout: "get_started": 0, 2024-05-09T06:37:24.781 INFO:teuthology.orchestra.run.smithi190.stdout: "get": 0, 2024-05-09T06:37:24.781 INFO:teuthology.orchestra.run.smithi190.stdout: "get_sum": 0, 2024-05-09T06:37:24.781 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_fail": 0, 2024-05-09T06:37:24.781 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_success": 0, 2024-05-09T06:37:24.782 INFO:teuthology.orchestra.run.smithi190.stdout: "take": 0, 2024-05-09T06:37:24.782 INFO:teuthology.orchestra.run.smithi190.stdout: "take_sum": 0, 2024-05-09T06:37:24.782 INFO:teuthology.orchestra.run.smithi190.stdout: "put": 0, 2024-05-09T06:37:24.782 INFO:teuthology.orchestra.run.smithi190.stdout: "put_sum": 0, 2024-05-09T06:37:24.782 INFO:teuthology.orchestra.run.smithi190.stdout: "wait": { 2024-05-09T06:37:24.782 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-05-09T06:37:24.782 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-05-09T06:37:24.782 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-05-09T06:37:24.782 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-05-09T06:37:24.782 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-05-09T06:37:24.782 INFO:teuthology.orchestra.run.smithi190.stdout: "throttle-objecter_bytes-0x55b09e6fedc0": { 2024-05-09T06:37:24.782 INFO:teuthology.orchestra.run.smithi190.stdout: "val": 0, 2024-05-09T06:37:24.782 INFO:teuthology.orchestra.run.smithi190.stdout: "max": 104857600, 2024-05-09T06:37:24.782 INFO:teuthology.orchestra.run.smithi190.stdout: "get_started": 0, 2024-05-09T06:37:24.782 INFO:teuthology.orchestra.run.smithi190.stdout: "get": 163, 2024-05-09T06:37:24.783 INFO:teuthology.orchestra.run.smithi190.stdout: "get_sum": 2433049, 2024-05-09T06:37:24.783 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_fail": 0, 2024-05-09T06:37:24.783 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_success": 163, 2024-05-09T06:37:24.783 INFO:teuthology.orchestra.run.smithi190.stdout: "take": 0, 2024-05-09T06:37:24.783 INFO:teuthology.orchestra.run.smithi190.stdout: "take_sum": 0, 2024-05-09T06:37:24.783 INFO:teuthology.orchestra.run.smithi190.stdout: "put": 163, 2024-05-09T06:37:24.783 INFO:teuthology.orchestra.run.smithi190.stdout: "put_sum": 2433049, 2024-05-09T06:37:24.783 INFO:teuthology.orchestra.run.smithi190.stdout: "wait": { 2024-05-09T06:37:24.783 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-05-09T06:37:24.783 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-05-09T06:37:24.783 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-05-09T06:37:24.783 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-05-09T06:37:24.783 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-05-09T06:37:24.783 INFO:teuthology.orchestra.run.smithi190.stdout: "throttle-objecter_bytes-0x55b09e6ff360": { 2024-05-09T06:37:24.783 INFO:teuthology.orchestra.run.smithi190.stdout: "val": 0, 2024-05-09T06:37:24.784 INFO:teuthology.orchestra.run.smithi190.stdout: "max": 104857600, 2024-05-09T06:37:24.784 INFO:teuthology.orchestra.run.smithi190.stdout: "get_started": 0, 2024-05-09T06:37:24.784 INFO:teuthology.orchestra.run.smithi190.stdout: "get": 0, 2024-05-09T06:37:24.784 INFO:teuthology.orchestra.run.smithi190.stdout: "get_sum": 0, 2024-05-09T06:37:24.784 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_fail": 0, 2024-05-09T06:37:24.784 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_success": 0, 2024-05-09T06:37:24.784 INFO:teuthology.orchestra.run.smithi190.stdout: "take": 0, 2024-05-09T06:37:24.784 INFO:teuthology.orchestra.run.smithi190.stdout: "take_sum": 0, 2024-05-09T06:37:24.784 INFO:teuthology.orchestra.run.smithi190.stdout: "put": 0, 2024-05-09T06:37:24.784 INFO:teuthology.orchestra.run.smithi190.stdout: "put_sum": 0, 2024-05-09T06:37:24.784 INFO:teuthology.orchestra.run.smithi190.stdout: "wait": { 2024-05-09T06:37:24.784 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-05-09T06:37:24.784 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-05-09T06:37:24.784 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-05-09T06:37:24.784 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-05-09T06:37:24.784 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-05-09T06:37:24.785 INFO:teuthology.orchestra.run.smithi190.stdout: "throttle-objecter_bytes-0x55b09e6ff860": { 2024-05-09T06:37:24.785 INFO:teuthology.orchestra.run.smithi190.stdout: "val": 0, 2024-05-09T06:37:24.785 INFO:teuthology.orchestra.run.smithi190.stdout: "max": 104857600, 2024-05-09T06:37:24.785 INFO:teuthology.orchestra.run.smithi190.stdout: "get_started": 0, 2024-05-09T06:37:24.785 INFO:teuthology.orchestra.run.smithi190.stdout: "get": 0, 2024-05-09T06:37:24.785 INFO:teuthology.orchestra.run.smithi190.stdout: "get_sum": 0, 2024-05-09T06:37:24.785 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_fail": 0, 2024-05-09T06:37:24.785 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_success": 0, 2024-05-09T06:37:24.785 INFO:teuthology.orchestra.run.smithi190.stdout: "take": 0, 2024-05-09T06:37:24.785 INFO:teuthology.orchestra.run.smithi190.stdout: "take_sum": 0, 2024-05-09T06:37:24.785 INFO:teuthology.orchestra.run.smithi190.stdout: "put": 0, 2024-05-09T06:37:24.785 INFO:teuthology.orchestra.run.smithi190.stdout: "put_sum": 0, 2024-05-09T06:37:24.785 INFO:teuthology.orchestra.run.smithi190.stdout: "wait": { 2024-05-09T06:37:24.785 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-05-09T06:37:24.785 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-05-09T06:37:24.785 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-05-09T06:37:24.786 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-05-09T06:37:24.786 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-05-09T06:37:24.786 INFO:teuthology.orchestra.run.smithi190.stdout: "throttle-objecter_bytes-0x55b09e94a500": { 2024-05-09T06:37:24.786 INFO:teuthology.orchestra.run.smithi190.stdout: "val": 0, 2024-05-09T06:37:24.786 INFO:teuthology.orchestra.run.smithi190.stdout: "max": 104857600, 2024-05-09T06:37:24.786 INFO:teuthology.orchestra.run.smithi190.stdout: "get_started": 0, 2024-05-09T06:37:24.786 INFO:teuthology.orchestra.run.smithi190.stdout: "get": 1, 2024-05-09T06:37:24.786 INFO:teuthology.orchestra.run.smithi190.stdout: "get_sum": 0, 2024-05-09T06:37:24.786 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_fail": 0, 2024-05-09T06:37:24.786 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_success": 1, 2024-05-09T06:37:24.786 INFO:teuthology.orchestra.run.smithi190.stdout: "take": 0, 2024-05-09T06:37:24.786 INFO:teuthology.orchestra.run.smithi190.stdout: "take_sum": 0, 2024-05-09T06:37:24.786 INFO:teuthology.orchestra.run.smithi190.stdout: "put": 1, 2024-05-09T06:37:24.786 INFO:teuthology.orchestra.run.smithi190.stdout: "put_sum": 0, 2024-05-09T06:37:24.786 INFO:teuthology.orchestra.run.smithi190.stdout: "wait": { 2024-05-09T06:37:24.786 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-05-09T06:37:24.787 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-05-09T06:37:24.787 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-05-09T06:37:24.787 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-05-09T06:37:24.787 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-05-09T06:37:24.787 INFO:teuthology.orchestra.run.smithi190.stdout: "throttle-objecter_bytes-0x55b09e94b860": { 2024-05-09T06:37:24.787 INFO:teuthology.orchestra.run.smithi190.stdout: "val": 0, 2024-05-09T06:37:24.787 INFO:teuthology.orchestra.run.smithi190.stdout: "max": 104857600, 2024-05-09T06:37:24.787 INFO:teuthology.orchestra.run.smithi190.stdout: "get_started": 0, 2024-05-09T06:37:24.787 INFO:teuthology.orchestra.run.smithi190.stdout: "get": 0, 2024-05-09T06:37:24.787 INFO:teuthology.orchestra.run.smithi190.stdout: "get_sum": 0, 2024-05-09T06:37:24.787 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_fail": 0, 2024-05-09T06:37:24.787 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_success": 0, 2024-05-09T06:37:24.787 INFO:teuthology.orchestra.run.smithi190.stdout: "take": 0, 2024-05-09T06:37:24.787 INFO:teuthology.orchestra.run.smithi190.stdout: "take_sum": 0, 2024-05-09T06:37:24.787 INFO:teuthology.orchestra.run.smithi190.stdout: "put": 0, 2024-05-09T06:37:24.787 INFO:teuthology.orchestra.run.smithi190.stdout: "put_sum": 0, 2024-05-09T06:37:24.788 INFO:teuthology.orchestra.run.smithi190.stdout: "wait": { 2024-05-09T06:37:24.788 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-05-09T06:37:24.788 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-05-09T06:37:24.788 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-05-09T06:37:24.788 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-05-09T06:37:24.788 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-05-09T06:37:24.788 INFO:teuthology.orchestra.run.smithi190.stdout: "throttle-objecter_bytes-0x55b09f6c2000": { 2024-05-09T06:37:24.788 INFO:teuthology.orchestra.run.smithi190.stdout: "val": 0, 2024-05-09T06:37:24.788 INFO:teuthology.orchestra.run.smithi190.stdout: "max": 104857600, 2024-05-09T06:37:24.788 INFO:teuthology.orchestra.run.smithi190.stdout: "get_started": 0, 2024-05-09T06:37:24.788 INFO:teuthology.orchestra.run.smithi190.stdout: "get": 0, 2024-05-09T06:37:24.788 INFO:teuthology.orchestra.run.smithi190.stdout: "get_sum": 0, 2024-05-09T06:37:24.789 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_fail": 0, 2024-05-09T06:37:24.789 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_success": 0, 2024-05-09T06:37:24.789 INFO:teuthology.orchestra.run.smithi190.stdout: "take": 3, 2024-05-09T06:37:24.789 INFO:teuthology.orchestra.run.smithi190.stdout: "take_sum": 410, 2024-05-09T06:37:24.789 INFO:teuthology.orchestra.run.smithi190.stdout: "put": 3, 2024-05-09T06:37:24.789 INFO:teuthology.orchestra.run.smithi190.stdout: "put_sum": 410, 2024-05-09T06:37:24.789 INFO:teuthology.orchestra.run.smithi190.stdout: "wait": { 2024-05-09T06:37:24.789 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-05-09T06:37:24.789 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-05-09T06:37:24.790 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-05-09T06:37:24.790 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-05-09T06:37:24.790 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-05-09T06:37:24.790 INFO:teuthology.orchestra.run.smithi190.stdout: "throttle-objecter_bytes-0x55b09f6c25a0": { 2024-05-09T06:37:24.790 INFO:teuthology.orchestra.run.smithi190.stdout: "val": 0, 2024-05-09T06:37:24.790 INFO:teuthology.orchestra.run.smithi190.stdout: "max": 104857600, 2024-05-09T06:37:24.790 INFO:teuthology.orchestra.run.smithi190.stdout: "get_started": 0, 2024-05-09T06:37:24.790 INFO:teuthology.orchestra.run.smithi190.stdout: "get": 0, 2024-05-09T06:37:24.791 INFO:teuthology.orchestra.run.smithi190.stdout: "get_sum": 0, 2024-05-09T06:37:24.791 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_fail": 0, 2024-05-09T06:37:24.791 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_success": 0, 2024-05-09T06:37:24.791 INFO:teuthology.orchestra.run.smithi190.stdout: "take": 0, 2024-05-09T06:37:24.791 INFO:teuthology.orchestra.run.smithi190.stdout: "take_sum": 0, 2024-05-09T06:37:24.791 INFO:teuthology.orchestra.run.smithi190.stdout: "put": 0, 2024-05-09T06:37:24.791 INFO:teuthology.orchestra.run.smithi190.stdout: "put_sum": 0, 2024-05-09T06:37:24.791 INFO:teuthology.orchestra.run.smithi190.stdout: "wait": { 2024-05-09T06:37:24.791 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-05-09T06:37:24.791 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-05-09T06:37:24.791 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-05-09T06:37:24.791 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-05-09T06:37:24.791 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-05-09T06:37:24.791 INFO:teuthology.orchestra.run.smithi190.stdout: "throttle-objecter_ops": { 2024-05-09T06:37:24.792 INFO:teuthology.orchestra.run.smithi190.stdout: "val": 0, 2024-05-09T06:37:24.792 INFO:teuthology.orchestra.run.smithi190.stdout: "max": 1024, 2024-05-09T06:37:24.792 INFO:teuthology.orchestra.run.smithi190.stdout: "get_started": 0, 2024-05-09T06:37:24.792 INFO:teuthology.orchestra.run.smithi190.stdout: "get": 0, 2024-05-09T06:37:24.792 INFO:teuthology.orchestra.run.smithi190.stdout: "get_sum": 0, 2024-05-09T06:37:24.792 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_fail": 0, 2024-05-09T06:37:24.792 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_success": 0, 2024-05-09T06:37:24.792 INFO:teuthology.orchestra.run.smithi190.stdout: "take": 0, 2024-05-09T06:37:24.792 INFO:teuthology.orchestra.run.smithi190.stdout: "take_sum": 0, 2024-05-09T06:37:24.792 INFO:teuthology.orchestra.run.smithi190.stdout: "put": 0, 2024-05-09T06:37:24.792 INFO:teuthology.orchestra.run.smithi190.stdout: "put_sum": 0, 2024-05-09T06:37:24.792 INFO:teuthology.orchestra.run.smithi190.stdout: "wait": { 2024-05-09T06:37:24.793 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-05-09T06:37:24.793 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-05-09T06:37:24.793 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-05-09T06:37:24.793 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-05-09T06:37:24.793 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-05-09T06:37:24.793 INFO:teuthology.orchestra.run.smithi190.stdout: "throttle-objecter_ops-0x55b09e6fee60": { 2024-05-09T06:37:24.793 INFO:teuthology.orchestra.run.smithi190.stdout: "val": 0, 2024-05-09T06:37:24.793 INFO:teuthology.orchestra.run.smithi190.stdout: "max": 1024, 2024-05-09T06:37:24.793 INFO:teuthology.orchestra.run.smithi190.stdout: "get_started": 0, 2024-05-09T06:37:24.793 INFO:teuthology.orchestra.run.smithi190.stdout: "get": 163, 2024-05-09T06:37:24.793 INFO:teuthology.orchestra.run.smithi190.stdout: "get_sum": 163, 2024-05-09T06:37:24.793 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_fail": 0, 2024-05-09T06:37:24.794 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_success": 163, 2024-05-09T06:37:24.794 INFO:teuthology.orchestra.run.smithi190.stdout: "take": 0, 2024-05-09T06:37:24.794 INFO:teuthology.orchestra.run.smithi190.stdout: "take_sum": 0, 2024-05-09T06:37:24.794 INFO:teuthology.orchestra.run.smithi190.stdout: "put": 163, 2024-05-09T06:37:24.794 INFO:teuthology.orchestra.run.smithi190.stdout: "put_sum": 163, 2024-05-09T06:37:24.794 INFO:teuthology.orchestra.run.smithi190.stdout: "wait": { 2024-05-09T06:37:24.794 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-05-09T06:37:24.794 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-05-09T06:37:24.794 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-05-09T06:37:24.794 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-05-09T06:37:24.794 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-05-09T06:37:24.794 INFO:teuthology.orchestra.run.smithi190.stdout: "throttle-objecter_ops-0x55b09e6ff0e0": { 2024-05-09T06:37:24.794 INFO:teuthology.orchestra.run.smithi190.stdout: "val": 0, 2024-05-09T06:37:24.795 INFO:teuthology.orchestra.run.smithi190.stdout: "max": 1024, 2024-05-09T06:37:24.795 INFO:teuthology.orchestra.run.smithi190.stdout: "get_started": 0, 2024-05-09T06:37:24.795 INFO:teuthology.orchestra.run.smithi190.stdout: "get": 0, 2024-05-09T06:37:24.795 INFO:teuthology.orchestra.run.smithi190.stdout: "get_sum": 0, 2024-05-09T06:37:24.795 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_fail": 0, 2024-05-09T06:37:24.795 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_success": 0, 2024-05-09T06:37:24.795 INFO:teuthology.orchestra.run.smithi190.stdout: "take": 0, 2024-05-09T06:37:24.795 INFO:teuthology.orchestra.run.smithi190.stdout: "take_sum": 0, 2024-05-09T06:37:24.795 INFO:teuthology.orchestra.run.smithi190.stdout: "put": 0, 2024-05-09T06:37:24.795 INFO:teuthology.orchestra.run.smithi190.stdout: "put_sum": 0, 2024-05-09T06:37:24.795 INFO:teuthology.orchestra.run.smithi190.stdout: "wait": { 2024-05-09T06:37:24.795 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-05-09T06:37:24.795 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-05-09T06:37:24.795 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-05-09T06:37:24.795 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-05-09T06:37:24.795 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-05-09T06:37:24.796 INFO:teuthology.orchestra.run.smithi190.stdout: "throttle-objecter_ops-0x55b09e6ff7c0": { 2024-05-09T06:37:24.796 INFO:teuthology.orchestra.run.smithi190.stdout: "val": 0, 2024-05-09T06:37:24.796 INFO:teuthology.orchestra.run.smithi190.stdout: "max": 1024, 2024-05-09T06:37:24.796 INFO:teuthology.orchestra.run.smithi190.stdout: "get_started": 0, 2024-05-09T06:37:24.796 INFO:teuthology.orchestra.run.smithi190.stdout: "get": 0, 2024-05-09T06:37:24.796 INFO:teuthology.orchestra.run.smithi190.stdout: "get_sum": 0, 2024-05-09T06:37:24.796 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_fail": 0, 2024-05-09T06:37:24.796 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_success": 0, 2024-05-09T06:37:24.796 INFO:teuthology.orchestra.run.smithi190.stdout: "take": 0, 2024-05-09T06:37:24.796 INFO:teuthology.orchestra.run.smithi190.stdout: "take_sum": 0, 2024-05-09T06:37:24.796 INFO:teuthology.orchestra.run.smithi190.stdout: "put": 0, 2024-05-09T06:37:24.796 INFO:teuthology.orchestra.run.smithi190.stdout: "put_sum": 0, 2024-05-09T06:37:24.796 INFO:teuthology.orchestra.run.smithi190.stdout: "wait": { 2024-05-09T06:37:24.796 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-05-09T06:37:24.796 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-05-09T06:37:24.796 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-05-09T06:37:24.797 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-05-09T06:37:24.797 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-05-09T06:37:24.797 INFO:teuthology.orchestra.run.smithi190.stdout: "throttle-objecter_ops-0x55b09e94ab40": { 2024-05-09T06:37:24.797 INFO:teuthology.orchestra.run.smithi190.stdout: "val": 0, 2024-05-09T06:37:24.797 INFO:teuthology.orchestra.run.smithi190.stdout: "max": 1024, 2024-05-09T06:37:24.797 INFO:teuthology.orchestra.run.smithi190.stdout: "get_started": 0, 2024-05-09T06:37:24.797 INFO:teuthology.orchestra.run.smithi190.stdout: "get": 1, 2024-05-09T06:37:24.797 INFO:teuthology.orchestra.run.smithi190.stdout: "get_sum": 1, 2024-05-09T06:37:24.797 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_fail": 0, 2024-05-09T06:37:24.797 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_success": 1, 2024-05-09T06:37:24.797 INFO:teuthology.orchestra.run.smithi190.stdout: "take": 0, 2024-05-09T06:37:24.797 INFO:teuthology.orchestra.run.smithi190.stdout: "take_sum": 0, 2024-05-09T06:37:24.797 INFO:teuthology.orchestra.run.smithi190.stdout: "put": 1, 2024-05-09T06:37:24.797 INFO:teuthology.orchestra.run.smithi190.stdout: "put_sum": 1, 2024-05-09T06:37:24.797 INFO:teuthology.orchestra.run.smithi190.stdout: "wait": { 2024-05-09T06:37:24.798 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-05-09T06:37:24.798 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-05-09T06:37:24.798 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-05-09T06:37:24.798 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-05-09T06:37:24.798 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-05-09T06:37:24.798 INFO:teuthology.orchestra.run.smithi190.stdout: "throttle-objecter_ops-0x55b09e94bc20": { 2024-05-09T06:37:24.798 INFO:teuthology.orchestra.run.smithi190.stdout: "val": 0, 2024-05-09T06:37:24.798 INFO:teuthology.orchestra.run.smithi190.stdout: "max": 1024, 2024-05-09T06:37:24.798 INFO:teuthology.orchestra.run.smithi190.stdout: "get_started": 0, 2024-05-09T06:37:24.798 INFO:teuthology.orchestra.run.smithi190.stdout: "get": 0, 2024-05-09T06:37:24.798 INFO:teuthology.orchestra.run.smithi190.stdout: "get_sum": 0, 2024-05-09T06:37:24.798 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_fail": 0, 2024-05-09T06:37:24.798 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_success": 0, 2024-05-09T06:37:24.799 INFO:teuthology.orchestra.run.smithi190.stdout: "take": 0, 2024-05-09T06:37:24.799 INFO:teuthology.orchestra.run.smithi190.stdout: "take_sum": 0, 2024-05-09T06:37:24.799 INFO:teuthology.orchestra.run.smithi190.stdout: "put": 0, 2024-05-09T06:37:24.799 INFO:teuthology.orchestra.run.smithi190.stdout: "put_sum": 0, 2024-05-09T06:37:24.799 INFO:teuthology.orchestra.run.smithi190.stdout: "wait": { 2024-05-09T06:37:24.799 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-05-09T06:37:24.799 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-05-09T06:37:24.799 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-05-09T06:37:24.799 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-05-09T06:37:24.799 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-05-09T06:37:24.799 INFO:teuthology.orchestra.run.smithi190.stdout: "throttle-objecter_ops-0x55b09f6c2500": { 2024-05-09T06:37:24.799 INFO:teuthology.orchestra.run.smithi190.stdout: "val": 0, 2024-05-09T06:37:24.799 INFO:teuthology.orchestra.run.smithi190.stdout: "max": 1024, 2024-05-09T06:37:24.799 INFO:teuthology.orchestra.run.smithi190.stdout: "get_started": 0, 2024-05-09T06:37:24.800 INFO:teuthology.orchestra.run.smithi190.stdout: "get": 0, 2024-05-09T06:37:24.800 INFO:teuthology.orchestra.run.smithi190.stdout: "get_sum": 0, 2024-05-09T06:37:24.800 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_fail": 0, 2024-05-09T06:37:24.800 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_success": 0, 2024-05-09T06:37:24.800 INFO:teuthology.orchestra.run.smithi190.stdout: "take": 0, 2024-05-09T06:37:24.800 INFO:teuthology.orchestra.run.smithi190.stdout: "take_sum": 0, 2024-05-09T06:37:24.800 INFO:teuthology.orchestra.run.smithi190.stdout: "put": 0, 2024-05-09T06:37:24.800 INFO:teuthology.orchestra.run.smithi190.stdout: "put_sum": 0, 2024-05-09T06:37:24.800 INFO:teuthology.orchestra.run.smithi190.stdout: "wait": { 2024-05-09T06:37:24.800 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-05-09T06:37:24.800 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-05-09T06:37:24.800 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-05-09T06:37:24.800 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-05-09T06:37:24.800 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-05-09T06:37:24.800 INFO:teuthology.orchestra.run.smithi190.stdout: "throttle-objecter_ops-0x55b09f6c28c0": { 2024-05-09T06:37:24.800 INFO:teuthology.orchestra.run.smithi190.stdout: "val": 0, 2024-05-09T06:37:24.801 INFO:teuthology.orchestra.run.smithi190.stdout: "max": 1024, 2024-05-09T06:37:24.801 INFO:teuthology.orchestra.run.smithi190.stdout: "get_started": 0, 2024-05-09T06:37:24.801 INFO:teuthology.orchestra.run.smithi190.stdout: "get": 0, 2024-05-09T06:37:24.801 INFO:teuthology.orchestra.run.smithi190.stdout: "get_sum": 0, 2024-05-09T06:37:24.801 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_fail": 0, 2024-05-09T06:37:24.801 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_success": 0, 2024-05-09T06:37:24.801 INFO:teuthology.orchestra.run.smithi190.stdout: "take": 3, 2024-05-09T06:37:24.801 INFO:teuthology.orchestra.run.smithi190.stdout: "take_sum": 3, 2024-05-09T06:37:24.801 INFO:teuthology.orchestra.run.smithi190.stdout: "put": 3, 2024-05-09T06:37:24.801 INFO:teuthology.orchestra.run.smithi190.stdout: "put_sum": 3, 2024-05-09T06:37:24.801 INFO:teuthology.orchestra.run.smithi190.stdout: "wait": { 2024-05-09T06:37:24.801 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-05-09T06:37:24.801 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-05-09T06:37:24.801 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-05-09T06:37:24.801 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-05-09T06:37:24.801 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-05-09T06:37:24.802 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-05-09T06:37:24.803 INFO:tasks.check_counter:Daemon mgr.x finisher-volumes.complete_latency.avgcount=9 2024-05-09T06:37:24.803 INFO:tasks.check_counter:Daemon mgr.x finisher-volumes.queue_len=0 2024-05-09T06:37:24.804 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-05-09T06:37:24.814 INFO:tasks.kclient:Unmounting kernel clients... 2024-05-09T06:37:24.815 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-05-09T06:37:24.815 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T06:37:24.835 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057103,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13674 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249452k,nr_inodes=812363,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-09T06:37:24.835 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-05-09T06:37:24.835 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T06:37:24.895 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057103,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13674 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249452k,nr_inodes=812363,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-09T06:37:24.895 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-05-09T06:37:24.895 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T06:37:24.915 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056337,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497676k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19681 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497676k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-09T06:37:24.915 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-05-09T06:37:24.915 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T06:37:24.972 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056337,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497676k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19681 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497676k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-09T06:37:24.972 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:37:24.972 DEBUG:teuthology.orchestra.run.smithi003:> ip netns list 2024-05-09T06:37:24.992 INFO:teuthology.orchestra.run.smithi003.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-05-09T06:37:24.992 INFO:teuthology.orchestra.run.smithi003.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-09T06:37:24.992 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:37:24.992 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-09T06:37:25.063 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:37:25.063 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-09T06:37:25.134 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:37:25.134 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip link delete ceph-brx 2024-05-09T06:37:25.229 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:37:25.229 DEBUG:teuthology.orchestra.run.smithi003:> ip netns list 2024-05-09T06:37:25.246 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:37:25.247 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip link delete ceph-brx 2024-05-09T06:37:25.316 INFO:teuthology.orchestra.run.smithi003.stderr:Cannot find device "ceph-brx" 2024-05-09T06:37:25.318 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T06:37:25.318 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:37:25.318 DEBUG:teuthology.orchestra.run.smithi190:> ip netns list 2024-05-09T06:37:25.337 INFO:teuthology.orchestra.run.smithi190.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-05-09T06:37:25.337 INFO:teuthology.orchestra.run.smithi190.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-05-09T06:37:25.337 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:37:25.338 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2024-05-09T06:37:25.406 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:37:25.406 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-09T06:37:25.476 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:37:25.476 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip link delete ceph-brx 2024-05-09T06:37:25.578 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:37:25.578 DEBUG:teuthology.orchestra.run.smithi190:> ip netns list 2024-05-09T06:37:25.594 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:37:25.594 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip link delete ceph-brx 2024-05-09T06:37:25.663 INFO:teuthology.orchestra.run.smithi190.stderr:Cannot find device "ceph-brx" 2024-05-09T06:37:25.664 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T06:37:25.664 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-05-09T06:37:25.674 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-05-09T06:37:25.675 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-09T06:37:25.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:25.806+0000 7f25c0112640 1 Processor -- start 2024-05-09T06:37:25.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:25.806+0000 7f25c0112640 1 -- start start 2024-05-09T06:37:25.788 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:25.807+0000 7f25c0112640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f25b80ccf80 0x7f25b80cf720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:25.788 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:25.807+0000 7f25c0112640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f25b813e4a0 con 0x7f25b81408f0 2024-05-09T06:37:25.788 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:25.807+0000 7f25c0112640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f25b80ce960 con 0x7f25b813e0c0 2024-05-09T06:37:25.788 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:25.807+0000 7f25c0112640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f25b8141410 con 0x7f25b80ccf80 2024-05-09T06:37:25.788 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:25.807+0000 7f25bd6ad640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f25b80ccf80 0x7f25b80cf720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:25.788 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:25.807+0000 7f25be6af640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f25b81408f0 0x7f25b8140d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:43100/0 (socket says 172.21.15.3:43100) 2024-05-09T06:37:25.789 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:25.807+0000 7f25be6af640 1 -- 172.21.15.3:0/2741457738 learned_addr learned my addr 172.21.15.3:0/2741457738 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-05-09T06:37:25.789 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:25.807+0000 7f25bd6ad640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f25b80ccf80 0x7f25b80cf720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40644/0 (socket says 172.21.15.3:40644) 2024-05-09T06:37:25.789 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:25.807+0000 7f25bd6ad640 1 -- 172.21.15.3:0/2741457738 >> v1:172.21.15.190:6789/0 conn(0x7f25b813e0c0 legacy=0x7f25b80cc870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:25.789 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:25.807+0000 7f25bd6ad640 1 -- 172.21.15.3:0/2741457738 >> v1:172.21.15.3:6789/0 conn(0x7f25b81408f0 legacy=0x7f25b8140d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:25.789 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:25.807+0000 7f25bd6ad640 1 -- 172.21.15.3:0/2741457738 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25b80cec40 con 0x7f25b80ccf80 2024-05-09T06:37:25.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:25.808+0000 7f25bd6ad640 1 --2- 172.21.15.3:0/2741457738 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f25b80ccf80 0x7f25b80cf720 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f25a80097a0 tx=0x7f25a802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9cac0a8d535ac27 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:25.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:25.808+0000 7f25bceac640 1 -- 172.21.15.3:0/2741457738 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25a8037070 con 0x7f25b80ccf80 2024-05-09T06:37:25.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:25.808+0000 7f25bceac640 1 -- 172.21.15.3:0/2741457738 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f25a8032440 con 0x7f25b80ccf80 2024-05-09T06:37:25.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:25.808+0000 7f25bceac640 1 -- 172.21.15.3:0/2741457738 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25a8032810 con 0x7f25b80ccf80 2024-05-09T06:37:25.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:25.809+0000 7f25c0112640 1 -- 172.21.15.3:0/2741457738 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f25b80ccf80 msgr2=0x7f25b80cf720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:25.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:25.809+0000 7f25c0112640 1 --2- 172.21.15.3:0/2741457738 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f25b80ccf80 0x7f25b80cf720 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f25a80097a0 tx=0x7f25a802a230 comp rx=0 tx=0).stop 2024-05-09T06:37:25.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:25.809+0000 7f25c0112640 1 -- 172.21.15.3:0/2741457738 shutdown_connections 2024-05-09T06:37:25.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:25.809+0000 7f25c0112640 1 --2- 172.21.15.3:0/2741457738 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f25b80ccf80 0x7f25b80cf720 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:25.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:25.809+0000 7f25c0112640 1 -- 172.21.15.3:0/2741457738 >> 172.21.15.3:0/2741457738 conn(0x7f25b806b430 msgr2=0x7f25b806b810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:25.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:25.809+0000 7f25c0112640 1 -- 172.21.15.3:0/2741457738 shutdown_connections 2024-05-09T06:37:25.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:25.809+0000 7f25c0112640 1 -- 172.21.15.3:0/2741457738 wait complete. 2024-05-09T06:37:25.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:25.810+0000 7f25c0112640 1 Processor -- start 2024-05-09T06:37:25.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:25.810+0000 7f25c0112640 1 -- start start 2024-05-09T06:37:25.795 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:25.811+0000 7f25c0112640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f25b80ccf80 0x7f25b813d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:25.795 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:25.811+0000 7f25c0112640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f25b813e0c0 0x7f25b815a020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:25.795 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:25.811+0000 7f25c0112640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f25b81408f0 0x7f25b80b0510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:25.795 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:25.811+0000 7f25c0112640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f25b8141660 con 0x7f25b813e0c0 2024-05-09T06:37:25.795 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:25.811+0000 7f25c0112640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f25b80cf200 con 0x7f25b80ccf80 2024-05-09T06:37:25.796 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:25.811+0000 7f25c0112640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f25b8071080 con 0x7f25b81408f0 2024-05-09T06:37:25.796 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:25.811+0000 7f25bd6ad640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f25b813e0c0 0x7f25b815a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:25.796 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:25.811+0000 7f25bd6ad640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f25b813e0c0 0x7f25b815a020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40656/0 (socket says 172.21.15.3:40656) 2024-05-09T06:37:25.796 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:25.811+0000 7f25bd6ad640 1 -- 172.21.15.3:0/125314316 learned_addr learned my addr 172.21.15.3:0/125314316 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:37:25.796 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:25.811+0000 7f25be6af640 1 --2- 172.21.15.3:0/125314316 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f25b81408f0 0x7f25b80b0510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:25.796 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:25.811+0000 7f25bdeae640 1 --2- 172.21.15.3:0/125314316 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f25b80ccf80 0x7f25b813d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:25.796 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:25.811+0000 7f25bd6ad640 1 -- 172.21.15.3:0/125314316 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f25b81408f0 msgr2=0x7f25b80b0510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:25.796 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:25.811+0000 7f25bd6ad640 1 --2- 172.21.15.3:0/125314316 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f25b81408f0 0x7f25b80b0510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:25.796 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:25.811+0000 7f25bd6ad640 1 -- 172.21.15.3:0/125314316 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f25b80ccf80 msgr2=0x7f25b813d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:25.796 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:25.811+0000 7f25bd6ad640 1 --2- 172.21.15.3:0/125314316 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f25b80ccf80 0x7f25b813d6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:25.796 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:25.811+0000 7f25bd6ad640 1 -- 172.21.15.3:0/125314316 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25b8158f80 con 0x7f25b813e0c0 2024-05-09T06:37:25.796 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:25.811+0000 7f25bdeae640 1 --2- 172.21.15.3:0/125314316 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f25b80ccf80 0x7f25b813d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:37:25.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:25.812+0000 7f25bd6ad640 1 --2- 172.21.15.3:0/125314316 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f25b813e0c0 0x7f25b815a020 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f25a80098c0 tx=0x7f25a80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:25.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:25.812+0000 7f25ae7fc640 1 -- 172.21.15.3:0/125314316 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25a8037040 con 0x7f25b813e0c0 2024-05-09T06:37:25.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:25.812+0000 7f25ae7fc640 1 -- 172.21.15.3:0/125314316 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f25a8030770 con 0x7f25b813e0c0 2024-05-09T06:37:25.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:25.812+0000 7f25c0112640 1 -- 172.21.15.3:0/125314316 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25b8159260 con 0x7f25b813e0c0 2024-05-09T06:37:25.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:25.812+0000 7f25ae7fc640 1 -- 172.21.15.3:0/125314316 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25a8030b40 con 0x7f25b813e0c0 2024-05-09T06:37:25.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:25.812+0000 7f25c0112640 1 -- 172.21.15.3:0/125314316 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25b8159820 con 0x7f25b813e0c0 2024-05-09T06:37:25.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:25.813+0000 7f25c0112640 1 -- 172.21.15.3:0/125314316 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25b804f660 con 0x7f25b813e0c0 2024-05-09T06:37:25.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:25.813+0000 7f25ae7fc640 1 -- 172.21.15.3:0/125314316 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 7) ==== 90651+0+0 (secure 0 0 0) 0x7f25a8044020 con 0x7f25b813e0c0 2024-05-09T06:37:25.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:25.814+0000 7f25ae7fc640 1 --2- 172.21.15.3:0/125314316 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f259c06bfe0 0x7f259c06e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:25.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:25.814+0000 7f25ae7fc640 1 -- 172.21.15.3:0/125314316 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 4943+0+0 (secure 0 0 0) 0x7f25a80ab4c0 con 0x7f25b813e0c0 2024-05-09T06:37:25.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:25.814+0000 7f25bdeae640 1 --2- 172.21.15.3:0/125314316 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f259c06bfe0 0x7f259c06e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:25.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:25.815+0000 7f25bdeae640 1 --2- 172.21.15.3:0/125314316 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f259c06bfe0 0x7f259c06e4a0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f25b4001680 tx=0x7f25b4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:25.799 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:25.818+0000 7f25ae7fc640 1 -- 172.21.15.3:0/125314316 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f25a8032e20 con 0x7f25b813e0c0 2024-05-09T06:37:25.980 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:25.999+0000 7f25c0112640 1 -- 172.21.15.3:0/125314316 --> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f25b80730a0 con 0x7f259c06bfe0 2024-05-09T06:37:25.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:26.000+0000 7f25ae7fc640 1 -- 172.21.15.3:0/125314316 <== mgr.4105 v2:172.21.15.190:6832/975893321 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13478 (secure 0 0 0) 0x7f25b80730a0 con 0x7f259c06bfe0 2024-05-09T06:37:25.981 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-09T06:37:25.981 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2024-05-09T06:37:25.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:26.002+0000 7f25c0112640 1 -- 172.21.15.3:0/125314316 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f259c06bfe0 msgr2=0x7f259c06e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:25.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:26.002+0000 7f25c0112640 1 --2- 172.21.15.3:0/125314316 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f259c06bfe0 0x7f259c06e4a0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f25b4001680 tx=0x7f25b4009040 comp rx=0 tx=0).stop 2024-05-09T06:37:25.984 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:26.002+0000 7f25c0112640 1 -- 172.21.15.3:0/125314316 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f25b813e0c0 msgr2=0x7f25b815a020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:25.984 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:26.002+0000 7f25c0112640 1 --2- 172.21.15.3:0/125314316 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f25b813e0c0 0x7f25b815a020 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f25a80098c0 tx=0x7f25a80046c0 comp rx=0 tx=0).stop 2024-05-09T06:37:25.984 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:26.003+0000 7f25c0112640 1 -- 172.21.15.3:0/125314316 shutdown_connections 2024-05-09T06:37:25.984 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:26.003+0000 7f25c0112640 1 --2- 172.21.15.3:0/125314316 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f259c06bfe0 0x7f259c06e4a0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:25.984 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:26.003+0000 7f25c0112640 1 --2- 172.21.15.3:0/125314316 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f25b81408f0 0x7f25b80b0510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:25.984 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:26.003+0000 7f25c0112640 1 --2- 172.21.15.3:0/125314316 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f25b813e0c0 0x7f25b815a020 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:25.984 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:26.003+0000 7f25c0112640 1 --2- 172.21.15.3:0/125314316 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f25b80ccf80 0x7f25b813d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:25.984 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:26.003+0000 7f25c0112640 1 -- 172.21.15.3:0/125314316 >> 172.21.15.3:0/125314316 conn(0x7f25b806b430 msgr2=0x7f25b813eae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:25.984 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:26.003+0000 7f25c0112640 1 -- 172.21.15.3:0/125314316 shutdown_connections 2024-05-09T06:37:25.984 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:26.003+0000 7f25c0112640 1 -- 172.21.15.3:0/125314316 wait complete. 2024-05-09T06:37:25.994 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":75,"stamp":"2024-05-09T06:37:24.793018+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":135,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":220680,"kb_used_data":4508,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754040,"statfs":{"total":773094113280,"available":772868136960,"internally_reserved":0,"allocated":4616192,"data_stored":2611984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-129907,"num_objects":-23,"num_object_clones":0,"num_object_copies":-46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-23,"num_whiteouts":0,"num_read":-3,"num_read_kb":-3,"num_write":-97,"num_write_kb":-165,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.129516"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":198,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-09T06:37:22.578579+0000","last_change":"2024-05-09T06:35:55.895214+0000","last_active":"2024-05-09T06:37:22.578579+0000","last_peered":"2024-05-09T06:37:22.578579+0000","last_clean":"2024-05-09T06:37:22.578579+0000","last_became_active":"2024-05-09T06:35:55.894087+0000","last_became_peered":"2024-05-09T06:35:55.894087+0000","last_unstale":"2024-05-09T06:37:22.578579+0000","last_undegraded":"2024-05-09T06:37:22.578579+0000","last_fullsized":"2024-05-09T06:37:22.578579+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:35:54.820102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:35:54.820102+0000","last_clean_scrub_stamp":"2024-05-09T06:35:54.820102+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:29:20.408099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574870,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28012,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343828,"statfs":{"total":96636764160,"available":96608079872,"internally_reserved":0,"allocated":1011712,"data_stored":754633,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574870,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27432,"kb_used_data":408,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344408,"statfs":{"total":96636764160,"available":96608673792,"internally_reserved":0,"allocated":417792,"data_stored":164265,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574869,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27412,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344428,"statfs":{"total":96636764160,"available":96608694272,"internally_reserved":0,"allocated":401408,"data_stored":158700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574869,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27428,"kb_used_data":408,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344412,"statfs":{"total":96636764160,"available":96608677888,"internally_reserved":0,"allocated":417792,"data_stored":164265,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574870,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27544,"kb_used_data":524,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344296,"statfs":{"total":96636764160,"available":96608559104,"internally_reserved":0,"allocated":536576,"data_stored":292523,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574869,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27428,"kb_used_data":408,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344412,"statfs":{"total":96636764160,"available":96608677888,"internally_reserved":0,"allocated":417792,"data_stored":164265,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574869,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27412,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344428,"statfs":{"total":96636764160,"available":96608694272,"internally_reserved":0,"allocated":401408,"data_stored":158700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574870,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28012,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343828,"statfs":{"total":96636764160,"available":96608079872,"internally_reserved":0,"allocated":1011712,"data_stored":754633,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-09T06:37:25.995 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-09T06:37:26.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:26.118+0000 7fa85f83f640 1 Processor -- start 2024-05-09T06:37:26.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:26.118+0000 7fa85f83f640 1 -- start start 2024-05-09T06:37:26.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:26.119+0000 7fa85f83f640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa85813e0c0 0x7fa8580cf720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:26.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:26.119+0000 7fa85f83f640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa8581408f0 0x7fa858140d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:26.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:26.119+0000 7fa85f83f640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa85813e4a0 con 0x7fa8580cc870 2024-05-09T06:37:26.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:26.119+0000 7fa85f83f640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fa8580ce960 con 0x7fa8581408f0 2024-05-09T06:37:26.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:26.119+0000 7fa85f83f640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fa8580cec40 con 0x7fa85813e0c0 2024-05-09T06:37:26.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:26.119+0000 7fa85dddc640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa8581408f0 0x7fa858140d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:26.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:26.119+0000 7fa85dddc640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa8581408f0 0x7fa858140d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40666/0 (socket says 172.21.15.3:40666) 2024-05-09T06:37:26.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:26.119+0000 7fa85dddc640 1 -- 172.21.15.3:0/2735539976 learned_addr learned my addr 172.21.15.3:0/2735539976 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:37:26.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:26.119+0000 7fa85d5db640 1 --2- 172.21.15.3:0/2735539976 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa85813e0c0 0x7fa8580cf720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:26.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:26.119+0000 7fa85dddc640 1 -- 172.21.15.3:0/2735539976 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa85813e0c0 msgr2=0x7fa8580cf720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:37:26.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:26.119+0000 7fa85dddc640 1 --2- 172.21.15.3:0/2735539976 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa85813e0c0 0x7fa8580cf720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:26.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:26.119+0000 7fa85dddc640 1 -- 172.21.15.3:0/2735539976 >> v1:172.21.15.190:6789/0 conn(0x7fa8580cc870 legacy=0x7fa8580cccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:26.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:26.119+0000 7fa85dddc640 1 -- 172.21.15.3:0/2735539976 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8580cef20 con 0x7fa8581408f0 2024-05-09T06:37:26.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:26.119+0000 7fa85d5db640 1 --2- 172.21.15.3:0/2735539976 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa85813e0c0 0x7fa8580cf720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:37:26.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:26.120+0000 7fa85dddc640 1 --2- 172.21.15.3:0/2735539976 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa8581408f0 0x7fa858140d40 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fa850009e50 tx=0x7fa85002a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c42ca7cc8ec98054 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:26.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:26.120+0000 7fa84ffff640 1 -- 172.21.15.3:0/2735539976 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa850036070 con 0x7fa8581408f0 2024-05-09T06:37:26.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:26.120+0000 7fa84ffff640 1 -- 172.21.15.3:0/2735539976 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa850031600 con 0x7fa8581408f0 2024-05-09T06:37:26.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:26.120+0000 7fa84ffff640 1 -- 172.21.15.3:0/2735539976 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8500319d0 con 0x7fa8581408f0 2024-05-09T06:37:26.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:26.121+0000 7fa85f83f640 1 -- 172.21.15.3:0/2735539976 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa8581408f0 msgr2=0x7fa858140d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:26.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:26.121+0000 7fa85f83f640 1 --2- 172.21.15.3:0/2735539976 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa8581408f0 0x7fa858140d40 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fa850009e50 tx=0x7fa85002a710 comp rx=0 tx=0).stop 2024-05-09T06:37:26.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:26.121+0000 7fa85f83f640 1 -- 172.21.15.3:0/2735539976 shutdown_connections 2024-05-09T06:37:26.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:26.121+0000 7fa85f83f640 1 --2- 172.21.15.3:0/2735539976 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa8581408f0 0x7fa858140d40 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:26.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:26.121+0000 7fa85f83f640 1 --2- 172.21.15.3:0/2735539976 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa85813e0c0 0x7fa8580cf720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:26.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:26.121+0000 7fa85f83f640 1 -- 172.21.15.3:0/2735539976 >> 172.21.15.3:0/2735539976 conn(0x7fa85806b430 msgr2=0x7fa85806b810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:26.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:26.121+0000 7fa85f83f640 1 -- 172.21.15.3:0/2735539976 shutdown_connections 2024-05-09T06:37:26.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:26.121+0000 7fa85f83f640 1 -- 172.21.15.3:0/2735539976 wait complete. 2024-05-09T06:37:26.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:26.122+0000 7fa85f83f640 1 Processor -- start 2024-05-09T06:37:26.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:26.122+0000 7fa85f83f640 1 -- start start 2024-05-09T06:37:26.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:26.123+0000 7fa85f83f640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa8580cc870 0x7fa8580b2480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:26.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:26.123+0000 7fa85f83f640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa85813e0c0 0x7fa85814ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:26.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:26.123+0000 7fa85f83f640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa8581408f0 0x7fa85814f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:26.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:26.123+0000 7fa85f83f640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fa858141280 con 0x7fa85813e0c0 2024-05-09T06:37:26.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:26.123+0000 7fa85f83f640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fa858141660 con 0x7fa8580cc870 2024-05-09T06:37:26.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:26.123+0000 7fa85f83f640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fa858141400 con 0x7fa8581408f0 2024-05-09T06:37:26.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:26.123+0000 7fa85cdda640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa85813e0c0 0x7fa85814ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:26.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:26.123+0000 7fa85cdda640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa85813e0c0 0x7fa85814ebe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40680/0 (socket says 172.21.15.3:40680) 2024-05-09T06:37:26.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:26.123+0000 7fa85cdda640 1 -- 172.21.15.3:0/1832207942 learned_addr learned my addr 172.21.15.3:0/1832207942 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:37:26.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:26.123+0000 7fa85dddc640 1 --2- 172.21.15.3:0/1832207942 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa8581408f0 0x7fa85814f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:26.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:26.123+0000 7fa85d5db640 1 --2- 172.21.15.3:0/1832207942 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa8580cc870 0x7fa8580b2480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:26.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:26.123+0000 7fa85cdda640 1 -- 172.21.15.3:0/1832207942 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa8581408f0 msgr2=0x7fa85814f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:26.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:26.123+0000 7fa85cdda640 1 --2- 172.21.15.3:0/1832207942 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa8581408f0 0x7fa85814f120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:26.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:26.123+0000 7fa85cdda640 1 -- 172.21.15.3:0/1832207942 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa8580cc870 msgr2=0x7fa8580b2480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:26.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:26.123+0000 7fa85cdda640 1 --2- 172.21.15.3:0/1832207942 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa8580cc870 0x7fa8580b2480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:26.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:26.123+0000 7fa85cdda640 1 -- 172.21.15.3:0/1832207942 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8580b29c0 con 0x7fa85813e0c0 2024-05-09T06:37:26.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:26.123+0000 7fa85cdda640 1 --2- 172.21.15.3:0/1832207942 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa85813e0c0 0x7fa85814ebe0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fa84800c430 tx=0x7fa84800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:26.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:26.123+0000 7fa84dffb640 1 -- 172.21.15.3:0/1832207942 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8480052d0 con 0x7fa85813e0c0 2024-05-09T06:37:26.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:26.123+0000 7fa84dffb640 1 -- 172.21.15.3:0/1832207942 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa848005490 con 0x7fa85813e0c0 2024-05-09T06:37:26.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:26.123+0000 7fa85f83f640 1 -- 172.21.15.3:0/1832207942 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8581507a0 con 0x7fa85813e0c0 2024-05-09T06:37:26.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:26.123+0000 7fa84dffb640 1 -- 172.21.15.3:0/1832207942 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa848007260 con 0x7fa85813e0c0 2024-05-09T06:37:26.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:26.124+0000 7fa85f83f640 1 -- 172.21.15.3:0/1832207942 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa858150d60 con 0x7fa85813e0c0 2024-05-09T06:37:26.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:26.125+0000 7fa85f83f640 1 -- 172.21.15.3:0/1832207942 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa85804f660 con 0x7fa85813e0c0 2024-05-09T06:37:26.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:26.125+0000 7fa84dffb640 1 -- 172.21.15.3:0/1832207942 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 7) ==== 90651+0+0 (secure 0 0 0) 0x7fa848007400 con 0x7fa85813e0c0 2024-05-09T06:37:26.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:26.126+0000 7fa84dffb640 1 --2- 172.21.15.3:0/1832207942 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fa83c06c010 0x7fa83c06e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:26.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:26.126+0000 7fa84dffb640 1 -- 172.21.15.3:0/1832207942 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 4943+0+0 (secure 0 0 0) 0x7fa84808a250 con 0x7fa85813e0c0 2024-05-09T06:37:26.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:26.126+0000 7fa85d5db640 1 --2- 172.21.15.3:0/1832207942 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fa83c06c010 0x7fa83c06e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:26.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:26.126+0000 7fa85d5db640 1 --2- 172.21.15.3:0/1832207942 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fa83c06c010 0x7fa83c06e4d0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fa854006e80 tx=0x7fa854008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:26.111 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:26.130+0000 7fa84dffb640 1 -- 172.21.15.3:0/1832207942 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fa85813e4a0 con 0x7fa85813e0c0 2024-05-09T06:37:26.290 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:26.309+0000 7fa85f83f640 1 -- 172.21.15.3:0/1832207942 --> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa85806e9c0 con 0x7fa83c06c010 2024-05-09T06:37:26.290 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:26.311+0000 7fa84dffb640 1 -- 172.21.15.3:0/1832207942 <== mgr.4105 v2:172.21.15.190:6832/975893321 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13478 (secure 0 0 0) 0x7fa85806e9c0 con 0x7fa83c06c010 2024-05-09T06:37:26.291 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-09T06:37:26.291 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2024-05-09T06:37:26.293 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:26.312+0000 7fa85f83f640 1 -- 172.21.15.3:0/1832207942 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fa83c06c010 msgr2=0x7fa83c06e4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:26.293 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:26.312+0000 7fa85f83f640 1 --2- 172.21.15.3:0/1832207942 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fa83c06c010 0x7fa83c06e4d0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fa854006e80 tx=0x7fa854008040 comp rx=0 tx=0).stop 2024-05-09T06:37:26.293 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:26.312+0000 7fa85f83f640 1 -- 172.21.15.3:0/1832207942 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa85813e0c0 msgr2=0x7fa85814ebe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:26.293 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:26.312+0000 7fa85f83f640 1 --2- 172.21.15.3:0/1832207942 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa85813e0c0 0x7fa85814ebe0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fa84800c430 tx=0x7fa84800c8f0 comp rx=0 tx=0).stop 2024-05-09T06:37:26.293 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:26.312+0000 7fa85f83f640 1 -- 172.21.15.3:0/1832207942 shutdown_connections 2024-05-09T06:37:26.293 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:26.312+0000 7fa85f83f640 1 --2- 172.21.15.3:0/1832207942 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fa83c06c010 0x7fa83c06e4d0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:26.294 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:26.312+0000 7fa85f83f640 1 --2- 172.21.15.3:0/1832207942 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa8581408f0 0x7fa85814f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:26.294 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:26.312+0000 7fa85f83f640 1 --2- 172.21.15.3:0/1832207942 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa85813e0c0 0x7fa85814ebe0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:26.294 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:26.312+0000 7fa85f83f640 1 --2- 172.21.15.3:0/1832207942 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa8580cc870 0x7fa8580b2480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:26.294 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:26.312+0000 7fa85f83f640 1 -- 172.21.15.3:0/1832207942 >> 172.21.15.3:0/1832207942 conn(0x7fa85806b430 msgr2=0x7fa85806dcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:26.294 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:26.312+0000 7fa85f83f640 1 -- 172.21.15.3:0/1832207942 shutdown_connections 2024-05-09T06:37:26.294 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:26.312+0000 7fa85f83f640 1 -- 172.21.15.3:0/1832207942 wait complete. 2024-05-09T06:37:26.303 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":75,"stamp":"2024-05-09T06:37:24.793018+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":135,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":220680,"kb_used_data":4508,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754040,"statfs":{"total":773094113280,"available":772868136960,"internally_reserved":0,"allocated":4616192,"data_stored":2611984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-129907,"num_objects":-23,"num_object_clones":0,"num_object_copies":-46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-23,"num_whiteouts":0,"num_read":-3,"num_read_kb":-3,"num_write":-97,"num_write_kb":-165,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.129516"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":198,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-09T06:37:22.578579+0000","last_change":"2024-05-09T06:35:55.895214+0000","last_active":"2024-05-09T06:37:22.578579+0000","last_peered":"2024-05-09T06:37:22.578579+0000","last_clean":"2024-05-09T06:37:22.578579+0000","last_became_active":"2024-05-09T06:35:55.894087+0000","last_became_peered":"2024-05-09T06:35:55.894087+0000","last_unstale":"2024-05-09T06:37:22.578579+0000","last_undegraded":"2024-05-09T06:37:22.578579+0000","last_fullsized":"2024-05-09T06:37:22.578579+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:35:54.820102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:35:54.820102+0000","last_clean_scrub_stamp":"2024-05-09T06:35:54.820102+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:29:20.408099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574870,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28012,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343828,"statfs":{"total":96636764160,"available":96608079872,"internally_reserved":0,"allocated":1011712,"data_stored":754633,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574870,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27432,"kb_used_data":408,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344408,"statfs":{"total":96636764160,"available":96608673792,"internally_reserved":0,"allocated":417792,"data_stored":164265,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574869,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27412,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344428,"statfs":{"total":96636764160,"available":96608694272,"internally_reserved":0,"allocated":401408,"data_stored":158700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574869,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27428,"kb_used_data":408,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344412,"statfs":{"total":96636764160,"available":96608677888,"internally_reserved":0,"allocated":417792,"data_stored":164265,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574870,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27544,"kb_used_data":524,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344296,"statfs":{"total":96636764160,"available":96608559104,"internally_reserved":0,"allocated":536576,"data_stored":292523,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574869,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27428,"kb_used_data":408,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344412,"statfs":{"total":96636764160,"available":96608677888,"internally_reserved":0,"allocated":417792,"data_stored":164265,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574869,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27412,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344428,"statfs":{"total":96636764160,"available":96608694272,"internally_reserved":0,"allocated":401408,"data_stored":158700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574870,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28012,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343828,"statfs":{"total":96636764160,"available":96608079872,"internally_reserved":0,"allocated":1011712,"data_stored":754633,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-09T06:37:26.304 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-05-09T06:37:26.304 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-09T06:37:26.408 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:26.428+0000 7f9d12f89640 1 Processor -- start 2024-05-09T06:37:26.409 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:26.428+0000 7f9d12f89640 1 -- start start 2024-05-09T06:37:26.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:26.429+0000 7f9d12f89640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9d0c0ccf80 0x7f9d0c0cf720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:26.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:26.429+0000 7f9d12f89640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f9d0c1408f0 0x7f9d0c140d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:26.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:26.429+0000 7f9d12f89640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9d0c13e4a0 con 0x7f9d0c13e0c0 2024-05-09T06:37:26.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:26.429+0000 7f9d12f89640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f9d0c0ce960 con 0x7f9d0c1408f0 2024-05-09T06:37:26.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:26.429+0000 7f9d12f89640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f9d0c0cec40 con 0x7f9d0c0ccf80 2024-05-09T06:37:26.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:26.429+0000 7f9d11526640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f9d0c1408f0 0x7f9d0c140d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:26.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:26.429+0000 7f9d11526640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f9d0c1408f0 0x7f9d0c140d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40688/0 (socket says 172.21.15.3:40688) 2024-05-09T06:37:26.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:26.429+0000 7f9d11526640 1 -- 172.21.15.3:0/1637235017 learned_addr learned my addr 172.21.15.3:0/1637235017 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:37:26.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:26.429+0000 7f9d0bfff640 1 --2- 172.21.15.3:0/1637235017 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9d0c0ccf80 0x7f9d0c0cf720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:26.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:26.429+0000 7f9d11526640 1 -- 172.21.15.3:0/1637235017 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9d0c0ccf80 msgr2=0x7f9d0c0cf720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:37:26.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:26.429+0000 7f9d11526640 1 --2- 172.21.15.3:0/1637235017 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9d0c0ccf80 0x7f9d0c0cf720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:26.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:26.429+0000 7f9d11526640 1 -- 172.21.15.3:0/1637235017 >> v1:172.21.15.190:6789/0 conn(0x7f9d0c13e0c0 legacy=0x7f9d0c0cc870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:26.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:26.429+0000 7f9d11526640 1 -- 172.21.15.3:0/1637235017 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d0c0cef20 con 0x7f9d0c1408f0 2024-05-09T06:37:26.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:26.430+0000 7f9d11526640 1 --2- 172.21.15.3:0/1637235017 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9d0c1408f0 0x7f9d0c140d40 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f9d0400d150 tx=0x7f9d0402c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cceb36c3990d27a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:26.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:26.430+0000 7f9d0b7fe640 1 -- 172.21.15.3:0/1637235017 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d04009600 con 0x7f9d0c1408f0 2024-05-09T06:37:26.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:26.430+0000 7f9d0b7fe640 1 -- 172.21.15.3:0/1637235017 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d04037070 con 0x7f9d0c1408f0 2024-05-09T06:37:26.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:26.430+0000 7f9d0b7fe640 1 -- 172.21.15.3:0/1637235017 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d040099c0 con 0x7f9d0c1408f0 2024-05-09T06:37:26.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:26.430+0000 7f9d12f89640 1 -- 172.21.15.3:0/1637235017 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9d0c1408f0 msgr2=0x7f9d0c140d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:26.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:26.430+0000 7f9d12f89640 1 --2- 172.21.15.3:0/1637235017 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9d0c1408f0 0x7f9d0c140d40 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f9d0400d150 tx=0x7f9d0402c930 comp rx=0 tx=0).stop 2024-05-09T06:37:26.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:26.431+0000 7f9d12f89640 1 -- 172.21.15.3:0/1637235017 shutdown_connections 2024-05-09T06:37:26.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:26.431+0000 7f9d12f89640 1 --2- 172.21.15.3:0/1637235017 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9d0c1408f0 0x7f9d0c140d40 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:26.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:26.431+0000 7f9d12f89640 1 --2- 172.21.15.3:0/1637235017 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9d0c0ccf80 0x7f9d0c0cf720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:26.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:26.431+0000 7f9d12f89640 1 -- 172.21.15.3:0/1637235017 >> 172.21.15.3:0/1637235017 conn(0x7f9d0c06b430 msgr2=0x7f9d0c06b810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:26.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:26.431+0000 7f9d12f89640 1 -- 172.21.15.3:0/1637235017 shutdown_connections 2024-05-09T06:37:26.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:26.431+0000 7f9d12f89640 1 -- 172.21.15.3:0/1637235017 wait complete. 2024-05-09T06:37:26.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:26.432+0000 7f9d12f89640 1 Processor -- start 2024-05-09T06:37:26.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:26.432+0000 7f9d12f89640 1 -- start start 2024-05-09T06:37:26.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:26.433+0000 7f9d12f89640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9d0c0ccf80 0x7f9d0c137860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:26.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:26.433+0000 7f9d12f89640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9d0c13e0c0 0x7f9d0c15cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:26.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:26.433+0000 7f9d12f89640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9d0c1408f0 0x7f9d0c15d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:26.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:26.433+0000 7f9d12f89640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f9d0c141280 con 0x7f9d0c13e0c0 2024-05-09T06:37:26.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:26.433+0000 7f9d12f89640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f9d0c141660 con 0x7f9d0c0ccf80 2024-05-09T06:37:26.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:26.433+0000 7f9d12f89640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f9d0c141400 con 0x7f9d0c1408f0 2024-05-09T06:37:26.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:26.433+0000 7f9d0bfff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9d0c13e0c0 0x7f9d0c15cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:26.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:26.433+0000 7f9d0bfff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9d0c13e0c0 0x7f9d0c15cc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40704/0 (socket says 172.21.15.3:40704) 2024-05-09T06:37:26.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:26.433+0000 7f9d0bfff640 1 -- 172.21.15.3:0/4237463018 learned_addr learned my addr 172.21.15.3:0/4237463018 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:37:26.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:26.433+0000 7f9d10d25640 1 --2- 172.21.15.3:0/4237463018 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9d0c0ccf80 0x7f9d0c137860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:26.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:26.433+0000 7f9d11526640 1 --2- 172.21.15.3:0/4237463018 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9d0c1408f0 0x7f9d0c15d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:26.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:26.433+0000 7f9d0bfff640 1 -- 172.21.15.3:0/4237463018 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9d0c1408f0 msgr2=0x7f9d0c15d1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:26.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:26.433+0000 7f9d0bfff640 1 --2- 172.21.15.3:0/4237463018 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9d0c1408f0 0x7f9d0c15d1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:26.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:26.433+0000 7f9d0bfff640 1 -- 172.21.15.3:0/4237463018 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9d0c0ccf80 msgr2=0x7f9d0c137860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:26.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:26.433+0000 7f9d0bfff640 1 --2- 172.21.15.3:0/4237463018 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9d0c0ccf80 0x7f9d0c137860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:26.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:26.433+0000 7f9d0bfff640 1 -- 172.21.15.3:0/4237463018 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d0c15de30 con 0x7f9d0c13e0c0 2024-05-09T06:37:26.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:26.433+0000 7f9d10d25640 1 --2- 172.21.15.3:0/4237463018 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9d0c0ccf80 0x7f9d0c137860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:37:26.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:26.433+0000 7f9d11526640 1 --2- 172.21.15.3:0/4237463018 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9d0c1408f0 0x7f9d0c15d1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:37:26.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:26.433+0000 7f9d0bfff640 1 --2- 172.21.15.3:0/4237463018 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9d0c13e0c0 0x7f9d0c15cc80 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f9cf800e980 tx=0x7f9cf800ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:26.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:26.433+0000 7f9d097fa640 1 -- 172.21.15.3:0/4237463018 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cf8017070 con 0x7f9d0c13e0c0 2024-05-09T06:37:26.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:26.434+0000 7f9d097fa640 1 -- 172.21.15.3:0/4237463018 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9cf800ca00 con 0x7f9d0c13e0c0 2024-05-09T06:37:26.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:26.434+0000 7f9d12f89640 1 -- 172.21.15.3:0/4237463018 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d0c15e110 con 0x7f9d0c13e0c0 2024-05-09T06:37:26.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:26.434+0000 7f9d097fa640 1 -- 172.21.15.3:0/4237463018 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cf800cdd0 con 0x7f9d0c13e0c0 2024-05-09T06:37:26.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:26.434+0000 7f9d12f89640 1 -- 172.21.15.3:0/4237463018 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d0c15e3f0 con 0x7f9d0c13e0c0 2024-05-09T06:37:26.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:26.435+0000 7f9d12f89640 1 -- 172.21.15.3:0/4237463018 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d0c04f660 con 0x7f9d0c13e0c0 2024-05-09T06:37:26.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:26.435+0000 7f9d097fa640 1 -- 172.21.15.3:0/4237463018 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 7) ==== 90651+0+0 (secure 0 0 0) 0x7f9cf801a070 con 0x7f9d0c13e0c0 2024-05-09T06:37:26.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:26.436+0000 7f9d097fa640 1 --2- 172.21.15.3:0/4237463018 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f9cf006bfe0 0x7f9cf006e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:26.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:26.436+0000 7f9d097fa640 1 -- 172.21.15.3:0/4237463018 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 4943+0+0 (secure 0 0 0) 0x7f9cf808b200 con 0x7f9d0c13e0c0 2024-05-09T06:37:26.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:26.436+0000 7f9d10d25640 1 --2- 172.21.15.3:0/4237463018 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f9cf006bfe0 0x7f9cf006e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:26.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:26.436+0000 7f9d10d25640 1 --2- 172.21.15.3:0/4237463018 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f9cf006bfe0 0x7f9cf006e4a0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f9d00001700 tx=0x7f9d00009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:26.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:26.440+0000 7f9d097fa640 1 -- 172.21.15.3:0/4237463018 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f9cf8059cc0 con 0x7f9d0c13e0c0 2024-05-09T06:37:26.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:26.619+0000 7f9d12f89640 1 -- 172.21.15.3:0/4237463018 --> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f9d0c0730a0 con 0x7f9cf006bfe0 2024-05-09T06:37:26.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:26.621+0000 7f9d097fa640 1 -- 172.21.15.3:0/4237463018 <== mgr.4105 v2:172.21.15.190:6832/975893321 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13478 (secure 0 0 0) 0x7f9d0c0730a0 con 0x7f9cf006bfe0 2024-05-09T06:37:26.601 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-09T06:37:26.602 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2024-05-09T06:37:26.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:26.622+0000 7f9d12f89640 1 -- 172.21.15.3:0/4237463018 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f9cf006bfe0 msgr2=0x7f9cf006e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:26.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:26.622+0000 7f9d12f89640 1 --2- 172.21.15.3:0/4237463018 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f9cf006bfe0 0x7f9cf006e4a0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f9d00001700 tx=0x7f9d00009040 comp rx=0 tx=0).stop 2024-05-09T06:37:26.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:26.622+0000 7f9d12f89640 1 -- 172.21.15.3:0/4237463018 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9d0c13e0c0 msgr2=0x7f9d0c15cc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:26.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:26.622+0000 7f9d12f89640 1 --2- 172.21.15.3:0/4237463018 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9d0c13e0c0 0x7f9d0c15cc80 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f9cf800e980 tx=0x7f9cf800ee40 comp rx=0 tx=0).stop 2024-05-09T06:37:26.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:26.622+0000 7f9d12f89640 1 -- 172.21.15.3:0/4237463018 shutdown_connections 2024-05-09T06:37:26.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:26.622+0000 7f9d12f89640 1 --2- 172.21.15.3:0/4237463018 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f9cf006bfe0 0x7f9cf006e4a0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:26.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:26.622+0000 7f9d12f89640 1 --2- 172.21.15.3:0/4237463018 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9d0c1408f0 0x7f9d0c15d1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:26.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:26.622+0000 7f9d12f89640 1 --2- 172.21.15.3:0/4237463018 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9d0c13e0c0 0x7f9d0c15cc80 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:26.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:26.622+0000 7f9d12f89640 1 --2- 172.21.15.3:0/4237463018 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9d0c0ccf80 0x7f9d0c137860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:26.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:26.622+0000 7f9d12f89640 1 -- 172.21.15.3:0/4237463018 >> 172.21.15.3:0/4237463018 conn(0x7f9d0c06b430 msgr2=0x7f9d0c071080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:26.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:26.623+0000 7f9d12f89640 1 -- 172.21.15.3:0/4237463018 shutdown_connections 2024-05-09T06:37:26.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:26.623+0000 7f9d12f89640 1 -- 172.21.15.3:0/4237463018 wait complete. 2024-05-09T06:37:26.614 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":75,"stamp":"2024-05-09T06:37:24.793018+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":135,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":220680,"kb_used_data":4508,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754040,"statfs":{"total":773094113280,"available":772868136960,"internally_reserved":0,"allocated":4616192,"data_stored":2611984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-129907,"num_objects":-23,"num_object_clones":0,"num_object_copies":-46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-23,"num_whiteouts":0,"num_read":-3,"num_read_kb":-3,"num_write":-97,"num_write_kb":-165,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.129516"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":198,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-09T06:37:22.578579+0000","last_change":"2024-05-09T06:35:55.895214+0000","last_active":"2024-05-09T06:37:22.578579+0000","last_peered":"2024-05-09T06:37:22.578579+0000","last_clean":"2024-05-09T06:37:22.578579+0000","last_became_active":"2024-05-09T06:35:55.894087+0000","last_became_peered":"2024-05-09T06:35:55.894087+0000","last_unstale":"2024-05-09T06:37:22.578579+0000","last_undegraded":"2024-05-09T06:37:22.578579+0000","last_fullsized":"2024-05-09T06:37:22.578579+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:35:54.820102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:35:54.820102+0000","last_clean_scrub_stamp":"2024-05-09T06:35:54.820102+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:29:20.408099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574870,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28012,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343828,"statfs":{"total":96636764160,"available":96608079872,"internally_reserved":0,"allocated":1011712,"data_stored":754633,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574870,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27432,"kb_used_data":408,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344408,"statfs":{"total":96636764160,"available":96608673792,"internally_reserved":0,"allocated":417792,"data_stored":164265,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574869,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27412,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344428,"statfs":{"total":96636764160,"available":96608694272,"internally_reserved":0,"allocated":401408,"data_stored":158700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574869,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27428,"kb_used_data":408,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344412,"statfs":{"total":96636764160,"available":96608677888,"internally_reserved":0,"allocated":417792,"data_stored":164265,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574870,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27544,"kb_used_data":524,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344296,"statfs":{"total":96636764160,"available":96608559104,"internally_reserved":0,"allocated":536576,"data_stored":292523,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574869,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27428,"kb_used_data":408,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344412,"statfs":{"total":96636764160,"available":96608677888,"internally_reserved":0,"allocated":417792,"data_stored":164265,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574869,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27412,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344428,"statfs":{"total":96636764160,"available":96608694272,"internally_reserved":0,"allocated":401408,"data_stored":158700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574870,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28012,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343828,"statfs":{"total":96636764160,"available":96608079872,"internally_reserved":0,"allocated":1011712,"data_stored":754633,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-09T06:37:26.615 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T06:37:26.720 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:26.739+0000 7ff909407640 1 Processor -- start 2024-05-09T06:37:26.720 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:26.739+0000 7ff909407640 1 -- start start 2024-05-09T06:37:26.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:26.740+0000 7ff909407640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff90415e420 0x7ff90406b250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:26.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:26.740+0000 7ff909407640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff9040ae600 con 0x7ff9040b4d50 2024-05-09T06:37:26.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:26.740+0000 7ff909407640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff9040b5290 con 0x7ff90406b790 2024-05-09T06:37:26.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:26.740+0000 7ff909407640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7ff90415e800 con 0x7ff90415e420 2024-05-09T06:37:26.731 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:26.740+0000 7ff902ffd640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff90415e420 0x7ff90406b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:26.731 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:26.740+0000 7ff902ffd640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff90415e420 0x7ff90406b250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40710/0 (socket says 172.21.15.3:40710) 2024-05-09T06:37:26.731 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:26.740+0000 7ff902ffd640 1 -- 172.21.15.3:0/4173759618 learned_addr learned my addr 172.21.15.3:0/4173759618 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:37:26.731 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:26.740+0000 7ff9037fe640 1 --1- 172.21.15.3:0/4173759618 >> v1:172.21.15.3:6789/0 conn(0x7ff9040b4d50 0x7ff9040aeae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:43110/0 (socket says 172.21.15.3:43110) 2024-05-09T06:37:26.731 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:26.740+0000 7ff902ffd640 1 -- 172.21.15.3:0/4173759618 >> v1:172.21.15.190:6789/0 conn(0x7ff90406b790 legacy=0x7ff9040b4640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:37:26.731 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:26.740+0000 7ff902ffd640 1 -- 172.21.15.3:0/4173759618 >> v1:172.21.15.3:6789/0 conn(0x7ff9040b4d50 legacy=0x7ff9040aeae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:26.731 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:26.740+0000 7ff901ffb640 1 -- 172.21.15.3:0/4173759618 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1267194476 0 0) 0x7ff9040ae600 con 0x7ff9040b4d50 2024-05-09T06:37:26.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:26.740+0000 7ff902ffd640 1 -- 172.21.15.3:0/4173759618 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9040b5570 con 0x7ff90415e420 2024-05-09T06:37:26.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:26.741+0000 7ff902ffd640 1 --2- 172.21.15.3:0/4173759618 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff90415e420 0x7ff90406b250 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7ff8fc009e50 tx=0x7ff8fc02a6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=218abac82faefb84 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:26.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:26.741+0000 7ff901ffb640 1 -- 172.21.15.3:0/4173759618 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8fc036070 con 0x7ff90415e420 2024-05-09T06:37:26.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:26.741+0000 7ff901ffb640 1 -- 172.21.15.3:0/4173759618 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff8fc031600 con 0x7ff90415e420 2024-05-09T06:37:26.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:26.741+0000 7ff901ffb640 1 -- 172.21.15.3:0/4173759618 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8fc0319d0 con 0x7ff90415e420 2024-05-09T06:37:26.733 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:26.742+0000 7ff909407640 1 -- 172.21.15.3:0/4173759618 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff90415e420 msgr2=0x7ff90406b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:26.733 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:26.742+0000 7ff909407640 1 --2- 172.21.15.3:0/4173759618 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff90415e420 0x7ff90406b250 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7ff8fc009e50 tx=0x7ff8fc02a6d0 comp rx=0 tx=0).stop 2024-05-09T06:37:26.733 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:26.742+0000 7ff909407640 1 -- 172.21.15.3:0/4173759618 shutdown_connections 2024-05-09T06:37:26.733 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:26.742+0000 7ff909407640 1 --2- 172.21.15.3:0/4173759618 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff90415e420 0x7ff90406b250 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:26.733 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:26.742+0000 7ff909407640 1 -- 172.21.15.3:0/4173759618 >> 172.21.15.3:0/4173759618 conn(0x7ff90406e3d0 msgr2=0x7ff90406e7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:26.733 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:26.742+0000 7ff909407640 1 -- 172.21.15.3:0/4173759618 shutdown_connections 2024-05-09T06:37:26.733 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:26.742+0000 7ff909407640 1 -- 172.21.15.3:0/4173759618 wait complete. 2024-05-09T06:37:26.733 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:26.743+0000 7ff909407640 1 Processor -- start 2024-05-09T06:37:26.733 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:26.743+0000 7ff909407640 1 -- start start 2024-05-09T06:37:26.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:26.744+0000 7ff909407640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff90406b790 0x7ff904152710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:26.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:26.744+0000 7ff909407640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff9040b4d50 0x7ff904152c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:26.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:26.744+0000 7ff909407640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff90415e420 0x7ff9041f3c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:26.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:26.744+0000 7ff909407640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7ff9040af1f0 con 0x7ff9040b4d50 2024-05-09T06:37:26.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:26.744+0000 7ff909407640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7ff9040b5b30 con 0x7ff90406b790 2024-05-09T06:37:26.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:26.744+0000 7ff909407640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7ff9040af370 con 0x7ff90415e420 2024-05-09T06:37:26.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:26.744+0000 7ff9027fc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff9040b4d50 0x7ff904152c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:26.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:26.744+0000 7ff9027fc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff9040b4d50 0x7ff904152c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40720/0 (socket says 172.21.15.3:40720) 2024-05-09T06:37:26.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:26.744+0000 7ff9027fc640 1 -- 172.21.15.3:0/1313938429 learned_addr learned my addr 172.21.15.3:0/1313938429 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:37:26.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:26.744+0000 7ff902ffd640 1 --2- 172.21.15.3:0/1313938429 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff90406b790 0x7ff904152710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:26.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:26.744+0000 7ff9037fe640 1 --2- 172.21.15.3:0/1313938429 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff90415e420 0x7ff9041f3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:26.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:26.744+0000 7ff9027fc640 1 -- 172.21.15.3:0/1313938429 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff90415e420 msgr2=0x7ff9041f3c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:26.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:26.744+0000 7ff9027fc640 1 --2- 172.21.15.3:0/1313938429 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff90415e420 0x7ff9041f3c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:26.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:26.744+0000 7ff9027fc640 1 -- 172.21.15.3:0/1313938429 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff90406b790 msgr2=0x7ff904152710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:26.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:26.744+0000 7ff9027fc640 1 --2- 172.21.15.3:0/1313938429 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff90406b790 0x7ff904152710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:26.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:26.744+0000 7ff9027fc640 1 -- 172.21.15.3:0/1313938429 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff904153390 con 0x7ff9040b4d50 2024-05-09T06:37:26.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:26.745+0000 7ff9027fc640 1 --2- 172.21.15.3:0/1313938429 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff9040b4d50 0x7ff904152c50 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7ff8ec00c460 tx=0x7ff8ec00c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:26.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:26.745+0000 7ff8fb7fe640 1 -- 172.21.15.3:0/1313938429 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8ec016020 con 0x7ff9040b4d50 2024-05-09T06:37:26.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:26.745+0000 7ff8fb7fe640 1 -- 172.21.15.3:0/1313938429 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff8ec012870 con 0x7ff9040b4d50 2024-05-09T06:37:26.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:26.745+0000 7ff909407640 1 -- 172.21.15.3:0/1313938429 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff904153670 con 0x7ff9040b4d50 2024-05-09T06:37:26.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:26.745+0000 7ff8fb7fe640 1 -- 172.21.15.3:0/1313938429 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8ec012c60 con 0x7ff9040b4d50 2024-05-09T06:37:26.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:26.745+0000 7ff909407640 1 -- 172.21.15.3:0/1313938429 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff90415ed00 con 0x7ff9040b4d50 2024-05-09T06:37:26.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:26.746+0000 7ff909407640 1 -- 172.21.15.3:0/1313938429 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff90404f660 con 0x7ff9040b4d50 2024-05-09T06:37:26.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:26.746+0000 7ff8fb7fe640 1 -- 172.21.15.3:0/1313938429 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 7) ==== 90651+0+0 (secure 0 0 0) 0x7ff8ec019020 con 0x7ff9040b4d50 2024-05-09T06:37:26.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:26.747+0000 7ff8fb7fe640 1 --2- 172.21.15.3:0/1313938429 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7ff8e406bfe0 0x7ff8e406e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:26.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:26.747+0000 7ff8fb7fe640 1 -- 172.21.15.3:0/1313938429 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 4943+0+0 (secure 0 0 0) 0x7ff8ec08ab20 con 0x7ff9040b4d50 2024-05-09T06:37:26.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:26.747+0000 7ff902ffd640 1 --2- 172.21.15.3:0/1313938429 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7ff8e406bfe0 0x7ff8e406e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:26.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:26.747+0000 7ff902ffd640 1 --2- 172.21.15.3:0/1313938429 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7ff8e406bfe0 0x7ff8e406e4a0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7ff8fc033000 tx=0x7ff8fc009000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:26.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:26.751+0000 7ff8fb7fe640 1 -- 172.21.15.3:0/1313938429 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7ff8ec020030 con 0x7ff9040b4d50 2024-05-09T06:37:26.915 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:26.934+0000 7ff909407640 1 -- 172.21.15.3:0/1313938429 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff9040b0f20 con 0x7ff9040b4d50 2024-05-09T06:37:26.915 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:26.935+0000 7ff8fb7fe640 1 -- 172.21.15.3:0/1313938429 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v36) ==== 74+0+12934 (secure 0 0 0) 0x7ff8ec058e60 con 0x7ff9040b4d50 2024-05-09T06:37:26.916 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-09T06:37:26.916 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":36,"fsid":"16ed737d-2665-431a-8ac6-ef242be3975a","created":"2024-05-09T06:35:42.541368+0000","modified":"2024-05-09T06:37:21.671473+0000","last_up_change":"2024-05-09T06:35:53.778274+0000","last_in_change":"2024-05-09T06:35:44.780586+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T06:35:54.819999+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fae3a8db-4826-4f3c-b33a-12c09bb5df6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":1897849827},{"type":"v1","addr":"172.21.15.3:6817","nonce":1897849827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":1897849827},{"type":"v1","addr":"172.21.15.3:6819","nonce":1897849827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":1897849827},{"type":"v1","addr":"172.21.15.3:6823","nonce":1897849827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":1897849827},{"type":"v1","addr":"172.21.15.3:6821","nonce":1897849827}]},"public_addr":"172.21.15.3:6817/1897849827","cluster_addr":"172.21.15.3:6819/1897849827","heartbeat_back_addr":"172.21.15.3:6823/1897849827","heartbeat_front_addr":"172.21.15.3:6821/1897849827","state":["exists","up"]},{"osd":1,"uuid":"646e882d-4ce4-4720-8c84-f71df563a35e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":926805498},{"type":"v1","addr":"172.21.15.3:6801","nonce":926805498}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":926805498},{"type":"v1","addr":"172.21.15.3:6803","nonce":926805498}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":926805498},{"type":"v1","addr":"172.21.15.3:6807","nonce":926805498}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":926805498},{"type":"v1","addr":"172.21.15.3:6805","nonce":926805498}]},"public_addr":"172.21.15.3:6801/926805498","cluster_addr":"172.21.15.3:6803/926805498","heartbeat_back_addr":"172.21.15.3:6807/926805498","heartbeat_front_addr":"172.21.15.3:6805/926805498","state":["exists","up"]},{"osd":2,"uuid":"1abbadb5-a570-4359-9643-acff3ede35a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":801295451},{"type":"v1","addr":"172.21.15.3:6809","nonce":801295451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":801295451},{"type":"v1","addr":"172.21.15.3:6811","nonce":801295451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":801295451},{"type":"v1","addr":"172.21.15.3:6815","nonce":801295451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":801295451},{"type":"v1","addr":"172.21.15.3:6813","nonce":801295451}]},"public_addr":"172.21.15.3:6809/801295451","cluster_addr":"172.21.15.3:6811/801295451","heartbeat_back_addr":"172.21.15.3:6815/801295451","heartbeat_front_addr":"172.21.15.3:6813/801295451","state":["exists","up"]},{"osd":3,"uuid":"62ac4dc1-2371-4dd2-acd3-e2dbb11aedf5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":1903389857},{"type":"v1","addr":"172.21.15.3:6825","nonce":1903389857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":1903389857},{"type":"v1","addr":"172.21.15.3:6827","nonce":1903389857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":1903389857},{"type":"v1","addr":"172.21.15.3:6831","nonce":1903389857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":1903389857},{"type":"v1","addr":"172.21.15.3:6829","nonce":1903389857}]},"public_addr":"172.21.15.3:6825/1903389857","cluster_addr":"172.21.15.3:6827/1903389857","heartbeat_back_addr":"172.21.15.3:6831/1903389857","heartbeat_front_addr":"172.21.15.3:6829/1903389857","state":["exists","up"]},{"osd":4,"uuid":"ae808398-8326-4d7c-9319-a2a0eafffb59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":2143624576},{"type":"v1","addr":"172.21.15.190:6801","nonce":2143624576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":2143624576},{"type":"v1","addr":"172.21.15.190:6803","nonce":2143624576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":2143624576},{"type":"v1","addr":"172.21.15.190:6807","nonce":2143624576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":2143624576},{"type":"v1","addr":"172.21.15.190:6805","nonce":2143624576}]},"public_addr":"172.21.15.190:6801/2143624576","cluster_addr":"172.21.15.190:6803/2143624576","heartbeat_back_addr":"172.21.15.190:6807/2143624576","heartbeat_front_addr":"172.21.15.190:6805/2143624576","state":["exists","up"]},{"osd":5,"uuid":"918adf50-c440-48ac-9a8d-4837562d7275","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":1766923126},{"type":"v1","addr":"172.21.15.190:6814","nonce":1766923126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":1766923126},{"type":"v1","addr":"172.21.15.190:6818","nonce":1766923126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":1766923126},{"type":"v1","addr":"172.21.15.190:6823","nonce":1766923126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":1766923126},{"type":"v1","addr":"172.21.15.190:6821","nonce":1766923126}]},"public_addr":"172.21.15.190:6814/1766923126","cluster_addr":"172.21.15.190:6818/1766923126","heartbeat_back_addr":"172.21.15.190:6823/1766923126","heartbeat_front_addr":"172.21.15.190:6821/1766923126","state":["exists","up"]},{"osd":6,"uuid":"ac2882bc-2a7c-463c-8f6c-76ff49828adc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":342414248},{"type":"v1","addr":"172.21.15.190:6825","nonce":342414248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":342414248},{"type":"v1","addr":"172.21.15.190:6827","nonce":342414248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":342414248},{"type":"v1","addr":"172.21.15.190:6831","nonce":342414248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":342414248},{"type":"v1","addr":"172.21.15.190:6829","nonce":342414248}]},"public_addr":"172.21.15.190:6825/342414248","cluster_addr":"172.21.15.190:6827/342414248","heartbeat_back_addr":"172.21.15.190:6831/342414248","heartbeat_front_addr":"172.21.15.190:6829/342414248","state":["exists","up"]},{"osd":7,"uuid":"dd4e767a-4730-4c70-8bd5-3f088140a5a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":537849891},{"type":"v1","addr":"172.21.15.190:6809","nonce":537849891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":537849891},{"type":"v1","addr":"172.21.15.190:6811","nonce":537849891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6817","nonce":537849891},{"type":"v1","addr":"172.21.15.190:6819","nonce":537849891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6813","nonce":537849891},{"type":"v1","addr":"172.21.15.190:6815","nonce":537849891}]},"public_addr":"172.21.15.190:6809/537849891","cluster_addr":"172.21.15.190:6811/537849891","heartbeat_back_addr":"172.21.15.190:6819/537849891","heartbeat_front_addr":"172.21.15.190:6815/537849891","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.889784+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.974002+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.973549+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.955396+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.928471+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.902418+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.958696+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T06:35:49.926474+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.3:6835/2138710335":"2024-05-10T06:37:18.660898+0000","172.21.15.3:6833/2138710335":"2024-05-10T06:37:18.660898+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T06:37:26.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:26.937+0000 7ff909407640 1 -- 172.21.15.3:0/1313938429 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7ff8e406bfe0 msgr2=0x7ff8e406e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:26.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:26.937+0000 7ff909407640 1 --2- 172.21.15.3:0/1313938429 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7ff8e406bfe0 0x7ff8e406e4a0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7ff8fc033000 tx=0x7ff8fc009000 comp rx=0 tx=0).stop 2024-05-09T06:37:26.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:26.937+0000 7ff909407640 1 -- 172.21.15.3:0/1313938429 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff9040b4d50 msgr2=0x7ff904152c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:26.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:26.937+0000 7ff909407640 1 --2- 172.21.15.3:0/1313938429 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff9040b4d50 0x7ff904152c50 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7ff8ec00c460 tx=0x7ff8ec00c920 comp rx=0 tx=0).stop 2024-05-09T06:37:26.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:26.937+0000 7ff909407640 1 -- 172.21.15.3:0/1313938429 shutdown_connections 2024-05-09T06:37:26.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:26.937+0000 7ff909407640 1 --2- 172.21.15.3:0/1313938429 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7ff8e406bfe0 0x7ff8e406e4a0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:26.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:26.937+0000 7ff909407640 1 --2- 172.21.15.3:0/1313938429 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff90415e420 0x7ff9041f3c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:26.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:26.937+0000 7ff909407640 1 --2- 172.21.15.3:0/1313938429 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff9040b4d50 0x7ff904152c50 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:26.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:26.937+0000 7ff909407640 1 --2- 172.21.15.3:0/1313938429 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff90406b790 0x7ff904152710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:26.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:26.937+0000 7ff909407640 1 -- 172.21.15.3:0/1313938429 >> 172.21.15.3:0/1313938429 conn(0x7ff90406e3d0 msgr2=0x7ff9040b1ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:26.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:26.937+0000 7ff909407640 1 -- 172.21.15.3:0/1313938429 shutdown_connections 2024-05-09T06:37:26.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:26.937+0000 7ff909407640 1 -- 172.21.15.3:0/1313938429 wait complete. 2024-05-09T06:37:27.930 INFO:tasks.ceph:Scrubbing osd.0 2024-05-09T06:37:27.930 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-05-09T06:37:28.052 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.071+0000 7f873cb4f640 1 Processor -- start 2024-05-09T06:37:28.052 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.071+0000 7f873cb4f640 1 -- start start 2024-05-09T06:37:28.053 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.072+0000 7f873cb4f640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f87381458b0 0x7f873815c690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:28.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.072+0000 7f873cb4f640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f87380b4410 0x7f87380cd7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:28.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.072+0000 7f873cb4f640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f87380b4890 con 0x7f87380cdcf0 2024-05-09T06:37:28.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.072+0000 7f873cb4f640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f8738145c90 con 0x7f87380b4410 2024-05-09T06:37:28.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.072+0000 7f873cb4f640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f87380b4b60 con 0x7f87381458b0 2024-05-09T06:37:28.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.072+0000 7f8735d9b640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f87380b4410 0x7f87380cd7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:28.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.072+0000 7f8736d9d640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f87380cdcf0 0x7f873815fc80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:43114/0 (socket says 172.21.15.3:43114) 2024-05-09T06:37:28.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.072+0000 7f8735d9b640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f87380b4410 0x7f87380cd7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40732/0 (socket says 172.21.15.3:40732) 2024-05-09T06:37:28.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.072+0000 7f8736d9d640 1 -- 172.21.15.3:0/2970096875 learned_addr learned my addr 172.21.15.3:0/2970096875 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-05-09T06:37:28.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.073+0000 7f873659c640 1 --2- 172.21.15.3:0/2970096875 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f87381458b0 0x7f873815c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:28.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.073+0000 7f8735d9b640 1 -- 172.21.15.3:0/2970096875 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f87381458b0 msgr2=0x7f873815c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:37:28.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.073+0000 7f8735d9b640 1 --2- 172.21.15.3:0/2970096875 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f87381458b0 0x7f873815c690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:28.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.073+0000 7f8735d9b640 1 -- 172.21.15.3:0/2970096875 >> v1:172.21.15.3:6789/0 conn(0x7f87380cdcf0 legacy=0x7f873815fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:28.055 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.073+0000 7f8735d9b640 1 -- 172.21.15.3:0/2970096875 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87380b4e40 con 0x7f87380b4410 2024-05-09T06:37:28.055 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.073+0000 7f873559a640 1 -- 172.21.15.3:0/2970096875 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1660375732 0 0) 0x7f87380b4890 con 0x7f87380cdcf0 2024-05-09T06:37:28.055 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.073+0000 7f873659c640 1 --2- 172.21.15.3:0/2970096875 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f87381458b0 0x7f873815c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:37:28.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.073+0000 7f8735d9b640 1 --2- 172.21.15.3:0/2970096875 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f87380b4410 0x7f87380cd7b0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f87200097a0 tx=0x7f872002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a26ddf232cffe8c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:28.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.074+0000 7f873559a640 1 -- 172.21.15.3:0/2970096875 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8720037070 con 0x7f87380b4410 2024-05-09T06:37:28.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.074+0000 7f873559a640 1 -- 172.21.15.3:0/2970096875 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8720032440 con 0x7f87380b4410 2024-05-09T06:37:28.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.074+0000 7f873559a640 1 -- 172.21.15.3:0/2970096875 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8720032810 con 0x7f87380b4410 2024-05-09T06:37:28.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.074+0000 7f873cb4f640 1 -- 172.21.15.3:0/2970096875 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f87380b4410 msgr2=0x7f87380cd7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:28.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.074+0000 7f873cb4f640 1 --2- 172.21.15.3:0/2970096875 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f87380b4410 0x7f87380cd7b0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f87200097a0 tx=0x7f872002a180 comp rx=0 tx=0).stop 2024-05-09T06:37:28.057 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.074+0000 7f873cb4f640 1 -- 172.21.15.3:0/2970096875 shutdown_connections 2024-05-09T06:37:28.057 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.074+0000 7f873cb4f640 1 --2- 172.21.15.3:0/2970096875 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f87380b4410 0x7f87380cd7b0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:28.057 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.074+0000 7f873cb4f640 1 --2- 172.21.15.3:0/2970096875 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f87381458b0 0x7f873815c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:28.057 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.074+0000 7f873cb4f640 1 -- 172.21.15.3:0/2970096875 >> 172.21.15.3:0/2970096875 conn(0x7f873806b640 msgr2=0x7f873806ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:28.057 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.074+0000 7f873cb4f640 1 -- 172.21.15.3:0/2970096875 shutdown_connections 2024-05-09T06:37:28.057 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.074+0000 7f873cb4f640 1 -- 172.21.15.3:0/2970096875 wait complete. 2024-05-09T06:37:28.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.076+0000 7f873cb4f640 1 Processor -- start 2024-05-09T06:37:28.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.076+0000 7f873cb4f640 1 -- start start 2024-05-09T06:37:28.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.077+0000 7f873cb4f640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f87380b4410 0x7f87380ab3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:28.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.077+0000 7f873cb4f640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f87380cdcf0 0x7f87381f3790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:28.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.077+0000 7f873cb4f640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f87381458b0 0x7f87381f49e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:28.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.077+0000 7f873cb4f640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f873806d6b0 con 0x7f87380cdcf0 2024-05-09T06:37:28.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.077+0000 7f873cb4f640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f87380b5400 con 0x7f87381458b0 2024-05-09T06:37:28.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.077+0000 7f873cb4f640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f873806ebe0 con 0x7f87380b4410 2024-05-09T06:37:28.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.077+0000 7f8735d9b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f87380cdcf0 0x7f87381f3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:28.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.077+0000 7f8735d9b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f87380cdcf0 0x7f87381f3790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40740/0 (socket says 172.21.15.3:40740) 2024-05-09T06:37:28.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.077+0000 7f8735d9b640 1 -- 172.21.15.3:0/3463020357 learned_addr learned my addr 172.21.15.3:0/3463020357 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:37:28.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.077+0000 7f8736d9d640 1 --2- 172.21.15.3:0/3463020357 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f87381458b0 0x7f87381f49e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:28.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.077+0000 7f873659c640 1 --2- 172.21.15.3:0/3463020357 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f87380b4410 0x7f87380ab3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:28.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.077+0000 7f8735d9b640 1 -- 172.21.15.3:0/3463020357 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f87380b4410 msgr2=0x7f87380ab3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:28.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.077+0000 7f8735d9b640 1 --2- 172.21.15.3:0/3463020357 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f87380b4410 0x7f87380ab3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:28.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.077+0000 7f8735d9b640 1 -- 172.21.15.3:0/3463020357 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f87381458b0 msgr2=0x7f87381f49e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:28.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.077+0000 7f8735d9b640 1 --2- 172.21.15.3:0/3463020357 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f87381458b0 0x7f87381f49e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:28.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.077+0000 7f8735d9b640 1 -- 172.21.15.3:0/3463020357 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87381f54a0 con 0x7f87380cdcf0 2024-05-09T06:37:28.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.077+0000 7f8735d9b640 1 --2- 172.21.15.3:0/3463020357 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f87380cdcf0 0x7f87381f3790 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f87200097a0 tx=0x7f87200046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:28.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.078+0000 7f872effd640 1 -- 172.21.15.3:0/3463020357 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8720037050 con 0x7f87380cdcf0 2024-05-09T06:37:28.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.078+0000 7f872effd640 1 -- 172.21.15.3:0/3463020357 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8720038040 con 0x7f87380cdcf0 2024-05-09T06:37:28.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.078+0000 7f873cb4f640 1 -- 172.21.15.3:0/3463020357 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87381f5780 con 0x7f87380cdcf0 2024-05-09T06:37:28.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.078+0000 7f872effd640 1 -- 172.21.15.3:0/3463020357 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8720004490 con 0x7f87380cdcf0 2024-05-09T06:37:28.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.078+0000 7f873cb4f640 1 -- 172.21.15.3:0/3463020357 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87381f5d40 con 0x7f87380cdcf0 2024-05-09T06:37:28.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.079+0000 7f873cb4f640 1 -- 172.21.15.3:0/3463020357 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f8738069550 con 0x7f87380cdcf0 2024-05-09T06:37:28.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.079+0000 7f872effd640 1 -- 172.21.15.3:0/3463020357 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 7) ==== 90651+0+0 (secure 0 0 0) 0x7f8720030980 con 0x7f87380cdcf0 2024-05-09T06:37:28.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.080+0000 7f872effd640 1 --2- 172.21.15.3:0/3463020357 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f871806bfe0 0x7f871806e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:28.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.080+0000 7f872effd640 1 -- 172.21.15.3:0/3463020357 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 4943+0+0 (secure 0 0 0) 0x7f87200abbe0 con 0x7f87380cdcf0 2024-05-09T06:37:28.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.080+0000 7f872effd640 1 --2- 172.21.15.3:0/3463020357 >> [v2:172.21.15.3:6816/1897849827,v1:172.21.15.3:6817/1897849827] conn(0x7f8718071ea0 0x7f87180742c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:28.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.080+0000 7f872effd640 1 -- 172.21.15.3:0/3463020357 --> [v2:172.21.15.3:6816/1897849827,v1:172.21.15.3:6817/1897849827] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f872002be50 con 0x7f8718071ea0 2024-05-09T06:37:28.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.080+0000 7f872effd640 1 -- 172.21.15.3:0/3463020357 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f87200b0220 con 0x7f87380cdcf0 2024-05-09T06:37:28.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.080+0000 7f873659c640 1 --2- 172.21.15.3:0/3463020357 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f871806bfe0 0x7f871806e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:28.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.080+0000 7f8736d9d640 1 --2- 172.21.15.3:0/3463020357 >> [v2:172.21.15.3:6816/1897849827,v1:172.21.15.3:6817/1897849827] conn(0x7f8718071ea0 0x7f87180742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:28.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.080+0000 7f8736d9d640 1 --2- 172.21.15.3:0/3463020357 >> [v2:172.21.15.3:6816/1897849827,v1:172.21.15.3:6817/1897849827] conn(0x7f8718071ea0 0x7f87180742c0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:28.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.081+0000 7f873659c640 1 --2- 172.21.15.3:0/3463020357 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f871806bfe0 0x7f871806e4a0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f8730007510 tx=0x7f87300074a0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:28.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.081+0000 7f872effd640 1 -- 172.21.15.3:0/3463020357 <== osd.0 v2:172.21.15.3:6816/1897849827 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f872002be50 con 0x7f8718071ea0 2024-05-09T06:37:28.075 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.094+0000 7f873cb4f640 1 -- 172.21.15.3:0/3463020357 --> [v2:172.21.15.3:6816/1897849827,v1:172.21.15.3:6817/1897849827] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f87380cec40 con 0x7f8718071ea0 2024-05-09T06:37:28.075 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.095+0000 7f872effd640 1 -- 172.21.15.3:0/3463020357 <== osd.0 v2:172.21.15.3:6816/1897849827 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f87380cec40 con 0x7f8718071ea0 2024-05-09T06:37:28.076 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-05-09T06:37:28.076 INFO:teuthology.orchestra.run.smithi003.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-09T06:37:28.076 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-05-09T06:37:28.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.095+0000 7f873cb4f640 1 -- 172.21.15.3:0/3463020357 >> [v2:172.21.15.3:6816/1897849827,v1:172.21.15.3:6817/1897849827] conn(0x7f8718071ea0 msgr2=0x7f87180742c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:28.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.095+0000 7f873cb4f640 1 --2- 172.21.15.3:0/3463020357 >> [v2:172.21.15.3:6816/1897849827,v1:172.21.15.3:6817/1897849827] conn(0x7f8718071ea0 0x7f87180742c0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:28.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.095+0000 7f873cb4f640 1 -- 172.21.15.3:0/3463020357 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f871806bfe0 msgr2=0x7f871806e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:28.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.095+0000 7f873cb4f640 1 --2- 172.21.15.3:0/3463020357 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f871806bfe0 0x7f871806e4a0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f8730007510 tx=0x7f87300074a0 comp rx=0 tx=0).stop 2024-05-09T06:37:28.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.095+0000 7f873cb4f640 1 -- 172.21.15.3:0/3463020357 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f87380cdcf0 msgr2=0x7f87381f3790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:28.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.095+0000 7f873cb4f640 1 --2- 172.21.15.3:0/3463020357 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f87380cdcf0 0x7f87381f3790 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f87200097a0 tx=0x7f87200046c0 comp rx=0 tx=0).stop 2024-05-09T06:37:28.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.095+0000 7f8736d9d640 1 -- 172.21.15.3:0/3463020357 reap_dead start 2024-05-09T06:37:28.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.095+0000 7f873cb4f640 1 -- 172.21.15.3:0/3463020357 shutdown_connections 2024-05-09T06:37:28.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.095+0000 7f873cb4f640 1 -- 172.21.15.3:0/3463020357 >> 172.21.15.3:0/3463020357 conn(0x7f873806b640 msgr2=0x7f87380ce510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:28.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.095+0000 7f873cb4f640 1 -- 172.21.15.3:0/3463020357 shutdown_connections 2024-05-09T06:37:28.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.095+0000 7f873cb4f640 1 -- 172.21.15.3:0/3463020357 wait complete. 2024-05-09T06:37:28.087 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-05-09T06:37:28.193 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.212+0000 7f6ad7076640 1 Processor -- start 2024-05-09T06:37:28.193 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.212+0000 7f6ad7076640 1 -- start start 2024-05-09T06:37:28.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.213+0000 7f6ad7076640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f6ad015c0f0 0x7f6ad0160730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:28.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.213+0000 7f6ad7076640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f6ad00b4e30 0x7f6ad006e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:28.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.213+0000 7f6ad7076640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6ad006e790 0x7f6ad00cd5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:28.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.213+0000 7f6ad7076640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f6ad006ec00 con 0x7f6ad00b4e30 2024-05-09T06:37:28.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.213+0000 7f6ad7076640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f6ad0160c70 con 0x7f6ad015c0f0 2024-05-09T06:37:28.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.213+0000 7f6ad7076640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f6ad00b40f0 con 0x7f6ad006e790 2024-05-09T06:37:28.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.213+0000 7f6ac7fff640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f6ad00b4e30 0x7f6ad006e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:28.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.213+0000 7f6ac7fff640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f6ad00b4e30 0x7f6ad006e250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40742/0 (socket says 172.21.15.3:40742) 2024-05-09T06:37:28.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.213+0000 7f6ac7fff640 1 -- 172.21.15.3:0/1040058951 learned_addr learned my addr 172.21.15.3:0/1040058951 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:37:28.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.213+0000 7f6ad5613640 1 --2- 172.21.15.3:0/1040058951 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6ad006e790 0x7f6ad00cd5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:28.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.213+0000 7f6ad4e12640 1 --2- 172.21.15.3:0/1040058951 >> v2:172.21.15.190:3300/0 conn(0x7f6ad015c0f0 0x7f6ad0160730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:28.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.213+0000 7f6ac7fff640 1 -- 172.21.15.3:0/1040058951 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6ad006e790 msgr2=0x7f6ad00cd5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:37:28.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.213+0000 7f6ac7fff640 1 --2- 172.21.15.3:0/1040058951 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6ad006e790 0x7f6ad00cd5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:28.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.213+0000 7f6ac7fff640 1 -- 172.21.15.3:0/1040058951 >> v2:172.21.15.190:3300/0 conn(0x7f6ad015c0f0 msgr2=0x7f6ad0160730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:37:28.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.213+0000 7f6ac7fff640 1 --2- 172.21.15.3:0/1040058951 >> v2:172.21.15.190:3300/0 conn(0x7f6ad015c0f0 0x7f6ad0160730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:28.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.213+0000 7f6ac7fff640 1 -- 172.21.15.3:0/1040058951 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ad00b3050 con 0x7f6ad00b4e30 2024-05-09T06:37:28.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.213+0000 7f6ad5613640 1 --2- 172.21.15.3:0/1040058951 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6ad006e790 0x7f6ad00cd5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:37:28.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.213+0000 7f6ad4e12640 1 --2- 172.21.15.3:0/1040058951 >> v2:172.21.15.190:3300/0 conn(0x7f6ad015c0f0 0x7f6ad0160730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:37:28.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.214+0000 7f6ac7fff640 1 --2- 172.21.15.3:0/1040058951 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6ad00b4e30 0x7f6ad006e250 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f6ad00b2860 tx=0x7f6ac002a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf0630a962b39ea1 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:28.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.214+0000 7f6ac77fe640 1 -- 172.21.15.3:0/1040058951 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ac0037070 con 0x7f6ad00b4e30 2024-05-09T06:37:28.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.214+0000 7f6ac77fe640 1 -- 172.21.15.3:0/1040058951 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ac0032440 con 0x7f6ad00b4e30 2024-05-09T06:37:28.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.214+0000 7f6ac77fe640 1 -- 172.21.15.3:0/1040058951 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ac0032810 con 0x7f6ad00b4e30 2024-05-09T06:37:28.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.215+0000 7f6ad7076640 1 -- 172.21.15.3:0/1040058951 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6ad00b4e30 msgr2=0x7f6ad006e250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:28.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.215+0000 7f6ad7076640 1 --2- 172.21.15.3:0/1040058951 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6ad00b4e30 0x7f6ad006e250 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f6ad00b2860 tx=0x7f6ac002a140 comp rx=0 tx=0).stop 2024-05-09T06:37:28.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.215+0000 7f6ad7076640 1 -- 172.21.15.3:0/1040058951 shutdown_connections 2024-05-09T06:37:28.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.215+0000 7f6ad7076640 1 --2- 172.21.15.3:0/1040058951 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6ad006e790 0x7f6ad00cd5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:28.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.215+0000 7f6ad7076640 1 --2- 172.21.15.3:0/1040058951 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6ad00b4e30 0x7f6ad006e250 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:28.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.215+0000 7f6ad7076640 1 --2- 172.21.15.3:0/1040058951 >> v2:172.21.15.190:3300/0 conn(0x7f6ad015c0f0 0x7f6ad0160730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:28.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.215+0000 7f6ad7076640 1 -- 172.21.15.3:0/1040058951 >> 172.21.15.3:0/1040058951 conn(0x7f6ad00b1c20 msgr2=0x7f6ad00b2030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:28.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.215+0000 7f6ad7076640 1 -- 172.21.15.3:0/1040058951 shutdown_connections 2024-05-09T06:37:28.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.215+0000 7f6ad7076640 1 -- 172.21.15.3:0/1040058951 wait complete. 2024-05-09T06:37:28.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.216+0000 7f6ad7076640 1 Processor -- start 2024-05-09T06:37:28.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.216+0000 7f6ad7076640 1 -- start start 2024-05-09T06:37:28.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.217+0000 7f6ad7076640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6ad006e790 0x7f6ad014a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:28.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.217+0000 7f6ad7076640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6ad00b4e30 0x7f6ad00aee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:28.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.217+0000 7f6ad7076640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6ad015c0f0 0x7f6ad015f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:28.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.217+0000 7f6ad7076640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f6ad00b38f0 con 0x7f6ad00b4e30 2024-05-09T06:37:28.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.217+0000 7f6ad7076640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f6ad00b3610 con 0x7f6ad015c0f0 2024-05-09T06:37:28.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.217+0000 7f6ad7076640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f6ad00b3bd0 con 0x7f6ad006e790 2024-05-09T06:37:28.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.217+0000 7f6ac7fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6ad00b4e30 0x7f6ad00aee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:28.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.217+0000 7f6ac7fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6ad00b4e30 0x7f6ad00aee30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40754/0 (socket says 172.21.15.3:40754) 2024-05-09T06:37:28.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.217+0000 7f6ac7fff640 1 -- 172.21.15.3:0/3136452991 learned_addr learned my addr 172.21.15.3:0/3136452991 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:37:28.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.217+0000 7f6ad5613640 1 --2- 172.21.15.3:0/3136452991 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6ad015c0f0 0x7f6ad015f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:28.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.217+0000 7f6ad4e12640 1 --2- 172.21.15.3:0/3136452991 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6ad006e790 0x7f6ad014a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:28.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.217+0000 7f6ac7fff640 1 -- 172.21.15.3:0/3136452991 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6ad006e790 msgr2=0x7f6ad014a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:28.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.217+0000 7f6ac7fff640 1 --2- 172.21.15.3:0/3136452991 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6ad006e790 0x7f6ad014a3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:28.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.217+0000 7f6ac7fff640 1 -- 172.21.15.3:0/3136452991 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6ad015c0f0 msgr2=0x7f6ad015f1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:28.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.217+0000 7f6ac7fff640 1 --2- 172.21.15.3:0/3136452991 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6ad015c0f0 0x7f6ad015f1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:28.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.217+0000 7f6ac7fff640 1 -- 172.21.15.3:0/3136452991 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ad015d940 con 0x7f6ad00b4e30 2024-05-09T06:37:28.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.217+0000 7f6ac7fff640 1 --2- 172.21.15.3:0/3136452991 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6ad00b4e30 0x7f6ad00aee30 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f6ac002a180 tx=0x7f6ac00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:28.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.218+0000 7f6ac57fa640 1 -- 172.21.15.3:0/3136452991 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ac0037040 con 0x7f6ad00b4e30 2024-05-09T06:37:28.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.218+0000 7f6ac57fa640 1 -- 172.21.15.3:0/3136452991 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ac0042070 con 0x7f6ad00b4e30 2024-05-09T06:37:28.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.218+0000 7f6ad7076640 1 -- 172.21.15.3:0/3136452991 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ad015dc20 con 0x7f6ad00b4e30 2024-05-09T06:37:28.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.218+0000 7f6ac57fa640 1 -- 172.21.15.3:0/3136452991 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ac0030bb0 con 0x7f6ad00b4e30 2024-05-09T06:37:28.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.218+0000 7f6ad7076640 1 -- 172.21.15.3:0/3136452991 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ad00ae630 con 0x7f6ad00b4e30 2024-05-09T06:37:28.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.219+0000 7f6ad7076640 1 -- 172.21.15.3:0/3136452991 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ad0069550 con 0x7f6ad00b4e30 2024-05-09T06:37:28.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.219+0000 7f6ac57fa640 1 -- 172.21.15.3:0/3136452991 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 7) ==== 90651+0+0 (secure 0 0 0) 0x7f6ac0047460 con 0x7f6ad00b4e30 2024-05-09T06:37:28.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.220+0000 7f6ac57fa640 1 --2- 172.21.15.3:0/3136452991 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f6abc06bfe0 0x7f6abc06e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:28.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.220+0000 7f6ac57fa640 1 -- 172.21.15.3:0/3136452991 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 4943+0+0 (secure 0 0 0) 0x7f6ac00abab0 con 0x7f6ad00b4e30 2024-05-09T06:37:28.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.220+0000 7f6ad4e12640 1 --2- 172.21.15.3:0/3136452991 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f6abc06bfe0 0x7f6abc06e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:28.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.220+0000 7f6ad4e12640 1 --2- 172.21.15.3:0/3136452991 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f6abc06bfe0 0x7f6abc06e4a0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f6acc00a8d0 tx=0x7f6acc008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:28.204 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.224+0000 7f6ac57fa640 1 -- 172.21.15.3:0/3136452991 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f6ac0045360 con 0x7f6ad00b4e30 2024-05-09T06:37:28.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.405+0000 7f6ad7076640 1 -- 172.21.15.3:0/3136452991 --> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) -- 0x7f6ad00af370 con 0x7f6abc06bfe0 2024-05-09T06:37:28.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.406+0000 7f6ac57fa640 1 -- 172.21.15.3:0/3136452991 <== mgr.4105 v2:172.21.15.190:6832/975893321 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f6ad00af370 con 0x7f6abc06bfe0 2024-05-09T06:37:28.387 INFO:teuthology.orchestra.run.smithi003.stderr:instructed osd(s) 0 to deep-scrub 2024-05-09T06:37:28.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.408+0000 7f6ad7076640 1 -- 172.21.15.3:0/3136452991 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f6abc06bfe0 msgr2=0x7f6abc06e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:28.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.408+0000 7f6ad7076640 1 --2- 172.21.15.3:0/3136452991 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f6abc06bfe0 0x7f6abc06e4a0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f6acc00a8d0 tx=0x7f6acc008040 comp rx=0 tx=0).stop 2024-05-09T06:37:28.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.408+0000 7f6ad7076640 1 -- 172.21.15.3:0/3136452991 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6ad00b4e30 msgr2=0x7f6ad00aee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:28.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.408+0000 7f6ad7076640 1 --2- 172.21.15.3:0/3136452991 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6ad00b4e30 0x7f6ad00aee30 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f6ac002a180 tx=0x7f6ac00046c0 comp rx=0 tx=0).stop 2024-05-09T06:37:28.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.408+0000 7f6ad7076640 1 -- 172.21.15.3:0/3136452991 shutdown_connections 2024-05-09T06:37:28.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.408+0000 7f6ad7076640 1 --2- 172.21.15.3:0/3136452991 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f6abc06bfe0 0x7f6abc06e4a0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:28.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.408+0000 7f6ad7076640 1 --2- 172.21.15.3:0/3136452991 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6ad015c0f0 0x7f6ad015f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:28.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.408+0000 7f6ad7076640 1 --2- 172.21.15.3:0/3136452991 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6ad00b4e30 0x7f6ad00aee30 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:28.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.408+0000 7f6ad7076640 1 --2- 172.21.15.3:0/3136452991 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6ad006e790 0x7f6ad014a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:28.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.408+0000 7f6ad7076640 1 -- 172.21.15.3:0/3136452991 >> 172.21.15.3:0/3136452991 conn(0x7f6ad00b1c20 msgr2=0x7f6ad01510d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:28.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.408+0000 7f6ad7076640 1 -- 172.21.15.3:0/3136452991 shutdown_connections 2024-05-09T06:37:28.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.408+0000 7f6ad7076640 1 -- 172.21.15.3:0/3136452991 wait complete. 2024-05-09T06:37:28.400 INFO:tasks.ceph:Scrubbing osd.1 2024-05-09T06:37:28.400 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-05-09T06:37:28.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.528+0000 7fa3c9015640 1 Processor -- start 2024-05-09T06:37:28.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.528+0000 7fa3c9015640 1 -- start start 2024-05-09T06:37:28.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.528+0000 7fa3c9015640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa3c40b32b0 0x7fa3c40b71d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:28.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.528+0000 7fa3c9015640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fa3c40cee30 0x7fa3c40cf2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:28.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.528+0000 7fa3c9015640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa3c40cf800 con 0x7fa3c415e400 2024-05-09T06:37:28.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.528+0000 7fa3c9015640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fa3c40d2f50 con 0x7fa3c40b32b0 2024-05-09T06:37:28.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.528+0000 7fa3c9015640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7fa3c40d2620 con 0x7fa3c40cee30 2024-05-09T06:37:28.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.529+0000 7fa3c259c640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa3c40b32b0 0x7fa3c40b71d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:28.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.529+0000 7fa3c259c640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa3c40b32b0 0x7fa3c40b71d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40762/0 (socket says 172.21.15.3:40762) 2024-05-09T06:37:28.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.529+0000 7fa3c259c640 1 -- 172.21.15.3:0/1085437992 learned_addr learned my addr 172.21.15.3:0/1085437992 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:37:28.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.529+0000 7fa3c359e640 1 --2- 172.21.15.3:0/1085437992 >> v2:172.21.15.190:3300/0 conn(0x7fa3c40cee30 0x7fa3c40cf2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:28.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.529+0000 7fa3c259c640 1 -- 172.21.15.3:0/1085437992 >> v2:172.21.15.190:3300/0 conn(0x7fa3c40cee30 msgr2=0x7fa3c40cf2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:37:28.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.529+0000 7fa3c259c640 1 --2- 172.21.15.3:0/1085437992 >> v2:172.21.15.190:3300/0 conn(0x7fa3c40cee30 0x7fa3c40cf2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:28.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.529+0000 7fa3c259c640 1 -- 172.21.15.3:0/1085437992 >> v1:172.21.15.190:6789/0 conn(0x7fa3c415e400 legacy=0x7fa3c40b99c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:28.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.529+0000 7fa3c259c640 1 -- 172.21.15.3:0/1085437992 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3c40d2900 con 0x7fa3c40b32b0 2024-05-09T06:37:28.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.530+0000 7fa3c259c640 1 --2- 172.21.15.3:0/1085437992 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa3c40b32b0 0x7fa3c40b71d0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fa3c406d380 tx=0x7fa3b802a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de94d23c7e9d2fca server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:28.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.530+0000 7fa3c1d9b640 1 -- 172.21.15.3:0/1085437992 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3b8037070 con 0x7fa3c40b32b0 2024-05-09T06:37:28.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.530+0000 7fa3c1d9b640 1 -- 172.21.15.3:0/1085437992 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa3b802ae50 con 0x7fa3c40b32b0 2024-05-09T06:37:28.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.530+0000 7fa3c1d9b640 1 -- 172.21.15.3:0/1085437992 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3b8032860 con 0x7fa3c40b32b0 2024-05-09T06:37:28.513 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.530+0000 7fa3c9015640 1 -- 172.21.15.3:0/1085437992 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa3c40b32b0 msgr2=0x7fa3c40b71d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:28.513 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.530+0000 7fa3c9015640 1 --2- 172.21.15.3:0/1085437992 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa3c40b32b0 0x7fa3c40b71d0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fa3c406d380 tx=0x7fa3b802a300 comp rx=0 tx=0).stop 2024-05-09T06:37:28.513 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.530+0000 7fa3c9015640 1 -- 172.21.15.3:0/1085437992 shutdown_connections 2024-05-09T06:37:28.513 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.530+0000 7fa3c9015640 1 --2- 172.21.15.3:0/1085437992 >> v2:172.21.15.190:3300/0 conn(0x7fa3c40cee30 0x7fa3c40cf2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:28.513 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.530+0000 7fa3c9015640 1 --2- 172.21.15.3:0/1085437992 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa3c40b32b0 0x7fa3c40b71d0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:28.513 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.530+0000 7fa3c9015640 1 -- 172.21.15.3:0/1085437992 >> 172.21.15.3:0/1085437992 conn(0x7fa3c40675c0 msgr2=0x7fa3c40d0be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:28.513 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.530+0000 7fa3c9015640 1 -- 172.21.15.3:0/1085437992 shutdown_connections 2024-05-09T06:37:28.513 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.530+0000 7fa3c9015640 1 -- 172.21.15.3:0/1085437992 wait complete. 2024-05-09T06:37:28.513 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.532+0000 7fa3c9015640 1 Processor -- start 2024-05-09T06:37:28.513 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.532+0000 7fa3c9015640 1 -- start start 2024-05-09T06:37:28.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.533+0000 7fa3c9015640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa3c40b32b0 0x7fa3c415fb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:28.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.533+0000 7fa3c9015640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa3c40cee30 0x7fa3c40cca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:28.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.533+0000 7fa3c9015640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa3c415e400 0x7fa3c415e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:28.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.533+0000 7fa3c9015640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fa3c40b7890 con 0x7fa3c415e400 2024-05-09T06:37:28.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.533+0000 7fa3c9015640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fa3c40b7710 con 0x7fa3c40cee30 2024-05-09T06:37:28.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.533+0000 7fa3c9015640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fa3c40d33e0 con 0x7fa3c40b32b0 2024-05-09T06:37:28.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.533+0000 7fa3c359e640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa3c415e400 0x7fa3c415e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:28.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.533+0000 7fa3c359e640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa3c415e400 0x7fa3c415e7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40778/0 (socket says 172.21.15.3:40778) 2024-05-09T06:37:28.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.533+0000 7fa3c359e640 1 -- 172.21.15.3:0/2339501741 learned_addr learned my addr 172.21.15.3:0/2339501741 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:37:28.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.533+0000 7fa3c259c640 1 --2- 172.21.15.3:0/2339501741 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa3c40cee30 0x7fa3c40cca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:28.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.533+0000 7fa3c2d9d640 1 --2- 172.21.15.3:0/2339501741 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa3c40b32b0 0x7fa3c415fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:28.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.533+0000 7fa3c359e640 1 -- 172.21.15.3:0/2339501741 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa3c40b32b0 msgr2=0x7fa3c415fb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:28.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.533+0000 7fa3c359e640 1 --2- 172.21.15.3:0/2339501741 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa3c40b32b0 0x7fa3c415fb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:28.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.533+0000 7fa3c359e640 1 -- 172.21.15.3:0/2339501741 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa3c40cee30 msgr2=0x7fa3c40cca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:28.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.533+0000 7fa3c359e640 1 --2- 172.21.15.3:0/2339501741 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa3c40cee30 0x7fa3c40cca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:28.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.533+0000 7fa3c359e640 1 -- 172.21.15.3:0/2339501741 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3c4160350 con 0x7fa3c415e400 2024-05-09T06:37:28.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.533+0000 7fa3c2d9d640 1 --2- 172.21.15.3:0/2339501741 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa3c40b32b0 0x7fa3c415fb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:37:28.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.534+0000 7fa3c359e640 1 --2- 172.21.15.3:0/2339501741 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa3c415e400 0x7fa3c415e7e0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fa3bc00efc0 tx=0x7fa3bc00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:28.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.534+0000 7fa3ab7fe640 1 -- 172.21.15.3:0/2339501741 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3bc018020 con 0x7fa3c415e400 2024-05-09T06:37:28.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.534+0000 7fa3ab7fe640 1 -- 172.21.15.3:0/2339501741 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa3bc014480 con 0x7fa3c415e400 2024-05-09T06:37:28.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.534+0000 7fa3c9015640 1 -- 172.21.15.3:0/2339501741 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3c4160630 con 0x7fa3c415e400 2024-05-09T06:37:28.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.534+0000 7fa3ab7fe640 1 -- 172.21.15.3:0/2339501741 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3bc014870 con 0x7fa3c415e400 2024-05-09T06:37:28.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.534+0000 7fa3c9015640 1 -- 172.21.15.3:0/2339501741 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3c4160ed0 con 0x7fa3c415e400 2024-05-09T06:37:28.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.535+0000 7fa3c9015640 1 -- 172.21.15.3:0/2339501741 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa3c4069550 con 0x7fa3c415e400 2024-05-09T06:37:28.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.535+0000 7fa3ab7fe640 1 -- 172.21.15.3:0/2339501741 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 7) ==== 90651+0+0 (secure 0 0 0) 0x7fa3bc005d10 con 0x7fa3c415e400 2024-05-09T06:37:28.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.536+0000 7fa3ab7fe640 1 --2- 172.21.15.3:0/2339501741 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fa3b006bfe0 0x7fa3b006e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:28.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.536+0000 7fa3ab7fe640 1 -- 172.21.15.3:0/2339501741 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 4943+0+0 (secure 0 0 0) 0x7fa3bc08baf0 con 0x7fa3c415e400 2024-05-09T06:37:28.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.536+0000 7fa3ab7fe640 1 --2- 172.21.15.3:0/2339501741 >> [v2:172.21.15.3:6800/926805498,v1:172.21.15.3:6801/926805498] conn(0x7fa3b0071ea0 0x7fa3b00742c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:28.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.536+0000 7fa3c2d9d640 1 --2- 172.21.15.3:0/2339501741 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fa3b006bfe0 0x7fa3b006e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:28.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.536+0000 7fa3ab7fe640 1 -- 172.21.15.3:0/2339501741 --> [v2:172.21.15.3:6800/926805498,v1:172.21.15.3:6801/926805498] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa3b0074970 con 0x7fa3b0071ea0 2024-05-09T06:37:28.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.536+0000 7fa3ab7fe640 1 -- 172.21.15.3:0/2339501741 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fa3bc0902a0 con 0x7fa3c415e400 2024-05-09T06:37:28.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.536+0000 7fa3c259c640 1 --2- 172.21.15.3:0/2339501741 >> [v2:172.21.15.3:6800/926805498,v1:172.21.15.3:6801/926805498] conn(0x7fa3b0071ea0 0x7fa3b00742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:28.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.536+0000 7fa3c259c640 1 --2- 172.21.15.3:0/2339501741 >> [v2:172.21.15.3:6800/926805498,v1:172.21.15.3:6801/926805498] conn(0x7fa3b0071ea0 0x7fa3b00742c0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:28.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.536+0000 7fa3c2d9d640 1 --2- 172.21.15.3:0/2339501741 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fa3b006bfe0 0x7fa3b006e4a0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fa3b4001660 tx=0x7fa3b4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:28.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.537+0000 7fa3ab7fe640 1 -- 172.21.15.3:0/2339501741 <== osd.1 v2:172.21.15.3:6800/926805498 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa3b0074970 con 0x7fa3b0071ea0 2024-05-09T06:37:28.531 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.550+0000 7fa3c9015640 1 -- 172.21.15.3:0/2339501741 --> [v2:172.21.15.3:6800/926805498,v1:172.21.15.3:6801/926805498] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fa3c40b3090 con 0x7fa3b0071ea0 2024-05-09T06:37:28.531 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.550+0000 7fa3ab7fe640 1 -- 172.21.15.3:0/2339501741 <== osd.1 v2:172.21.15.3:6800/926805498 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fa3c40b3090 con 0x7fa3b0071ea0 2024-05-09T06:37:28.531 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-05-09T06:37:28.531 INFO:teuthology.orchestra.run.smithi003.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-09T06:37:28.531 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-05-09T06:37:28.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.551+0000 7fa3c9015640 1 -- 172.21.15.3:0/2339501741 >> [v2:172.21.15.3:6800/926805498,v1:172.21.15.3:6801/926805498] conn(0x7fa3b0071ea0 msgr2=0x7fa3b00742c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:28.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.551+0000 7fa3c9015640 1 --2- 172.21.15.3:0/2339501741 >> [v2:172.21.15.3:6800/926805498,v1:172.21.15.3:6801/926805498] conn(0x7fa3b0071ea0 0x7fa3b00742c0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:28.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.551+0000 7fa3c9015640 1 -- 172.21.15.3:0/2339501741 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fa3b006bfe0 msgr2=0x7fa3b006e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:28.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.551+0000 7fa3c9015640 1 --2- 172.21.15.3:0/2339501741 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fa3b006bfe0 0x7fa3b006e4a0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fa3b4001660 tx=0x7fa3b4009040 comp rx=0 tx=0).stop 2024-05-09T06:37:28.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.551+0000 7fa3c9015640 1 -- 172.21.15.3:0/2339501741 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa3c415e400 msgr2=0x7fa3c415e7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:28.533 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.551+0000 7fa3c9015640 1 --2- 172.21.15.3:0/2339501741 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa3c415e400 0x7fa3c415e7e0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fa3bc00efc0 tx=0x7fa3bc00c490 comp rx=0 tx=0).stop 2024-05-09T06:37:28.533 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.551+0000 7fa3c359e640 1 -- 172.21.15.3:0/2339501741 reap_dead start 2024-05-09T06:37:28.533 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.551+0000 7fa3c9015640 1 -- 172.21.15.3:0/2339501741 shutdown_connections 2024-05-09T06:37:28.533 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.551+0000 7fa3c9015640 1 -- 172.21.15.3:0/2339501741 >> 172.21.15.3:0/2339501741 conn(0x7fa3c40675c0 msgr2=0x7fa3c40b7a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:28.533 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.551+0000 7fa3c9015640 1 -- 172.21.15.3:0/2339501741 shutdown_connections 2024-05-09T06:37:28.533 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.551+0000 7fa3c9015640 1 -- 172.21.15.3:0/2339501741 wait complete. 2024-05-09T06:37:28.542 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-05-09T06:37:28.648 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.667+0000 7fe1b3e45640 1 Processor -- start 2024-05-09T06:37:28.648 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.667+0000 7fe1b3e45640 1 -- start start 2024-05-09T06:37:28.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.668+0000 7fe1b3e45640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fe1ac0cf990 0x7fe1ac0cc1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:28.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.668+0000 7fe1b3e45640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fe1ac15ea60 0x7fe1ac0b2f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:28.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.668+0000 7fe1b3e45640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe1ac071220 con 0x7fe1ac0cc6e0 2024-05-09T06:37:28.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.668+0000 7fe1b3e45640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fe1ac071500 con 0x7fe1ac15ea60 2024-05-09T06:37:28.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.668+0000 7fe1b3e45640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7fe1ac0717e0 con 0x7fe1ac0cf990 2024-05-09T06:37:28.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.668+0000 7fe1b23e2640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fe1ac15ea60 0x7fe1ac0b2f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:28.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.668+0000 7fe1b13e0640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fe1ac0cc6e0 0x7fe1ac15e350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:43126/0 (socket says 172.21.15.3:43126) 2024-05-09T06:37:28.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.668+0000 7fe1b13e0640 1 -- 172.21.15.3:0/2208477344 learned_addr learned my addr 172.21.15.3:0/2208477344 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-05-09T06:37:28.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.668+0000 7fe1b23e2640 1 -- 172.21.15.3:0/2208477344 >> v2:172.21.15.190:3300/0 conn(0x7fe1ac0cf990 msgr2=0x7fe1ac0cc1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:37:28.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.668+0000 7fe1b1be1640 1 --2- 172.21.15.3:0/2208477344 >> v2:172.21.15.190:3300/0 conn(0x7fe1ac0cf990 0x7fe1ac0cc1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:28.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.668+0000 7fe1b23e2640 1 --2- 172.21.15.3:0/2208477344 >> v2:172.21.15.190:3300/0 conn(0x7fe1ac0cf990 0x7fe1ac0cc1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:28.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.668+0000 7fe1b23e2640 1 -- 172.21.15.3:0/2208477344 >> v1:172.21.15.3:6789/0 conn(0x7fe1ac0cc6e0 legacy=0x7fe1ac15e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:28.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.668+0000 7fe1b23e2640 1 -- 172.21.15.3:0/2208477344 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1ac071ac0 con 0x7fe1ac15ea60 2024-05-09T06:37:28.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.668+0000 7fe1b1be1640 1 --2- 172.21.15.3:0/2208477344 >> v2:172.21.15.190:3300/0 conn(0x7fe1ac0cf990 0x7fe1ac0cc1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:37:28.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.669+0000 7fe1b23e2640 1 --2- 172.21.15.3:0/2208477344 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe1ac15ea60 0x7fe1ac0b2f20 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fe1ac073870 tx=0x7fe1a402a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c42bff46385bf5f2 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:28.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.669+0000 7fe1b0bdf640 1 -- 172.21.15.3:0/2208477344 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1a4007740 con 0x7fe1ac15ea60 2024-05-09T06:37:28.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.669+0000 7fe1b0bdf640 1 -- 172.21.15.3:0/2208477344 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe1a40078e0 con 0x7fe1ac15ea60 2024-05-09T06:37:28.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.669+0000 7fe1b0bdf640 1 -- 172.21.15.3:0/2208477344 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1a4007cb0 con 0x7fe1ac15ea60 2024-05-09T06:37:28.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.670+0000 7fe1b3e45640 1 -- 172.21.15.3:0/2208477344 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe1ac15ea60 msgr2=0x7fe1ac0b2f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:28.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.670+0000 7fe1b3e45640 1 --2- 172.21.15.3:0/2208477344 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe1ac15ea60 0x7fe1ac0b2f20 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fe1ac073870 tx=0x7fe1a402a680 comp rx=0 tx=0).stop 2024-05-09T06:37:28.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.670+0000 7fe1b3e45640 1 -- 172.21.15.3:0/2208477344 shutdown_connections 2024-05-09T06:37:28.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.670+0000 7fe1b3e45640 1 --2- 172.21.15.3:0/2208477344 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe1ac15ea60 0x7fe1ac0b2f20 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:28.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.670+0000 7fe1b3e45640 1 --2- 172.21.15.3:0/2208477344 >> v2:172.21.15.190:3300/0 conn(0x7fe1ac0cf990 0x7fe1ac0cc1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:28.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.670+0000 7fe1b3e45640 1 -- 172.21.15.3:0/2208477344 >> 172.21.15.3:0/2208477344 conn(0x7fe1ac0b1c20 msgr2=0x7fe1ac0b2030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:28.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.670+0000 7fe1b3e45640 1 -- 172.21.15.3:0/2208477344 shutdown_connections 2024-05-09T06:37:28.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.670+0000 7fe1b3e45640 1 -- 172.21.15.3:0/2208477344 wait complete. 2024-05-09T06:37:28.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.671+0000 7fe1b3e45640 1 Processor -- start 2024-05-09T06:37:28.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.671+0000 7fe1b3e45640 1 -- start start 2024-05-09T06:37:28.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.672+0000 7fe1b3e45640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe1ac0cc6e0 0x7fe1ac13f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:28.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.672+0000 7fe1b3e45640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe1ac0cf990 0x7fe1ac0b0d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:28.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.672+0000 7fe1b3e45640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe1ac15ea60 0x7fe1ac0b1250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:28.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.672+0000 7fe1b3e45640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fe1ac06e3d0 con 0x7fe1ac15ea60 2024-05-09T06:37:28.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.672+0000 7fe1b3e45640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fe1ac06e250 con 0x7fe1ac0cf990 2024-05-09T06:37:28.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.672+0000 7fe1b3e45640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fe1ac06e550 con 0x7fe1ac0cc6e0 2024-05-09T06:37:28.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.672+0000 7fe1b23e2640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe1ac15ea60 0x7fe1ac0b1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:28.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.672+0000 7fe1b23e2640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe1ac15ea60 0x7fe1ac0b1250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40796/0 (socket says 172.21.15.3:40796) 2024-05-09T06:37:28.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.672+0000 7fe1b23e2640 1 -- 172.21.15.3:0/1063169176 learned_addr learned my addr 172.21.15.3:0/1063169176 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:37:28.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.672+0000 7fe1b1be1640 1 --2- 172.21.15.3:0/1063169176 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe1ac0cc6e0 0x7fe1ac13f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:28.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.672+0000 7fe1b13e0640 1 --2- 172.21.15.3:0/1063169176 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe1ac0cf990 0x7fe1ac0b0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:28.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.672+0000 7fe1b23e2640 1 -- 172.21.15.3:0/1063169176 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe1ac0cc6e0 msgr2=0x7fe1ac13f2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:28.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.672+0000 7fe1b23e2640 1 --2- 172.21.15.3:0/1063169176 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe1ac0cc6e0 0x7fe1ac13f2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:28.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.672+0000 7fe1b23e2640 1 -- 172.21.15.3:0/1063169176 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe1ac0cf990 msgr2=0x7fe1ac0b0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:28.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.672+0000 7fe1b23e2640 1 --2- 172.21.15.3:0/1063169176 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe1ac0cf990 0x7fe1ac0b0d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:28.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.672+0000 7fe1b23e2640 1 -- 172.21.15.3:0/1063169176 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1ac13f820 con 0x7fe1ac15ea60 2024-05-09T06:37:28.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.673+0000 7fe1b23e2640 1 --2- 172.21.15.3:0/1063169176 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe1ac15ea60 0x7fe1ac0b1250 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fe1a402ac70 tx=0x7fe1a4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:28.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.673+0000 7fe1a27fc640 1 -- 172.21.15.3:0/1063169176 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1a403f070 con 0x7fe1ac15ea60 2024-05-09T06:37:28.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.673+0000 7fe1a27fc640 1 -- 172.21.15.3:0/1063169176 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe1a4031c90 con 0x7fe1ac15ea60 2024-05-09T06:37:28.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.673+0000 7fe1a27fc640 1 -- 172.21.15.3:0/1063169176 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1a4044680 con 0x7fe1ac15ea60 2024-05-09T06:37:28.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.673+0000 7fe1b3e45640 1 -- 172.21.15.3:0/1063169176 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1ac148540 con 0x7fe1ac15ea60 2024-05-09T06:37:28.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.673+0000 7fe1b3e45640 1 -- 172.21.15.3:0/1063169176 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1ac148b00 con 0x7fe1ac15ea60 2024-05-09T06:37:28.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.674+0000 7fe1b3e45640 1 -- 172.21.15.3:0/1063169176 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1ac069550 con 0x7fe1ac15ea60 2024-05-09T06:37:28.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.674+0000 7fe1a27fc640 1 -- 172.21.15.3:0/1063169176 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 7) ==== 90651+0+0 (secure 0 0 0) 0x7fe1a403b410 con 0x7fe1ac15ea60 2024-05-09T06:37:28.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.675+0000 7fe1a27fc640 1 --2- 172.21.15.3:0/1063169176 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fe19006bfe0 0x7fe19006e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:28.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.675+0000 7fe1a27fc640 1 -- 172.21.15.3:0/1063169176 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 4943+0+0 (secure 0 0 0) 0x7fe1a40ac7a0 con 0x7fe1ac15ea60 2024-05-09T06:37:28.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.675+0000 7fe1b1be1640 1 --2- 172.21.15.3:0/1063169176 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fe19006bfe0 0x7fe19006e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:28.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.676+0000 7fe1b1be1640 1 --2- 172.21.15.3:0/1063169176 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fe19006bfe0 0x7fe19006e4a0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fe1ac144dd0 tx=0x7fe1a8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:28.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.679+0000 7fe1a27fc640 1 -- 172.21.15.3:0/1063169176 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fe1ac13f820 con 0x7fe1ac15ea60 2024-05-09T06:37:28.844 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.863+0000 7fe1b3e45640 1 -- 172.21.15.3:0/1063169176 --> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) -- 0x7fe1ac0b50b0 con 0x7fe19006bfe0 2024-05-09T06:37:28.845 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.863+0000 7fe1a27fc640 1 -- 172.21.15.3:0/1063169176 <== mgr.4105 v2:172.21.15.190:6832/975893321 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fe1ac0b50b0 con 0x7fe19006bfe0 2024-05-09T06:37:28.845 INFO:teuthology.orchestra.run.smithi003.stderr:instructed osd(s) 1 to deep-scrub 2024-05-09T06:37:28.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.865+0000 7fe1b3e45640 1 -- 172.21.15.3:0/1063169176 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fe19006bfe0 msgr2=0x7fe19006e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:28.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.865+0000 7fe1b3e45640 1 --2- 172.21.15.3:0/1063169176 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fe19006bfe0 0x7fe19006e4a0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fe1ac144dd0 tx=0x7fe1a8008040 comp rx=0 tx=0).stop 2024-05-09T06:37:28.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.865+0000 7fe1b3e45640 1 -- 172.21.15.3:0/1063169176 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe1ac15ea60 msgr2=0x7fe1ac0b1250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:28.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.865+0000 7fe1b3e45640 1 --2- 172.21.15.3:0/1063169176 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe1ac15ea60 0x7fe1ac0b1250 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fe1a402ac70 tx=0x7fe1a4004a40 comp rx=0 tx=0).stop 2024-05-09T06:37:28.848 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.865+0000 7fe1b3e45640 1 -- 172.21.15.3:0/1063169176 shutdown_connections 2024-05-09T06:37:28.848 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.865+0000 7fe1b3e45640 1 --2- 172.21.15.3:0/1063169176 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fe19006bfe0 0x7fe19006e4a0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:28.848 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.865+0000 7fe1b3e45640 1 --2- 172.21.15.3:0/1063169176 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe1ac15ea60 0x7fe1ac0b1250 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:28.848 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.865+0000 7fe1b3e45640 1 --2- 172.21.15.3:0/1063169176 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe1ac0cf990 0x7fe1ac0b0d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:28.848 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.865+0000 7fe1b3e45640 1 --2- 172.21.15.3:0/1063169176 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe1ac0cc6e0 0x7fe1ac13f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:28.848 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.865+0000 7fe1b3e45640 1 -- 172.21.15.3:0/1063169176 >> 172.21.15.3:0/1063169176 conn(0x7fe1ac0b1c20 msgr2=0x7fe1ac0b38b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:28.848 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.865+0000 7fe1b3e45640 1 -- 172.21.15.3:0/1063169176 shutdown_connections 2024-05-09T06:37:28.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.865+0000 7fe1b3e45640 1 -- 172.21.15.3:0/1063169176 wait complete. 2024-05-09T06:37:28.857 INFO:tasks.ceph:Scrubbing osd.2 2024-05-09T06:37:28.857 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-05-09T06:37:28.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.985+0000 7f57e5638640 1 Processor -- start 2024-05-09T06:37:28.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.985+0000 7f57e5638640 1 -- start start 2024-05-09T06:37:28.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.986+0000 7f57e5638640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f57e015e850 0x7f57e00b0e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:28.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.986+0000 7f57e5638640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f57e00d4230 0x7f57e00d46a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:28.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.986+0000 7f57e5638640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f57e00b1530 con 0x7f57e00cf090 2024-05-09T06:37:28.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.986+0000 7f57e5638640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f57e00d4be0 con 0x7f57e00d4230 2024-05-09T06:37:28.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.986+0000 7f57e5638640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f57e00b35e0 con 0x7f57e015e850 2024-05-09T06:37:28.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.986+0000 7f57df7fe640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f57e00d4230 0x7f57e00d46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:28.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.986+0000 7f57de7fc640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f57e00cf090 0x7f57e00d2f60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:43128/0 (socket says 172.21.15.3:43128) 2024-05-09T06:37:28.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.986+0000 7f57df7fe640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f57e00d4230 0x7f57e00d46a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40804/0 (socket says 172.21.15.3:40804) 2024-05-09T06:37:28.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.986+0000 7f57de7fc640 1 -- 172.21.15.3:0/1159201038 learned_addr learned my addr 172.21.15.3:0/1159201038 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-05-09T06:37:28.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.987+0000 7f57deffd640 1 --2- 172.21.15.3:0/1159201038 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f57e015e850 0x7f57e00b0e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:28.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.987+0000 7f57df7fe640 1 -- 172.21.15.3:0/1159201038 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f57e015e850 msgr2=0x7f57e00b0e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:37:28.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.987+0000 7f57df7fe640 1 --2- 172.21.15.3:0/1159201038 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f57e015e850 0x7f57e00b0e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:28.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.987+0000 7f57df7fe640 1 -- 172.21.15.3:0/1159201038 >> v1:172.21.15.3:6789/0 conn(0x7f57e00cf090 legacy=0x7f57e00d2f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:28.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.987+0000 7f57df7fe640 1 -- 172.21.15.3:0/1159201038 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57e00b1810 con 0x7f57e00d4230 2024-05-09T06:37:28.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.987+0000 7f57deffd640 1 --2- 172.21.15.3:0/1159201038 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f57e015e850 0x7f57e00b0e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:37:28.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.987+0000 7f57df7fe640 1 --2- 172.21.15.3:0/1159201038 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f57e00d4230 0x7f57e00d46a0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f57d80097a0 tx=0x7f57d802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dda6a58f58e7bdd4 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:28.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.988+0000 7f57ddffb640 1 -- 172.21.15.3:0/1159201038 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57d8037070 con 0x7f57e00d4230 2024-05-09T06:37:28.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.988+0000 7f57ddffb640 1 -- 172.21.15.3:0/1159201038 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57d8032440 con 0x7f57e00d4230 2024-05-09T06:37:28.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.988+0000 7f57ddffb640 1 -- 172.21.15.3:0/1159201038 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57d8032810 con 0x7f57e00d4230 2024-05-09T06:37:28.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.988+0000 7f57e5638640 1 -- 172.21.15.3:0/1159201038 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f57e00d4230 msgr2=0x7f57e00d46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:28.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.988+0000 7f57e5638640 1 --2- 172.21.15.3:0/1159201038 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f57e00d4230 0x7f57e00d46a0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f57d80097a0 tx=0x7f57d802a180 comp rx=0 tx=0).stop 2024-05-09T06:37:28.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.988+0000 7f57e5638640 1 -- 172.21.15.3:0/1159201038 shutdown_connections 2024-05-09T06:37:28.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.988+0000 7f57e5638640 1 --2- 172.21.15.3:0/1159201038 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f57e00d4230 0x7f57e00d46a0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:28.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.988+0000 7f57e5638640 1 --2- 172.21.15.3:0/1159201038 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f57e015e850 0x7f57e00b0e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:28.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.988+0000 7f57e5638640 1 -- 172.21.15.3:0/1159201038 >> 172.21.15.3:0/1159201038 conn(0x7f57e008dc60 msgr2=0x7f57e009d580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:28.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.988+0000 7f57e5638640 1 -- 172.21.15.3:0/1159201038 shutdown_connections 2024-05-09T06:37:28.978 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.988+0000 7f57e5638640 1 -- 172.21.15.3:0/1159201038 wait complete. 2024-05-09T06:37:28.978 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.990+0000 7f57e5638640 1 Processor -- start 2024-05-09T06:37:28.978 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.990+0000 7f57e5638640 1 -- start start 2024-05-09T06:37:28.978 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.990+0000 7f57e5638640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f57e00cf090 0x7f57e01539c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:28.978 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.990+0000 7f57e5638640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f57e00d4230 0x7f57e009e900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:28.978 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.990+0000 7f57e5638640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f57e015e850 0x7f57e00b7840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:28.978 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.990+0000 7f57e5638640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f57e00b1dd0 con 0x7f57e015e850 2024-05-09T06:37:28.978 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.990+0000 7f57e5638640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f57e015ec30 con 0x7f57e00d4230 2024-05-09T06:37:28.979 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.990+0000 7f57e5638640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f57e00b20b0 con 0x7f57e00cf090 2024-05-09T06:37:28.979 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.991+0000 7f57df7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f57e015e850 0x7f57e00b7840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:28.979 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.991+0000 7f57df7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f57e015e850 0x7f57e00b7840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40820/0 (socket says 172.21.15.3:40820) 2024-05-09T06:37:28.979 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.991+0000 7f57df7fe640 1 -- 172.21.15.3:0/3258956125 learned_addr learned my addr 172.21.15.3:0/3258956125 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:37:28.979 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.991+0000 7f57deffd640 1 --2- 172.21.15.3:0/3258956125 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f57e00cf090 0x7f57e01539c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:28.979 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.991+0000 7f57de7fc640 1 --2- 172.21.15.3:0/3258956125 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f57e00d4230 0x7f57e009e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:28.979 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.991+0000 7f57df7fe640 1 -- 172.21.15.3:0/3258956125 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f57e00cf090 msgr2=0x7f57e01539c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:28.979 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.991+0000 7f57df7fe640 1 --2- 172.21.15.3:0/3258956125 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f57e00cf090 0x7f57e01539c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:28.979 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.991+0000 7f57df7fe640 1 -- 172.21.15.3:0/3258956125 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f57e00d4230 msgr2=0x7f57e009e900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:28.980 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.991+0000 7f57df7fe640 1 --2- 172.21.15.3:0/3258956125 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f57e00d4230 0x7f57e009e900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:28.980 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.991+0000 7f57df7fe640 1 -- 172.21.15.3:0/3258956125 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57e015eee0 con 0x7f57e015e850 2024-05-09T06:37:28.980 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.991+0000 7f57deffd640 1 --2- 172.21.15.3:0/3258956125 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f57e00cf090 0x7f57e01539c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:37:28.980 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.991+0000 7f57de7fc640 1 --2- 172.21.15.3:0/3258956125 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f57e00d4230 0x7f57e009e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:37:28.980 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.991+0000 7f57df7fe640 1 --2- 172.21.15.3:0/3258956125 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f57e015e850 0x7f57e00b7840 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f57d80098c0 tx=0x7f57d80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:28.980 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.991+0000 7f57c77fe640 1 -- 172.21.15.3:0/3258956125 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57d8037040 con 0x7f57e015e850 2024-05-09T06:37:28.980 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.991+0000 7f57c77fe640 1 -- 172.21.15.3:0/3258956125 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57d8042070 con 0x7f57e015e850 2024-05-09T06:37:28.980 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.991+0000 7f57e5638640 1 -- 172.21.15.3:0/3258956125 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57e015f1c0 con 0x7f57e015e850 2024-05-09T06:37:28.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.991+0000 7f57c77fe640 1 -- 172.21.15.3:0/3258956125 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57d8030b50 con 0x7f57e015e850 2024-05-09T06:37:28.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.991+0000 7f57e5638640 1 -- 172.21.15.3:0/3258956125 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57e015f780 con 0x7f57e015e850 2024-05-09T06:37:28.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.992+0000 7f57e5638640 1 -- 172.21.15.3:0/3258956125 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f57e00b8220 con 0x7f57e015e850 2024-05-09T06:37:28.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.993+0000 7f57c77fe640 1 -- 172.21.15.3:0/3258956125 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 7) ==== 90651+0+0 (secure 0 0 0) 0x7f57d8030cf0 con 0x7f57e015e850 2024-05-09T06:37:28.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.993+0000 7f57c77fe640 1 --2- 172.21.15.3:0/3258956125 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f57cc06bfe0 0x7f57cc06e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:28.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.993+0000 7f57c77fe640 1 -- 172.21.15.3:0/3258956125 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 4943+0+0 (secure 0 0 0) 0x7f57d80ab9f0 con 0x7f57e015e850 2024-05-09T06:37:28.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.993+0000 7f57c77fe640 1 --2- 172.21.15.3:0/3258956125 >> [v2:172.21.15.3:6808/801295451,v1:172.21.15.3:6809/801295451] conn(0x7f57cc071ea0 0x7f57cc0742c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:28.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.993+0000 7f57c77fe640 1 -- 172.21.15.3:0/3258956125 --> [v2:172.21.15.3:6808/801295451,v1:172.21.15.3:6809/801295451] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f57d8045e80 con 0x7f57cc071ea0 2024-05-09T06:37:28.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.993+0000 7f57c77fe640 1 -- 172.21.15.3:0/3258956125 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f57d80b02a0 con 0x7f57e015e850 2024-05-09T06:37:28.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.993+0000 7f57deffd640 1 --2- 172.21.15.3:0/3258956125 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f57cc06bfe0 0x7f57cc06e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:28.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.993+0000 7f57de7fc640 1 --2- 172.21.15.3:0/3258956125 >> [v2:172.21.15.3:6808/801295451,v1:172.21.15.3:6809/801295451] conn(0x7f57cc071ea0 0x7f57cc0742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:28.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.994+0000 7f57de7fc640 1 --2- 172.21.15.3:0/3258956125 >> [v2:172.21.15.3:6808/801295451,v1:172.21.15.3:6809/801295451] conn(0x7f57cc071ea0 0x7f57cc0742c0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:28.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.994+0000 7f57deffd640 1 --2- 172.21.15.3:0/3258956125 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f57cc06bfe0 0x7f57cc06e4a0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f57d000d430 tx=0x7f57d0007530 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:28.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:28.994+0000 7f57c77fe640 1 -- 172.21.15.3:0/3258956125 <== osd.2 v2:172.21.15.3:6808/801295451 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f57d8045e80 con 0x7f57cc071ea0 2024-05-09T06:37:28.988 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.008+0000 7f57e5638640 1 -- 172.21.15.3:0/3258956125 --> [v2:172.21.15.3:6808/801295451,v1:172.21.15.3:6809/801295451] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f57e00682e0 con 0x7f57cc071ea0 2024-05-09T06:37:28.989 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.008+0000 7f57c77fe640 1 -- 172.21.15.3:0/3258956125 <== osd.2 v2:172.21.15.3:6808/801295451 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f57e00682e0 con 0x7f57cc071ea0 2024-05-09T06:37:28.989 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-05-09T06:37:28.989 INFO:teuthology.orchestra.run.smithi003.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-09T06:37:28.989 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-05-09T06:37:28.990 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.008+0000 7f57e5638640 1 -- 172.21.15.3:0/3258956125 >> [v2:172.21.15.3:6808/801295451,v1:172.21.15.3:6809/801295451] conn(0x7f57cc071ea0 msgr2=0x7f57cc0742c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:28.990 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.008+0000 7f57e5638640 1 --2- 172.21.15.3:0/3258956125 >> [v2:172.21.15.3:6808/801295451,v1:172.21.15.3:6809/801295451] conn(0x7f57cc071ea0 0x7f57cc0742c0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:28.990 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.008+0000 7f57e5638640 1 -- 172.21.15.3:0/3258956125 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f57cc06bfe0 msgr2=0x7f57cc06e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:28.990 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.008+0000 7f57e5638640 1 --2- 172.21.15.3:0/3258956125 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f57cc06bfe0 0x7f57cc06e4a0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f57d000d430 tx=0x7f57d0007530 comp rx=0 tx=0).stop 2024-05-09T06:37:28.991 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.008+0000 7f57e5638640 1 -- 172.21.15.3:0/3258956125 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f57e015e850 msgr2=0x7f57e00b7840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:28.991 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.008+0000 7f57e5638640 1 --2- 172.21.15.3:0/3258956125 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f57e015e850 0x7f57e00b7840 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f57d80098c0 tx=0x7f57d80046c0 comp rx=0 tx=0).stop 2024-05-09T06:37:28.991 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.008+0000 7f57df7fe640 1 -- 172.21.15.3:0/3258956125 reap_dead start 2024-05-09T06:37:28.991 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.009+0000 7f57e5638640 1 -- 172.21.15.3:0/3258956125 shutdown_connections 2024-05-09T06:37:28.991 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.009+0000 7f57e5638640 1 -- 172.21.15.3:0/3258956125 >> 172.21.15.3:0/3258956125 conn(0x7f57e008dc60 msgr2=0x7f57e0074320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:28.991 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.009+0000 7f57e5638640 1 -- 172.21.15.3:0/3258956125 shutdown_connections 2024-05-09T06:37:28.991 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.009+0000 7f57e5638640 1 -- 172.21.15.3:0/3258956125 wait complete. 2024-05-09T06:37:29.000 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-05-09T06:37:29.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.124+0000 7f52ec609640 1 Processor -- start 2024-05-09T06:37:29.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.124+0000 7f52ec609640 1 -- start start 2024-05-09T06:37:29.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.125+0000 7f52ec609640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f52e415c0f0 0x7f52e4160730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:29.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.125+0000 7f52ec609640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f52e40b4e30 0x7f52e406e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:29.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.125+0000 7f52ec609640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f52e4160c70 con 0x7f52e406e790 2024-05-09T06:37:29.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.125+0000 7f52ec609640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f52e406ec00 con 0x7f52e415c0f0 2024-05-09T06:37:29.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.125+0000 7f52ec609640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f52e40b40f0 con 0x7f52e40b4e30 2024-05-09T06:37:29.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.125+0000 7f52ea3a5640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f52e415c0f0 0x7f52e4160730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:29.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.125+0000 7f52ea3a5640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f52e415c0f0 0x7f52e4160730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40836/0 (socket says 172.21.15.3:40836) 2024-05-09T06:37:29.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.125+0000 7f52eaba6640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f52e406e790 0x7f52e40cd5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:43144/0 (socket says 172.21.15.3:43144) 2024-05-09T06:37:29.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.125+0000 7f52ea3a5640 1 -- 172.21.15.3:0/2588420749 learned_addr learned my addr 172.21.15.3:0/2588420749 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:37:29.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.126+0000 7f52e9ba4640 1 --2- 172.21.15.3:0/2588420749 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f52e40b4e30 0x7f52e406e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:29.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.126+0000 7f52ea3a5640 1 -- 172.21.15.3:0/2588420749 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f52e40b4e30 msgr2=0x7f52e406e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:37:29.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.126+0000 7f52ea3a5640 1 --2- 172.21.15.3:0/2588420749 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f52e40b4e30 0x7f52e406e250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:29.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.126+0000 7f52ea3a5640 1 -- 172.21.15.3:0/2588420749 >> v1:172.21.15.3:6789/0 conn(0x7f52e406e790 legacy=0x7f52e40cd5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:29.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.126+0000 7f52ea3a5640 1 -- 172.21.15.3:0/2588420749 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52e40b3050 con 0x7f52e415c0f0 2024-05-09T06:37:29.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.126+0000 7f52e93a3640 1 -- 172.21.15.3:0/2588420749 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1857699098 0 0) 0x7f52d4001490 con 0x7f52e406e790 2024-05-09T06:37:29.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.126+0000 7f52e9ba4640 1 --2- 172.21.15.3:0/2588420749 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f52e40b4e30 0x7f52e406e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:37:29.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.126+0000 7f52ea3a5640 1 --2- 172.21.15.3:0/2588420749 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f52e415c0f0 0x7f52e4160730 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f52dc009870 tx=0x7f52dc02a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b279a617572acd8e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:29.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.127+0000 7f52e93a3640 1 -- 172.21.15.3:0/2588420749 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52dc037070 con 0x7f52e415c0f0 2024-05-09T06:37:29.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.127+0000 7f52e93a3640 1 -- 172.21.15.3:0/2588420749 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52dc032440 con 0x7f52e415c0f0 2024-05-09T06:37:29.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.127+0000 7f52e93a3640 1 -- 172.21.15.3:0/2588420749 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52dc032810 con 0x7f52e415c0f0 2024-05-09T06:37:29.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.127+0000 7f52ec609640 1 -- 172.21.15.3:0/2588420749 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f52e415c0f0 msgr2=0x7f52e4160730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:29.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.127+0000 7f52ec609640 1 --2- 172.21.15.3:0/2588420749 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f52e415c0f0 0x7f52e4160730 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f52dc009870 tx=0x7f52dc02a1c0 comp rx=0 tx=0).stop 2024-05-09T06:37:29.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.127+0000 7f52ec609640 1 -- 172.21.15.3:0/2588420749 shutdown_connections 2024-05-09T06:37:29.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.127+0000 7f52ec609640 1 --2- 172.21.15.3:0/2588420749 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f52e40b4e30 0x7f52e406e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:29.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.127+0000 7f52ec609640 1 --2- 172.21.15.3:0/2588420749 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f52e415c0f0 0x7f52e4160730 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:29.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.127+0000 7f52ec609640 1 -- 172.21.15.3:0/2588420749 >> 172.21.15.3:0/2588420749 conn(0x7f52e40b1c20 msgr2=0x7f52e40b2030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:29.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.127+0000 7f52ec609640 1 -- 172.21.15.3:0/2588420749 shutdown_connections 2024-05-09T06:37:29.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.127+0000 7f52ec609640 1 -- 172.21.15.3:0/2588420749 wait complete. 2024-05-09T06:37:29.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.128+0000 7f52ec609640 1 Processor -- start 2024-05-09T06:37:29.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.129+0000 7f52ec609640 1 -- start start 2024-05-09T06:37:29.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.129+0000 7f52ec609640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f52e406e790 0x7f52e413f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:29.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.129+0000 7f52ec609640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f52e40b4e30 0x7f52e4143460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:29.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.129+0000 7f52ec609640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f52e415c0f0 0x7f52e41414b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:29.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.129+0000 7f52ec609640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f52e40b38f0 con 0x7f52e406e790 2024-05-09T06:37:29.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.129+0000 7f52ec609640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f52e40b3610 con 0x7f52e40b4e30 2024-05-09T06:37:29.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.129+0000 7f52ec609640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f52e40b3bd0 con 0x7f52e415c0f0 2024-05-09T06:37:29.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.129+0000 7f52ea3a5640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f52e406e790 0x7f52e413f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:29.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.129+0000 7f52ea3a5640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f52e406e790 0x7f52e413f7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40838/0 (socket says 172.21.15.3:40838) 2024-05-09T06:37:29.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.129+0000 7f52ea3a5640 1 -- 172.21.15.3:0/3153367167 learned_addr learned my addr 172.21.15.3:0/3153367167 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:37:29.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.130+0000 7f52e9ba4640 1 --2- 172.21.15.3:0/3153367167 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f52e40b4e30 0x7f52e4143460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:29.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.130+0000 7f52eaba6640 1 --2- 172.21.15.3:0/3153367167 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f52e415c0f0 0x7f52e41414b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:29.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.130+0000 7f52ea3a5640 1 -- 172.21.15.3:0/3153367167 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f52e415c0f0 msgr2=0x7f52e41414b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:29.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.130+0000 7f52ea3a5640 1 --2- 172.21.15.3:0/3153367167 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f52e415c0f0 0x7f52e41414b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:29.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.130+0000 7f52ea3a5640 1 -- 172.21.15.3:0/3153367167 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f52e40b4e30 msgr2=0x7f52e4143460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:29.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.130+0000 7f52ea3a5640 1 --2- 172.21.15.3:0/3153367167 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f52e40b4e30 0x7f52e4143460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:29.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.130+0000 7f52ea3a5640 1 -- 172.21.15.3:0/3153367167 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52e4140410 con 0x7f52e406e790 2024-05-09T06:37:29.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.130+0000 7f52eaba6640 1 --2- 172.21.15.3:0/3153367167 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f52e415c0f0 0x7f52e41414b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:37:29.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.130+0000 7f52ea3a5640 1 --2- 172.21.15.3:0/3153367167 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f52e406e790 0x7f52e413f7e0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f52dc035040 tx=0x7f52dc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:29.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.130+0000 7f52e2ffd640 1 -- 172.21.15.3:0/3153367167 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52dc037040 con 0x7f52e406e790 2024-05-09T06:37:29.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.130+0000 7f52e2ffd640 1 -- 172.21.15.3:0/3153367167 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52dc004230 con 0x7f52e406e790 2024-05-09T06:37:29.114 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.130+0000 7f52e2ffd640 1 -- 172.21.15.3:0/3153367167 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52dc003610 con 0x7f52e406e790 2024-05-09T06:37:29.114 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.130+0000 7f52ec609640 1 -- 172.21.15.3:0/3153367167 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52e41406f0 con 0x7f52e406e790 2024-05-09T06:37:29.114 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.130+0000 7f52ec609640 1 -- 172.21.15.3:0/3153367167 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52e4140cb0 con 0x7f52e406e790 2024-05-09T06:37:29.114 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.131+0000 7f52ec609640 1 -- 172.21.15.3:0/3153367167 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52e4069550 con 0x7f52e406e790 2024-05-09T06:37:29.114 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.132+0000 7f52e2ffd640 1 -- 172.21.15.3:0/3153367167 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 7) ==== 90651+0+0 (secure 0 0 0) 0x7f52dc02ad10 con 0x7f52e406e790 2024-05-09T06:37:29.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.132+0000 7f52e2ffd640 1 --2- 172.21.15.3:0/3153367167 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f52d006bfe0 0x7f52d006e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:29.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.132+0000 7f52e2ffd640 1 -- 172.21.15.3:0/3153367167 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 4943+0+0 (secure 0 0 0) 0x7f52dc0abad0 con 0x7f52e406e790 2024-05-09T06:37:29.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.132+0000 7f52e9ba4640 1 --2- 172.21.15.3:0/3153367167 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f52d006bfe0 0x7f52d006e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:29.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.133+0000 7f52e9ba4640 1 --2- 172.21.15.3:0/3153367167 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f52d006bfe0 0x7f52d006e4a0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f52e4146680 tx=0x7f52d8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:29.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.137+0000 7f52e2ffd640 1 -- 172.21.15.3:0/3153367167 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f52dc07a620 con 0x7f52e406e790 2024-05-09T06:37:29.302 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.321+0000 7f52ec609640 1 -- 172.21.15.3:0/3153367167 --> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) -- 0x7f52e40b42e0 con 0x7f52d006bfe0 2024-05-09T06:37:29.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.322+0000 7f52e2ffd640 1 -- 172.21.15.3:0/3153367167 <== mgr.4105 v2:172.21.15.190:6832/975893321 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f52e40b42e0 con 0x7f52d006bfe0 2024-05-09T06:37:29.303 INFO:teuthology.orchestra.run.smithi003.stderr:instructed osd(s) 2 to deep-scrub 2024-05-09T06:37:29.305 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.324+0000 7f52ec609640 1 -- 172.21.15.3:0/3153367167 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f52d006bfe0 msgr2=0x7f52d006e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:29.305 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.324+0000 7f52ec609640 1 --2- 172.21.15.3:0/3153367167 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f52d006bfe0 0x7f52d006e4a0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f52e4146680 tx=0x7f52d8008040 comp rx=0 tx=0).stop 2024-05-09T06:37:29.305 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.324+0000 7f52ec609640 1 -- 172.21.15.3:0/3153367167 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f52e406e790 msgr2=0x7f52e413f7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:29.305 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.324+0000 7f52ec609640 1 --2- 172.21.15.3:0/3153367167 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f52e406e790 0x7f52e413f7e0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f52dc035040 tx=0x7f52dc0046c0 comp rx=0 tx=0).stop 2024-05-09T06:37:29.305 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.324+0000 7f52ec609640 1 -- 172.21.15.3:0/3153367167 shutdown_connections 2024-05-09T06:37:29.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.324+0000 7f52ec609640 1 --2- 172.21.15.3:0/3153367167 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f52d006bfe0 0x7f52d006e4a0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:29.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.324+0000 7f52ec609640 1 --2- 172.21.15.3:0/3153367167 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f52e415c0f0 0x7f52e41414b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:29.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.324+0000 7f52ec609640 1 --2- 172.21.15.3:0/3153367167 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f52e40b4e30 0x7f52e4143460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:29.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.324+0000 7f52ec609640 1 --2- 172.21.15.3:0/3153367167 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f52e406e790 0x7f52e413f7e0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:29.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.324+0000 7f52ec609640 1 -- 172.21.15.3:0/3153367167 >> 172.21.15.3:0/3153367167 conn(0x7f52e40b1c20 msgr2=0x7f52e40711f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:29.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.324+0000 7f52ec609640 1 -- 172.21.15.3:0/3153367167 shutdown_connections 2024-05-09T06:37:29.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.324+0000 7f52ec609640 1 -- 172.21.15.3:0/3153367167 wait complete. 2024-05-09T06:37:29.316 INFO:tasks.ceph:Scrubbing osd.3 2024-05-09T06:37:29.316 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-05-09T06:37:29.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.444+0000 7f6267961640 1 Processor -- start 2024-05-09T06:37:29.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.444+0000 7f6267961640 1 -- start start 2024-05-09T06:37:29.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.445+0000 7f6267961640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f62601458b0 0x7f626015c690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:29.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.445+0000 7f6267961640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f62600b4410 0x7f62600cd7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:29.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.445+0000 7f6267961640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f62600ce0f0 con 0x7f62600cdcf0 2024-05-09T06:37:29.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.445+0000 7f6267961640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f62600b48c0 con 0x7f62600b4410 2024-05-09T06:37:29.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.445+0000 7f6267961640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f6260145c90 con 0x7f62601458b0 2024-05-09T06:37:29.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.445+0000 7f6264efc640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f62600b4410 0x7f62600cd7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:29.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.445+0000 7f6265efe640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f62600cdcf0 0x7f626015fc80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:43154/0 (socket says 172.21.15.3:43154) 2024-05-09T06:37:29.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.445+0000 7f6265efe640 1 -- 172.21.15.3:0/3458230407 learned_addr learned my addr 172.21.15.3:0/3458230407 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-05-09T06:37:29.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.445+0000 7f62656fd640 1 --2- 172.21.15.3:0/3458230407 >> v2:172.21.15.190:3300/0 conn(0x7f62601458b0 0x7f626015c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:29.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.445+0000 7f6264efc640 1 -- 172.21.15.3:0/3458230407 >> v2:172.21.15.190:3300/0 conn(0x7f62601458b0 msgr2=0x7f626015c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:37:29.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.445+0000 7f6264efc640 1 --2- 172.21.15.3:0/3458230407 >> v2:172.21.15.190:3300/0 conn(0x7f62601458b0 0x7f626015c690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:29.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.445+0000 7f6264efc640 1 -- 172.21.15.3:0/3458230407 >> v1:172.21.15.3:6789/0 conn(0x7f62600cdcf0 legacy=0x7f626015fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:29.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.445+0000 7f6264efc640 1 -- 172.21.15.3:0/3458230407 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62600b4b60 con 0x7f62600b4410 2024-05-09T06:37:29.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.445+0000 7f6257fff640 1 -- 172.21.15.3:0/3458230407 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1523177272 0 0) 0x7f62600ce0f0 con 0x7f62600cdcf0 2024-05-09T06:37:29.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.446+0000 7f6264efc640 1 --2- 172.21.15.3:0/3458230407 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f62600b4410 0x7f62600cd7b0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f626006f510 tx=0x7f625002a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2123241f0fa4f340 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:29.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.446+0000 7f6257fff640 1 -- 172.21.15.3:0/3458230407 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6250037070 con 0x7f62600b4410 2024-05-09T06:37:29.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.446+0000 7f6257fff640 1 -- 172.21.15.3:0/3458230407 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6250032440 con 0x7f62600b4410 2024-05-09T06:37:29.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.446+0000 7f6257fff640 1 -- 172.21.15.3:0/3458230407 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f625002ae60 con 0x7f62600b4410 2024-05-09T06:37:29.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.447+0000 7f6267961640 1 -- 172.21.15.3:0/3458230407 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f62600b4410 msgr2=0x7f62600cd7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:29.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.447+0000 7f6267961640 1 --2- 172.21.15.3:0/3458230407 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f62600b4410 0x7f62600cd7b0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f626006f510 tx=0x7f625002a110 comp rx=0 tx=0).stop 2024-05-09T06:37:29.432 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.447+0000 7f6267961640 1 -- 172.21.15.3:0/3458230407 shutdown_connections 2024-05-09T06:37:29.432 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.447+0000 7f6267961640 1 --2- 172.21.15.3:0/3458230407 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f62600b4410 0x7f62600cd7b0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:29.432 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.447+0000 7f6267961640 1 --2- 172.21.15.3:0/3458230407 >> v2:172.21.15.190:3300/0 conn(0x7f62601458b0 0x7f626015c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:29.432 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.447+0000 7f6267961640 1 -- 172.21.15.3:0/3458230407 >> 172.21.15.3:0/3458230407 conn(0x7f626006b640 msgr2=0x7f626006ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:29.432 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.447+0000 7f6267961640 1 -- 172.21.15.3:0/3458230407 shutdown_connections 2024-05-09T06:37:29.432 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.447+0000 7f6267961640 1 -- 172.21.15.3:0/3458230407 wait complete. 2024-05-09T06:37:29.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.448+0000 7f6267961640 1 Processor -- start 2024-05-09T06:37:29.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.448+0000 7f6267961640 1 -- start start 2024-05-09T06:37:29.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.449+0000 7f6267961640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f62600b4410 0x7f62601f6e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:29.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.449+0000 7f6267961640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f62600cdcf0 0x7f626012dff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:29.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.449+0000 7f6267961640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f62601458b0 0x7f626013d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:29.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.449+0000 7f6267961640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f62600b5400 con 0x7f62600b4410 2024-05-09T06:37:29.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.449+0000 7f6267961640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f62600b5120 con 0x7f62601458b0 2024-05-09T06:37:29.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.449+0000 7f6267961640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f626006d6b0 con 0x7f62600cdcf0 2024-05-09T06:37:29.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.449+0000 7f62656fd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f62600b4410 0x7f62601f6e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:29.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.449+0000 7f62656fd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f62600b4410 0x7f62601f6e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40854/0 (socket says 172.21.15.3:40854) 2024-05-09T06:37:29.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.449+0000 7f62656fd640 1 -- 172.21.15.3:0/1752799213 learned_addr learned my addr 172.21.15.3:0/1752799213 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:37:29.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.449+0000 7f6265efe640 1 --2- 172.21.15.3:0/1752799213 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f62601458b0 0x7f626013d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:29.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.449+0000 7f6264efc640 1 --2- 172.21.15.3:0/1752799213 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f62600cdcf0 0x7f626012dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:29.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.449+0000 7f62656fd640 1 -- 172.21.15.3:0/1752799213 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f62600cdcf0 msgr2=0x7f626012dff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:29.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.449+0000 7f62656fd640 1 --2- 172.21.15.3:0/1752799213 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f62600cdcf0 0x7f626012dff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:29.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.449+0000 7f62656fd640 1 -- 172.21.15.3:0/1752799213 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f62601458b0 msgr2=0x7f626013d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:29.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.449+0000 7f62656fd640 1 --2- 172.21.15.3:0/1752799213 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f62601458b0 0x7f626013d900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:29.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.449+0000 7f62656fd640 1 -- 172.21.15.3:0/1752799213 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f626012e820 con 0x7f62600b4410 2024-05-09T06:37:29.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.449+0000 7f6264efc640 1 --2- 172.21.15.3:0/1752799213 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f62600cdcf0 0x7f626012dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:37:29.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.449+0000 7f62656fd640 1 --2- 172.21.15.3:0/1752799213 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f62600b4410 0x7f62601f6e60 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f625c00d5f0 tx=0x7f625c00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:29.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.450+0000 7f6255ffb640 1 -- 172.21.15.3:0/1752799213 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f625c0125f0 con 0x7f62600b4410 2024-05-09T06:37:29.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.450+0000 7f6255ffb640 1 -- 172.21.15.3:0/1752799213 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f625c019070 con 0x7f62600b4410 2024-05-09T06:37:29.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.450+0000 7f6267961640 1 -- 172.21.15.3:0/1752799213 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f626012eb00 con 0x7f62600b4410 2024-05-09T06:37:29.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.450+0000 7f6255ffb640 1 -- 172.21.15.3:0/1752799213 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f625c0129d0 con 0x7f62600b4410 2024-05-09T06:37:29.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.450+0000 7f6267961640 1 -- 172.21.15.3:0/1752799213 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f626013ecb0 con 0x7f62600b4410 2024-05-09T06:37:29.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.451+0000 7f6267961640 1 -- 172.21.15.3:0/1752799213 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f62601f73a0 con 0x7f62600b4410 2024-05-09T06:37:29.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.451+0000 7f6255ffb640 1 -- 172.21.15.3:0/1752799213 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 7) ==== 90651+0+0 (secure 0 0 0) 0x7f625c012b70 con 0x7f62600b4410 2024-05-09T06:37:29.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.452+0000 7f6255ffb640 1 --2- 172.21.15.3:0/1752799213 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f624406bfe0 0x7f624406e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:29.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.452+0000 7f6255ffb640 1 -- 172.21.15.3:0/1752799213 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 4943+0+0 (secure 0 0 0) 0x7f625c08bc20 con 0x7f62600b4410 2024-05-09T06:37:29.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.452+0000 7f6255ffb640 1 --2- 172.21.15.3:0/1752799213 >> [v2:172.21.15.3:6824/1903389857,v1:172.21.15.3:6825/1903389857] conn(0x7f6244071ea0 0x7f62440742c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:29.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.452+0000 7f6255ffb640 1 -- 172.21.15.3:0/1752799213 --> [v2:172.21.15.3:6824/1903389857,v1:172.21.15.3:6825/1903389857] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f625c00cfb0 con 0x7f6244071ea0 2024-05-09T06:37:29.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.452+0000 7f6255ffb640 1 -- 172.21.15.3:0/1752799213 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f625c08c090 con 0x7f62600b4410 2024-05-09T06:37:29.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.452+0000 7f6264efc640 1 --2- 172.21.15.3:0/1752799213 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f624406bfe0 0x7f624406e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:29.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.452+0000 7f6265efe640 1 --2- 172.21.15.3:0/1752799213 >> [v2:172.21.15.3:6824/1903389857,v1:172.21.15.3:6825/1903389857] conn(0x7f6244071ea0 0x7f62440742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:29.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.453+0000 7f6265efe640 1 --2- 172.21.15.3:0/1752799213 >> [v2:172.21.15.3:6824/1903389857,v1:172.21.15.3:6825/1903389857] conn(0x7f6244071ea0 0x7f62440742c0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:29.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.453+0000 7f6264efc640 1 --2- 172.21.15.3:0/1752799213 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f624406bfe0 0x7f624406e4a0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f6250034000 tx=0x7f625002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:29.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.453+0000 7f6255ffb640 1 -- 172.21.15.3:0/1752799213 <== osd.3 v2:172.21.15.3:6824/1903389857 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f625c00cfb0 con 0x7f6244071ea0 2024-05-09T06:37:29.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.466+0000 7f6267961640 1 -- 172.21.15.3:0/1752799213 --> [v2:172.21.15.3:6824/1903389857,v1:172.21.15.3:6825/1903389857] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f6260069550 con 0x7f6244071ea0 2024-05-09T06:37:29.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.467+0000 7f6255ffb640 1 -- 172.21.15.3:0/1752799213 <== osd.3 v2:172.21.15.3:6824/1903389857 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f6260069550 con 0x7f6244071ea0 2024-05-09T06:37:29.447 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-05-09T06:37:29.448 INFO:teuthology.orchestra.run.smithi003.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-09T06:37:29.448 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-05-09T06:37:29.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.467+0000 7f6267961640 1 -- 172.21.15.3:0/1752799213 >> [v2:172.21.15.3:6824/1903389857,v1:172.21.15.3:6825/1903389857] conn(0x7f6244071ea0 msgr2=0x7f62440742c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:29.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.467+0000 7f6267961640 1 --2- 172.21.15.3:0/1752799213 >> [v2:172.21.15.3:6824/1903389857,v1:172.21.15.3:6825/1903389857] conn(0x7f6244071ea0 0x7f62440742c0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:29.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.467+0000 7f6267961640 1 -- 172.21.15.3:0/1752799213 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f624406bfe0 msgr2=0x7f624406e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:29.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.467+0000 7f6267961640 1 --2- 172.21.15.3:0/1752799213 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f624406bfe0 0x7f624406e4a0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f6250034000 tx=0x7f625002e000 comp rx=0 tx=0).stop 2024-05-09T06:37:29.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.467+0000 7f6267961640 1 -- 172.21.15.3:0/1752799213 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f62600b4410 msgr2=0x7f62601f6e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:29.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.467+0000 7f6267961640 1 --2- 172.21.15.3:0/1752799213 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f62600b4410 0x7f62601f6e60 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f625c00d5f0 tx=0x7f625c00dab0 comp rx=0 tx=0).stop 2024-05-09T06:37:29.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.467+0000 7f6265efe640 1 -- 172.21.15.3:0/1752799213 reap_dead start 2024-05-09T06:37:29.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.467+0000 7f6267961640 1 -- 172.21.15.3:0/1752799213 shutdown_connections 2024-05-09T06:37:29.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.467+0000 7f6267961640 1 -- 172.21.15.3:0/1752799213 >> 172.21.15.3:0/1752799213 conn(0x7f626006b640 msgr2=0x7f626006e2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:29.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.467+0000 7f6267961640 1 -- 172.21.15.3:0/1752799213 shutdown_connections 2024-05-09T06:37:29.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.467+0000 7f6267961640 1 -- 172.21.15.3:0/1752799213 wait complete. 2024-05-09T06:37:29.458 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-05-09T06:37:29.564 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.583+0000 7ffbe5261640 1 Processor -- start 2024-05-09T06:37:29.564 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.583+0000 7ffbe5261640 1 -- start start 2024-05-09T06:37:29.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.584+0000 7ffbe5261640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ffbe015c0f0 0x7ffbe0160730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:29.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.584+0000 7ffbe5261640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffbe0160c70 con 0x7ffbe006e960 2024-05-09T06:37:29.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.584+0000 7ffbe5261640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffbe00b40f0 con 0x7ffbe00b4e30 2024-05-09T06:37:29.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.584+0000 7ffbe5261640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7ffbe00b3050 con 0x7ffbe015c0f0 2024-05-09T06:37:29.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.584+0000 7ffbdeffd640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ffbe015c0f0 0x7ffbe0160730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:29.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.584+0000 7ffbdeffd640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ffbe015c0f0 0x7ffbe0160730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40858/0 (socket says 172.21.15.3:40858) 2024-05-09T06:37:29.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.584+0000 7ffbdeffd640 1 -- 172.21.15.3:0/4002989490 learned_addr learned my addr 172.21.15.3:0/4002989490 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:37:29.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.584+0000 7ffbdf7fe640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7ffbe006e960 0x7ffbe015c4d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:43168/0 (socket says 172.21.15.3:43168) 2024-05-09T06:37:29.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.584+0000 7ffbdeffd640 1 -- 172.21.15.3:0/4002989490 >> v1:172.21.15.190:6789/0 conn(0x7ffbe00b4e30 legacy=0x7ffbe006e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:37:29.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.584+0000 7ffbdeffd640 1 -- 172.21.15.3:0/4002989490 >> v1:172.21.15.3:6789/0 conn(0x7ffbe006e960 legacy=0x7ffbe015c4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:29.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.584+0000 7ffbdeffd640 1 -- 172.21.15.3:0/4002989490 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbe00b3330 con 0x7ffbe015c0f0 2024-05-09T06:37:29.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.585+0000 7ffbdeffd640 1 --2- 172.21.15.3:0/4002989490 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffbe015c0f0 0x7ffbe0160730 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7ffbd0009870 tx=0x7ffbd002a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb50576261f40b03 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:29.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.585+0000 7ffbde7fc640 1 -- 172.21.15.3:0/4002989490 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbd0037070 con 0x7ffbe015c0f0 2024-05-09T06:37:29.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.585+0000 7ffbde7fc640 1 -- 172.21.15.3:0/4002989490 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffbd0032440 con 0x7ffbe015c0f0 2024-05-09T06:37:29.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.585+0000 7ffbde7fc640 1 -- 172.21.15.3:0/4002989490 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbd0032810 con 0x7ffbe015c0f0 2024-05-09T06:37:29.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.586+0000 7ffbe5261640 1 -- 172.21.15.3:0/4002989490 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffbe015c0f0 msgr2=0x7ffbe0160730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:29.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.586+0000 7ffbe5261640 1 --2- 172.21.15.3:0/4002989490 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffbe015c0f0 0x7ffbe0160730 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7ffbd0009870 tx=0x7ffbd002a210 comp rx=0 tx=0).stop 2024-05-09T06:37:29.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.586+0000 7ffbe5261640 1 -- 172.21.15.3:0/4002989490 shutdown_connections 2024-05-09T06:37:29.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.586+0000 7ffbe5261640 1 --2- 172.21.15.3:0/4002989490 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffbe015c0f0 0x7ffbe0160730 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:29.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.586+0000 7ffbe5261640 1 -- 172.21.15.3:0/4002989490 >> 172.21.15.3:0/4002989490 conn(0x7ffbe00b1c20 msgr2=0x7ffbe00b2030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:29.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.586+0000 7ffbe5261640 1 -- 172.21.15.3:0/4002989490 shutdown_connections 2024-05-09T06:37:29.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.586+0000 7ffbe5261640 1 -- 172.21.15.3:0/4002989490 wait complete. 2024-05-09T06:37:29.570 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.587+0000 7ffbe5261640 1 Processor -- start 2024-05-09T06:37:29.570 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.587+0000 7ffbe5261640 1 -- start start 2024-05-09T06:37:29.570 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.588+0000 7ffbe5261640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffbe006e960 0x7ffbe00aed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:29.570 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.588+0000 7ffbe5261640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffbe00b4e30 0x7ffbe014db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:29.570 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.588+0000 7ffbe5261640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffbe015c0f0 0x7ffbe014e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:29.570 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.588+0000 7ffbe5261640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7ffbe00b3bd0 con 0x7ffbe015c0f0 2024-05-09T06:37:29.570 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.588+0000 7ffbe5261640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7ffbe00b38f0 con 0x7ffbe006e960 2024-05-09T06:37:29.570 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.588+0000 7ffbe5261640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7ffbe015cbe0 con 0x7ffbe00b4e30 2024-05-09T06:37:29.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.588+0000 7ffbdf7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffbe015c0f0 0x7ffbe014e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:29.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.588+0000 7ffbdf7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffbe015c0f0 0x7ffbe014e0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40874/0 (socket says 172.21.15.3:40874) 2024-05-09T06:37:29.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.588+0000 7ffbdf7fe640 1 -- 172.21.15.3:0/4229011862 learned_addr learned my addr 172.21.15.3:0/4229011862 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:37:29.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.588+0000 7ffbd7fff640 1 --2- 172.21.15.3:0/4229011862 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffbe00b4e30 0x7ffbe014db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:29.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.588+0000 7ffbdeffd640 1 --2- 172.21.15.3:0/4229011862 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffbe006e960 0x7ffbe00aed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:29.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.588+0000 7ffbdf7fe640 1 -- 172.21.15.3:0/4229011862 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffbe00b4e30 msgr2=0x7ffbe014db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:29.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.588+0000 7ffbdf7fe640 1 --2- 172.21.15.3:0/4229011862 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffbe00b4e30 0x7ffbe014db70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:29.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.588+0000 7ffbdf7fe640 1 -- 172.21.15.3:0/4229011862 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffbe006e960 msgr2=0x7ffbe00aed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:29.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.588+0000 7ffbdf7fe640 1 --2- 172.21.15.3:0/4229011862 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffbe006e960 0x7ffbe00aed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:29.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.588+0000 7ffbdf7fe640 1 -- 172.21.15.3:0/4229011862 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbe014b540 con 0x7ffbe015c0f0 2024-05-09T06:37:29.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.588+0000 7ffbdf7fe640 1 --2- 172.21.15.3:0/4229011862 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffbe015c0f0 0x7ffbe014e0b0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7ffbd800c760 tx=0x7ffbd800cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:29.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.588+0000 7ffbd77fe640 1 -- 172.21.15.3:0/4229011862 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbd8017070 con 0x7ffbe015c0f0 2024-05-09T06:37:29.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.588+0000 7ffbd77fe640 1 -- 172.21.15.3:0/4229011862 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffbd800a5e0 con 0x7ffbe015c0f0 2024-05-09T06:37:29.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.588+0000 7ffbd77fe640 1 -- 172.21.15.3:0/4229011862 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbd800a9d0 con 0x7ffbe015c0f0 2024-05-09T06:37:29.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.588+0000 7ffbe5261640 1 -- 172.21.15.3:0/4229011862 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffbe014b820 con 0x7ffbe015c0f0 2024-05-09T06:37:29.572 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.589+0000 7ffbe5261640 1 -- 172.21.15.3:0/4229011862 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffbe014c0c0 con 0x7ffbe015c0f0 2024-05-09T06:37:29.572 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.590+0000 7ffbe5261640 1 -- 172.21.15.3:0/4229011862 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffbe006f260 con 0x7ffbe015c0f0 2024-05-09T06:37:29.572 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.590+0000 7ffbd77fe640 1 -- 172.21.15.3:0/4229011862 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 7) ==== 90651+0+0 (secure 0 0 0) 0x7ffbd801a070 con 0x7ffbe015c0f0 2024-05-09T06:37:29.572 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.590+0000 7ffbd77fe640 1 --2- 172.21.15.3:0/4229011862 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7ffbc006bfe0 0x7ffbc006e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:29.573 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.591+0000 7ffbd77fe640 1 -- 172.21.15.3:0/4229011862 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 4943+0+0 (secure 0 0 0) 0x7ffbd808a330 con 0x7ffbe015c0f0 2024-05-09T06:37:29.573 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.591+0000 7ffbdeffd640 1 --2- 172.21.15.3:0/4229011862 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7ffbc006bfe0 0x7ffbc006e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:29.573 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.591+0000 7ffbdeffd640 1 --2- 172.21.15.3:0/4229011862 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7ffbc006bfe0 0x7ffbc006e4a0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7ffbd0034000 tx=0x7ffbd002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:29.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.595+0000 7ffbd77fe640 1 -- 172.21.15.3:0/4229011862 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7ffbe006f260 con 0x7ffbe015c0f0 2024-05-09T06:37:29.758 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.778+0000 7ffbe5261640 1 -- 172.21.15.3:0/4229011862 --> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) -- 0x7ffbe0069550 con 0x7ffbc006bfe0 2024-05-09T06:37:29.759 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.779+0000 7ffbd77fe640 1 -- 172.21.15.3:0/4229011862 <== mgr.4105 v2:172.21.15.190:6832/975893321 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7ffbe0069550 con 0x7ffbc006bfe0 2024-05-09T06:37:29.759 INFO:teuthology.orchestra.run.smithi003.stderr:instructed osd(s) 3 to deep-scrub 2024-05-09T06:37:29.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.781+0000 7ffbe5261640 1 -- 172.21.15.3:0/4229011862 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7ffbc006bfe0 msgr2=0x7ffbc006e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:29.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.781+0000 7ffbe5261640 1 --2- 172.21.15.3:0/4229011862 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7ffbc006bfe0 0x7ffbc006e4a0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7ffbd0034000 tx=0x7ffbd002e000 comp rx=0 tx=0).stop 2024-05-09T06:37:29.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.781+0000 7ffbe5261640 1 -- 172.21.15.3:0/4229011862 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffbe015c0f0 msgr2=0x7ffbe014e0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:29.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.781+0000 7ffbe5261640 1 --2- 172.21.15.3:0/4229011862 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffbe015c0f0 0x7ffbe014e0b0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7ffbd800c760 tx=0x7ffbd800cc20 comp rx=0 tx=0).stop 2024-05-09T06:37:29.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.781+0000 7ffbe5261640 1 -- 172.21.15.3:0/4229011862 shutdown_connections 2024-05-09T06:37:29.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.781+0000 7ffbe5261640 1 --2- 172.21.15.3:0/4229011862 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7ffbc006bfe0 0x7ffbc006e4a0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:29.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.781+0000 7ffbe5261640 1 --2- 172.21.15.3:0/4229011862 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffbe015c0f0 0x7ffbe014e0b0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:29.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.781+0000 7ffbe5261640 1 --2- 172.21.15.3:0/4229011862 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffbe00b4e30 0x7ffbe014db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:29.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.781+0000 7ffbe5261640 1 --2- 172.21.15.3:0/4229011862 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffbe006e960 0x7ffbe00aed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:29.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.781+0000 7ffbe5261640 1 -- 172.21.15.3:0/4229011862 >> 172.21.15.3:0/4229011862 conn(0x7ffbe00b1c20 msgr2=0x7ffbe00ce130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:29.763 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.781+0000 7ffbe5261640 1 -- 172.21.15.3:0/4229011862 shutdown_connections 2024-05-09T06:37:29.763 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.781+0000 7ffbe5261640 1 -- 172.21.15.3:0/4229011862 wait complete. 2024-05-09T06:37:29.773 INFO:tasks.ceph:Scrubbing osd.4 2024-05-09T06:37:29.773 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-05-09T06:37:29.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.901+0000 7f639129a640 1 Processor -- start 2024-05-09T06:37:29.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.901+0000 7f639129a640 1 -- start start 2024-05-09T06:37:29.885 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.902+0000 7f639129a640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f638c0ae0b0 0x7f638c15f280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:29.885 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.902+0000 7f639129a640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f638c06b250 0x7f638c06b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:29.885 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.902+0000 7f639129a640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f638c15f7c0 con 0x7f638c0b56f0 2024-05-09T06:37:29.885 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.902+0000 7f639129a640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f638c06cc80 con 0x7f638c0ae0b0 2024-05-09T06:37:29.886 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.902+0000 7f639129a640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f638c0b5260 con 0x7f638c06b250 2024-05-09T06:37:29.886 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.902+0000 7f638affd640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f638c0ae0b0 0x7f638c15f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:29.886 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.902+0000 7f638affd640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f638c0ae0b0 0x7f638c15f280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40878/0 (socket says 172.21.15.3:40878) 2024-05-09T06:37:29.886 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.902+0000 7f638affd640 1 -- 172.21.15.3:0/345796491 learned_addr learned my addr 172.21.15.3:0/345796491 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:37:29.886 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.902+0000 7f638b7fe640 1 --2- 172.21.15.3:0/345796491 >> v2:172.21.15.190:3300/0 conn(0x7f638c06b250 0x7f638c06b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:29.886 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.902+0000 7f638affd640 1 -- 172.21.15.3:0/345796491 >> v2:172.21.15.190:3300/0 conn(0x7f638c06b250 msgr2=0x7f638c06b6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:37:29.886 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.902+0000 7f638affd640 1 --2- 172.21.15.3:0/345796491 >> v2:172.21.15.190:3300/0 conn(0x7f638c06b250 0x7f638c06b6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:29.886 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.902+0000 7f638affd640 1 -- 172.21.15.3:0/345796491 >> v1:172.21.15.190:6789/0 conn(0x7f638c0b56f0 legacy=0x7f638c0ae490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:37:29.886 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.902+0000 7f638affd640 1 -- 172.21.15.3:0/345796491 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f638c0adb40 con 0x7f638c0ae0b0 2024-05-09T06:37:29.886 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.903+0000 7f638affd640 1 --2- 172.21.15.3:0/345796491 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f638c0ae0b0 0x7f638c15f280 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f638c0b5ba0 tx=0x7f637c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edcf662fdf41dff2 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:29.886 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.903+0000 7f638a7fc640 1 -- 172.21.15.3:0/345796491 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f637c037070 con 0x7f638c0ae0b0 2024-05-09T06:37:29.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.903+0000 7f638a7fc640 1 -- 172.21.15.3:0/345796491 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f637c032440 con 0x7f638c0ae0b0 2024-05-09T06:37:29.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.903+0000 7f638a7fc640 1 -- 172.21.15.3:0/345796491 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f637c032810 con 0x7f638c0ae0b0 2024-05-09T06:37:29.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.904+0000 7f639129a640 1 -- 172.21.15.3:0/345796491 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f638c0ae0b0 msgr2=0x7f638c15f280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:29.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.904+0000 7f639129a640 1 --2- 172.21.15.3:0/345796491 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f638c0ae0b0 0x7f638c15f280 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f638c0b5ba0 tx=0x7f637c02a1a0 comp rx=0 tx=0).stop 2024-05-09T06:37:29.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.904+0000 7f639129a640 1 -- 172.21.15.3:0/345796491 shutdown_connections 2024-05-09T06:37:29.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.904+0000 7f639129a640 1 --2- 172.21.15.3:0/345796491 >> v2:172.21.15.190:3300/0 conn(0x7f638c06b250 0x7f638c06b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:29.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.904+0000 7f639129a640 1 --2- 172.21.15.3:0/345796491 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f638c0ae0b0 0x7f638c15f280 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:29.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.904+0000 7f639129a640 1 -- 172.21.15.3:0/345796491 >> 172.21.15.3:0/345796491 conn(0x7f638c0d05c0 msgr2=0x7f638c0d09d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:29.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.904+0000 7f639129a640 1 -- 172.21.15.3:0/345796491 shutdown_connections 2024-05-09T06:37:29.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.904+0000 7f639129a640 1 -- 172.21.15.3:0/345796491 wait complete. 2024-05-09T06:37:29.891 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.905+0000 7f639129a640 1 Processor -- start 2024-05-09T06:37:29.891 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.905+0000 7f639129a640 1 -- start start 2024-05-09T06:37:29.891 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.906+0000 7f639129a640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f638c06b250 0x7f638c1f4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:29.891 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.906+0000 7f639129a640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f638c0ae0b0 0x7f638c142d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:29.892 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.906+0000 7f639129a640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f638c0b56f0 0x7f638c0cea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:29.892 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.906+0000 7f639129a640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f638c0b4c10 con 0x7f638c0b56f0 2024-05-09T06:37:29.892 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.906+0000 7f639129a640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f638c0b4930 con 0x7f638c06b250 2024-05-09T06:37:29.892 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.906+0000 7f639129a640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f638c0b4ef0 con 0x7f638c0ae0b0 2024-05-09T06:37:29.892 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.906+0000 7f638b7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f638c0b56f0 0x7f638c0cea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:29.892 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.906+0000 7f638b7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f638c0b56f0 0x7f638c0cea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40890/0 (socket says 172.21.15.3:40890) 2024-05-09T06:37:29.892 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.906+0000 7f638b7fe640 1 -- 172.21.15.3:0/3044041756 learned_addr learned my addr 172.21.15.3:0/3044041756 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:37:29.892 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.906+0000 7f638affd640 1 --2- 172.21.15.3:0/3044041756 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f638c06b250 0x7f638c1f4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:29.892 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.906+0000 7f6383fff640 1 --2- 172.21.15.3:0/3044041756 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f638c0ae0b0 0x7f638c142d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:29.892 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.906+0000 7f638b7fe640 1 -- 172.21.15.3:0/3044041756 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f638c0ae0b0 msgr2=0x7f638c142d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:29.892 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.906+0000 7f638b7fe640 1 --2- 172.21.15.3:0/3044041756 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f638c0ae0b0 0x7f638c142d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:29.892 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.906+0000 7f638b7fe640 1 -- 172.21.15.3:0/3044041756 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f638c06b250 msgr2=0x7f638c1f4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:29.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.906+0000 7f638b7fe640 1 --2- 172.21.15.3:0/3044041756 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f638c06b250 0x7f638c1f4e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:29.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.906+0000 7f638b7fe640 1 -- 172.21.15.3:0/3044041756 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f638c1f7dd0 con 0x7f638c0b56f0 2024-05-09T06:37:29.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.906+0000 7f638affd640 1 --2- 172.21.15.3:0/3044041756 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f638c06b250 0x7f638c1f4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:37:29.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.907+0000 7f638b7fe640 1 --2- 172.21.15.3:0/3044041756 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f638c0b56f0 0x7f638c0cea10 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f6384007cf0 tx=0x7f638400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:29.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.907+0000 7f63837fe640 1 -- 172.21.15.3:0/3044041756 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6384014420 con 0x7f638c0b56f0 2024-05-09T06:37:29.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.907+0000 7f63837fe640 1 -- 172.21.15.3:0/3044041756 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6384019070 con 0x7f638c0b56f0 2024-05-09T06:37:29.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.907+0000 7f639129a640 1 -- 172.21.15.3:0/3044041756 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f638c1f80b0 con 0x7f638c0b56f0 2024-05-09T06:37:29.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.907+0000 7f63837fe640 1 -- 172.21.15.3:0/3044041756 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6384014820 con 0x7f638c0b56f0 2024-05-09T06:37:29.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.907+0000 7f639129a640 1 -- 172.21.15.3:0/3044041756 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f638c1f8670 con 0x7f638c0b56f0 2024-05-09T06:37:29.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.907+0000 7f639129a640 1 -- 172.21.15.3:0/3044041756 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f638c069550 con 0x7f638c0b56f0 2024-05-09T06:37:29.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.908+0000 7f63837fe640 1 -- 172.21.15.3:0/3044041756 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 7) ==== 90651+0+0 (secure 0 0 0) 0x7f63840149c0 con 0x7f638c0b56f0 2024-05-09T06:37:29.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.909+0000 7f63837fe640 1 --2- 172.21.15.3:0/3044041756 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f636c06bfe0 0x7f636c06e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:29.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.909+0000 7f63837fe640 1 -- 172.21.15.3:0/3044041756 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 4943+0+0 (secure 0 0 0) 0x7f638408b6a0 con 0x7f638c0b56f0 2024-05-09T06:37:29.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.909+0000 7f63837fe640 1 --2- 172.21.15.3:0/3044041756 >> [v2:172.21.15.190:6800/2143624576,v1:172.21.15.190:6801/2143624576] conn(0x7f636c071ea0 0x7f636c0742c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:29.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.909+0000 7f638affd640 1 --2- 172.21.15.3:0/3044041756 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f636c06bfe0 0x7f636c06e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:29.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.909+0000 7f63837fe640 1 -- 172.21.15.3:0/3044041756 --> [v2:172.21.15.190:6800/2143624576,v1:172.21.15.190:6801/2143624576] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6384014e50 con 0x7f636c071ea0 2024-05-09T06:37:29.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.909+0000 7f63837fe640 1 -- 172.21.15.3:0/3044041756 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f6384092050 con 0x7f638c0b56f0 2024-05-09T06:37:29.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.909+0000 7f6383fff640 1 --2- 172.21.15.3:0/3044041756 >> [v2:172.21.15.190:6800/2143624576,v1:172.21.15.190:6801/2143624576] conn(0x7f636c071ea0 0x7f636c0742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:29.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.909+0000 7f638affd640 1 --2- 172.21.15.3:0/3044041756 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f636c06bfe0 0x7f636c06e4a0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f637c034000 tx=0x7f637c02e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:29.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.909+0000 7f6383fff640 1 --2- 172.21.15.3:0/3044041756 >> [v2:172.21.15.190:6800/2143624576,v1:172.21.15.190:6801/2143624576] conn(0x7f636c071ea0 0x7f636c0742c0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:29.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.910+0000 7f63837fe640 1 -- 172.21.15.3:0/3044041756 <== osd.4 v2:172.21.15.190:6800/2143624576 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6384014e50 con 0x7f636c071ea0 2024-05-09T06:37:29.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.923+0000 7f639129a640 1 -- 172.21.15.3:0/3044041756 --> [v2:172.21.15.190:6800/2143624576,v1:172.21.15.190:6801/2143624576] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f638c0b3ad0 con 0x7f636c071ea0 2024-05-09T06:37:29.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.924+0000 7f63837fe640 1 -- 172.21.15.3:0/3044041756 <== osd.4 v2:172.21.15.190:6800/2143624576 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f638c0b3ad0 con 0x7f636c071ea0 2024-05-09T06:37:29.905 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-05-09T06:37:29.905 INFO:teuthology.orchestra.run.smithi003.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-09T06:37:29.905 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-05-09T06:37:29.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.925+0000 7f639129a640 1 -- 172.21.15.3:0/3044041756 >> [v2:172.21.15.190:6800/2143624576,v1:172.21.15.190:6801/2143624576] conn(0x7f636c071ea0 msgr2=0x7f636c0742c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:29.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.925+0000 7f639129a640 1 --2- 172.21.15.3:0/3044041756 >> [v2:172.21.15.190:6800/2143624576,v1:172.21.15.190:6801/2143624576] conn(0x7f636c071ea0 0x7f636c0742c0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:29.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.925+0000 7f639129a640 1 -- 172.21.15.3:0/3044041756 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f636c06bfe0 msgr2=0x7f636c06e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:29.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.925+0000 7f639129a640 1 --2- 172.21.15.3:0/3044041756 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f636c06bfe0 0x7f636c06e4a0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f637c034000 tx=0x7f637c02e000 comp rx=0 tx=0).stop 2024-05-09T06:37:29.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.925+0000 7f639129a640 1 -- 172.21.15.3:0/3044041756 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f638c0b56f0 msgr2=0x7f638c0cea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:29.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.925+0000 7f639129a640 1 --2- 172.21.15.3:0/3044041756 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f638c0b56f0 0x7f638c0cea10 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f6384007cf0 tx=0x7f638400e4b0 comp rx=0 tx=0).stop 2024-05-09T06:37:29.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.925+0000 7f638b7fe640 1 -- 172.21.15.3:0/3044041756 reap_dead start 2024-05-09T06:37:29.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.925+0000 7f639129a640 1 -- 172.21.15.3:0/3044041756 shutdown_connections 2024-05-09T06:37:29.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.925+0000 7f639129a640 1 -- 172.21.15.3:0/3044041756 >> 172.21.15.3:0/3044041756 conn(0x7f638c0d05c0 msgr2=0x7f638c1573e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:29.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.925+0000 7f639129a640 1 -- 172.21.15.3:0/3044041756 shutdown_connections 2024-05-09T06:37:29.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:29.925+0000 7f639129a640 1 -- 172.21.15.3:0/3044041756 wait complete. 2024-05-09T06:37:29.916 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-05-09T06:37:30.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.040+0000 7fdaa1e8c640 1 Processor -- start 2024-05-09T06:37:30.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.041+0000 7fdaa1e8c640 1 -- start start 2024-05-09T06:37:30.024 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.041+0000 7fdaa1e8c640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fda9c15e5c0 0x7fda9c0d2800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:30.024 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.041+0000 7fdaa1e8c640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fda9c0d1300 0x7fda9c0b47e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:30.024 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.041+0000 7fdaa1e8c640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fda9c15e9a0 con 0x7fda9c0d4c40 2024-05-09T06:37:30.024 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.041+0000 7fdaa1e8c640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fda9c0d50d0 con 0x7fda9c15e5c0 2024-05-09T06:37:30.024 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.041+0000 7fdaa1e8c640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7fda9c0d31c0 con 0x7fda9c0d1300 2024-05-09T06:37:30.024 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.042+0000 7fda9b7fe640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fda9c15e5c0 0x7fda9c0d2800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:30.024 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.042+0000 7fda9b7fe640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fda9c15e5c0 0x7fda9c0d2800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40902/0 (socket says 172.21.15.3:40902) 2024-05-09T06:37:30.025 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.042+0000 7fda9b7fe640 1 -- 172.21.15.3:0/1399271735 learned_addr learned my addr 172.21.15.3:0/1399271735 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:37:30.025 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.042+0000 7fda9bfff640 1 --2- 172.21.15.3:0/1399271735 >> v2:172.21.15.190:3300/0 conn(0x7fda9c0d1300 0x7fda9c0b47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:30.025 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.042+0000 7fda9b7fe640 1 -- 172.21.15.3:0/1399271735 >> v2:172.21.15.190:3300/0 conn(0x7fda9c0d1300 msgr2=0x7fda9c0b47e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:37:30.025 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.042+0000 7fda9b7fe640 1 --2- 172.21.15.3:0/1399271735 >> v2:172.21.15.190:3300/0 conn(0x7fda9c0d1300 0x7fda9c0b47e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:30.025 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.042+0000 7fda9b7fe640 1 -- 172.21.15.3:0/1399271735 >> v1:172.21.15.190:6789/0 conn(0x7fda9c0d4c40 legacy=0x7fda9c0d0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:37:30.025 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.042+0000 7fda9b7fe640 1 -- 172.21.15.3:0/1399271735 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda9c0d2ee0 con 0x7fda9c15e5c0 2024-05-09T06:37:30.025 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.043+0000 7fda9b7fe640 1 --2- 172.21.15.3:0/1399271735 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fda9c15e5c0 0x7fda9c0d2800 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fda8c009870 tx=0x7fda8c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2c246bdb65ec38a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:30.025 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.043+0000 7fda9a7fc640 1 -- 172.21.15.3:0/1399271735 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda8c037070 con 0x7fda9c15e5c0 2024-05-09T06:37:30.025 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.043+0000 7fda9a7fc640 1 -- 172.21.15.3:0/1399271735 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda8c032440 con 0x7fda9c15e5c0 2024-05-09T06:37:30.025 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.043+0000 7fda9a7fc640 1 -- 172.21.15.3:0/1399271735 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda8c032810 con 0x7fda9c15e5c0 2024-05-09T06:37:30.025 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.043+0000 7fdaa1e8c640 1 -- 172.21.15.3:0/1399271735 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fda9c15e5c0 msgr2=0x7fda9c0d2800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:30.025 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.043+0000 7fdaa1e8c640 1 --2- 172.21.15.3:0/1399271735 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fda9c15e5c0 0x7fda9c0d2800 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fda8c009870 tx=0x7fda8c02a270 comp rx=0 tx=0).stop 2024-05-09T06:37:30.025 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.043+0000 7fdaa1e8c640 1 -- 172.21.15.3:0/1399271735 shutdown_connections 2024-05-09T06:37:30.026 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.043+0000 7fdaa1e8c640 1 --2- 172.21.15.3:0/1399271735 >> v2:172.21.15.190:3300/0 conn(0x7fda9c0d1300 0x7fda9c0b47e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:30.026 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.043+0000 7fdaa1e8c640 1 --2- 172.21.15.3:0/1399271735 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fda9c15e5c0 0x7fda9c0d2800 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:30.026 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.043+0000 7fdaa1e8c640 1 -- 172.21.15.3:0/1399271735 >> 172.21.15.3:0/1399271735 conn(0x7fda9c09d500 msgr2=0x7fda9c0d4220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:30.026 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.043+0000 7fdaa1e8c640 1 -- 172.21.15.3:0/1399271735 shutdown_connections 2024-05-09T06:37:30.026 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.043+0000 7fdaa1e8c640 1 -- 172.21.15.3:0/1399271735 wait complete. 2024-05-09T06:37:30.029 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.044+0000 7fdaa1e8c640 1 Processor -- start 2024-05-09T06:37:30.029 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.045+0000 7fdaa1e8c640 1 -- start start 2024-05-09T06:37:30.030 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.045+0000 7fdaa1e8c640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fda9c0d1300 0x7fda9c0ca4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:30.030 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.045+0000 7fdaa1e8c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fda9c0d4c40 0x7fda9c0b9830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:30.030 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.045+0000 7fdaa1e8c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fda9c15e5c0 0x7fda9c0cc230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:30.030 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.045+0000 7fdaa1e8c640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fda9c0b42c0 con 0x7fda9c0d4c40 2024-05-09T06:37:30.030 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.045+0000 7fdaa1e8c640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fda9c0b3fe0 con 0x7fda9c15e5c0 2024-05-09T06:37:30.030 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.045+0000 7fdaa1e8c640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fda9c0b4d20 con 0x7fda9c0d1300 2024-05-09T06:37:30.030 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.045+0000 7fda9affd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fda9c0d4c40 0x7fda9c0b9830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:30.030 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.046+0000 7fda9affd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fda9c0d4c40 0x7fda9c0b9830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40916/0 (socket says 172.21.15.3:40916) 2024-05-09T06:37:30.030 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.046+0000 7fda9affd640 1 -- 172.21.15.3:0/1351725476 learned_addr learned my addr 172.21.15.3:0/1351725476 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:37:30.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.046+0000 7fda9bfff640 1 --2- 172.21.15.3:0/1351725476 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fda9c15e5c0 0x7fda9c0cc230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:30.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.046+0000 7fda9b7fe640 1 --2- 172.21.15.3:0/1351725476 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fda9c0d1300 0x7fda9c0ca4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:30.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.046+0000 7fda9affd640 1 -- 172.21.15.3:0/1351725476 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fda9c0d1300 msgr2=0x7fda9c0ca4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:30.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.046+0000 7fda9affd640 1 --2- 172.21.15.3:0/1351725476 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fda9c0d1300 0x7fda9c0ca4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:30.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.046+0000 7fda9affd640 1 -- 172.21.15.3:0/1351725476 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fda9c15e5c0 msgr2=0x7fda9c0cc230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:30.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.046+0000 7fda9affd640 1 --2- 172.21.15.3:0/1351725476 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fda9c15e5c0 0x7fda9c0cc230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:30.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.046+0000 7fda9affd640 1 -- 172.21.15.3:0/1351725476 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda9c15fe30 con 0x7fda9c0d4c40 2024-05-09T06:37:30.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.046+0000 7fda9affd640 1 --2- 172.21.15.3:0/1351725476 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fda9c0d4c40 0x7fda9c0b9830 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fda9000c460 tx=0x7fda9000c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:30.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.046+0000 7fda83fff640 1 -- 172.21.15.3:0/1351725476 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda90016020 con 0x7fda9c0d4c40 2024-05-09T06:37:30.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.046+0000 7fda83fff640 1 -- 172.21.15.3:0/1351725476 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda90012870 con 0x7fda9c0d4c40 2024-05-09T06:37:30.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.046+0000 7fdaa1e8c640 1 -- 172.21.15.3:0/1351725476 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda9c160110 con 0x7fda9c0d4c40 2024-05-09T06:37:30.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.046+0000 7fda83fff640 1 -- 172.21.15.3:0/1351725476 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda90012c60 con 0x7fda9c0d4c40 2024-05-09T06:37:30.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.046+0000 7fdaa1e8c640 1 -- 172.21.15.3:0/1351725476 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda9c0b9d70 con 0x7fda9c0d4c40 2024-05-09T06:37:30.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.047+0000 7fdaa1e8c640 1 -- 172.21.15.3:0/1351725476 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda9c069550 con 0x7fda9c0d4c40 2024-05-09T06:37:30.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.048+0000 7fda83fff640 1 -- 172.21.15.3:0/1351725476 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 7) ==== 90651+0+0 (secure 0 0 0) 0x7fda90019020 con 0x7fda9c0d4c40 2024-05-09T06:37:30.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.048+0000 7fda83fff640 1 --2- 172.21.15.3:0/1351725476 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fda8806bfe0 0x7fda8806e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:30.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.048+0000 7fda83fff640 1 -- 172.21.15.3:0/1351725476 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 4943+0+0 (secure 0 0 0) 0x7fda9008ac80 con 0x7fda9c0d4c40 2024-05-09T06:37:30.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.048+0000 7fda9b7fe640 1 --2- 172.21.15.3:0/1351725476 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fda8806bfe0 0x7fda8806e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:30.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.049+0000 7fda9b7fe640 1 --2- 172.21.15.3:0/1351725476 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fda8806bfe0 0x7fda8806e4a0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fda8c034000 tx=0x7fda8c02e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:30.033 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.053+0000 7fda83fff640 1 -- 172.21.15.3:0/1351725476 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fda90020020 con 0x7fda9c0d4c40 2024-05-09T06:37:30.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.237+0000 7fdaa1e8c640 1 -- 172.21.15.3:0/1351725476 --> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) -- 0x7fda9c15fbe0 con 0x7fda8806bfe0 2024-05-09T06:37:30.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.237+0000 7fda83fff640 1 -- 172.21.15.3:0/1351725476 <== mgr.4105 v2:172.21.15.190:6832/975893321 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fda9c15fbe0 con 0x7fda8806bfe0 2024-05-09T06:37:30.218 INFO:teuthology.orchestra.run.smithi003.stderr:instructed osd(s) 4 to deep-scrub 2024-05-09T06:37:30.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.240+0000 7fdaa1e8c640 1 -- 172.21.15.3:0/1351725476 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fda8806bfe0 msgr2=0x7fda8806e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:30.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.240+0000 7fdaa1e8c640 1 --2- 172.21.15.3:0/1351725476 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fda8806bfe0 0x7fda8806e4a0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fda8c034000 tx=0x7fda8c02e000 comp rx=0 tx=0).stop 2024-05-09T06:37:30.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.240+0000 7fdaa1e8c640 1 -- 172.21.15.3:0/1351725476 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fda9c0d4c40 msgr2=0x7fda9c0b9830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:30.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.240+0000 7fdaa1e8c640 1 --2- 172.21.15.3:0/1351725476 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fda9c0d4c40 0x7fda9c0b9830 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fda9000c460 tx=0x7fda9000c920 comp rx=0 tx=0).stop 2024-05-09T06:37:30.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.240+0000 7fdaa1e8c640 1 -- 172.21.15.3:0/1351725476 shutdown_connections 2024-05-09T06:37:30.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.240+0000 7fdaa1e8c640 1 --2- 172.21.15.3:0/1351725476 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fda8806bfe0 0x7fda8806e4a0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:30.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.240+0000 7fdaa1e8c640 1 --2- 172.21.15.3:0/1351725476 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fda9c15e5c0 0x7fda9c0cc230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:30.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.240+0000 7fdaa1e8c640 1 --2- 172.21.15.3:0/1351725476 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fda9c0d4c40 0x7fda9c0b9830 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:30.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.240+0000 7fdaa1e8c640 1 --2- 172.21.15.3:0/1351725476 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fda9c0d1300 0x7fda9c0ca4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:30.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.240+0000 7fdaa1e8c640 1 -- 172.21.15.3:0/1351725476 >> 172.21.15.3:0/1351725476 conn(0x7fda9c09d500 msgr2=0x7fda9c0cf190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:30.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.240+0000 7fdaa1e8c640 1 -- 172.21.15.3:0/1351725476 shutdown_connections 2024-05-09T06:37:30.222 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.240+0000 7fdaa1e8c640 1 -- 172.21.15.3:0/1351725476 wait complete. 2024-05-09T06:37:30.232 INFO:tasks.ceph:Scrubbing osd.5 2024-05-09T06:37:30.232 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-05-09T06:37:30.339 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.359+0000 7f5c7d763640 1 Processor -- start 2024-05-09T06:37:30.340 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.359+0000 7f5c7d763640 1 -- start start 2024-05-09T06:37:30.341 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.360+0000 7f5c7d763640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f5c7815c690 0x7f5c780b4410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:30.341 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.360+0000 7f5c7d763640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5c78145c90 con 0x7f5c780cdec0 2024-05-09T06:37:30.341 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.360+0000 7f5c7d763640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5c780b4b60 con 0x7f5c781458b0 2024-05-09T06:37:30.341 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.360+0000 7f5c7d763640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f5c780b4e40 con 0x7f5c7815c690 2024-05-09T06:37:30.341 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.360+0000 7f5c6e7fc640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f5c780cdec0 0x7f5c7806dbd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:43176/0 (socket says 172.21.15.3:43176) 2024-05-09T06:37:30.341 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.360+0000 7f5c6e7fc640 1 -- 172.21.15.3:0/549818852 learned_addr learned my addr 172.21.15.3:0/549818852 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-05-09T06:37:30.341 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.360+0000 7f5c777fe640 1 --2- 172.21.15.3:0/549818852 >> v2:172.21.15.190:3300/0 conn(0x7f5c7815c690 0x7f5c780b4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:30.341 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.360+0000 7f5c767fc640 1 -- 172.21.15.3:0/549818852 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 453689282 0 0) 0x7f5c78145c90 con 0x7f5c780cdec0 2024-05-09T06:37:30.341 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.360+0000 7f5c767fc640 1 -- 172.21.15.3:0/549818852 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5c64003520 con 0x7f5c780cdec0 2024-05-09T06:37:30.341 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.360+0000 7f5c777fe640 1 -- 172.21.15.3:0/549818852 >> v1:172.21.15.190:6789/0 conn(0x7f5c781458b0 legacy=0x7f5c780cd7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:30.342 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.360+0000 7f5c777fe640 1 -- 172.21.15.3:0/549818852 >> v1:172.21.15.3:6789/0 conn(0x7f5c780cdec0 legacy=0x7f5c7806dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:30.342 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.360+0000 7f5c777fe640 1 -- 172.21.15.3:0/549818852 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c780b5120 con 0x7f5c7815c690 2024-05-09T06:37:30.343 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.361+0000 7f5c777fe640 1 --2- 172.21.15.3:0/549818852 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5c7815c690 0x7f5c780b4410 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f5c70009d90 tx=0x7f5c7002a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=245365ac2d231047 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:30.343 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.361+0000 7f5c767fc640 1 -- 172.21.15.3:0/549818852 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c70007740 con 0x7f5c7815c690 2024-05-09T06:37:30.343 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.361+0000 7f5c767fc640 1 -- 172.21.15.3:0/549818852 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c700078e0 con 0x7f5c7815c690 2024-05-09T06:37:30.343 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.361+0000 7f5c767fc640 1 -- 172.21.15.3:0/549818852 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c70007c90 con 0x7f5c7815c690 2024-05-09T06:37:30.343 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.362+0000 7f5c7d763640 1 -- 172.21.15.3:0/549818852 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5c7815c690 msgr2=0x7f5c780b4410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:30.343 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.362+0000 7f5c7d763640 1 --2- 172.21.15.3:0/549818852 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5c7815c690 0x7f5c780b4410 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f5c70009d90 tx=0x7f5c7002a660 comp rx=0 tx=0).stop 2024-05-09T06:37:30.343 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.362+0000 7f5c7d763640 1 -- 172.21.15.3:0/549818852 shutdown_connections 2024-05-09T06:37:30.343 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.362+0000 7f5c7d763640 1 --2- 172.21.15.3:0/549818852 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5c7815c690 0x7f5c780b4410 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:30.343 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.362+0000 7f5c7d763640 1 -- 172.21.15.3:0/549818852 >> 172.21.15.3:0/549818852 conn(0x7f5c7806b640 msgr2=0x7f5c7806ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:30.343 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.362+0000 7f5c7d763640 1 -- 172.21.15.3:0/549818852 shutdown_connections 2024-05-09T06:37:30.344 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.362+0000 7f5c7d763640 1 -- 172.21.15.3:0/549818852 wait complete. 2024-05-09T06:37:30.344 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.363+0000 7f5c7d763640 1 Processor -- start 2024-05-09T06:37:30.344 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.363+0000 7f5c7d763640 1 -- start start 2024-05-09T06:37:30.346 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.364+0000 7f5c7d763640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5c780cdec0 0x7f5c7813ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:30.346 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.364+0000 7f5c7d763640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5c781458b0 0x7f5c7815aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:30.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.364+0000 7f5c7d763640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5c7815c690 0x7f5c78155600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:30.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.364+0000 7f5c7d763640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f5c7806e2e0 con 0x7f5c7815c690 2024-05-09T06:37:30.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.364+0000 7f5c7d763640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f5c7806d6b0 con 0x7f5c780cdec0 2024-05-09T06:37:30.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.364+0000 7f5c7d763640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f5c7806e460 con 0x7f5c781458b0 2024-05-09T06:37:30.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.364+0000 7f5c777fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5c7815c690 0x7f5c78155600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:30.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.364+0000 7f5c777fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5c7815c690 0x7f5c78155600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40924/0 (socket says 172.21.15.3:40924) 2024-05-09T06:37:30.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.364+0000 7f5c777fe640 1 -- 172.21.15.3:0/2273665435 learned_addr learned my addr 172.21.15.3:0/2273665435 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:37:30.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.364+0000 7f5c76ffd640 1 --2- 172.21.15.3:0/2273665435 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5c780cdec0 0x7f5c7813ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:30.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.364+0000 7f5c6e7fc640 1 --2- 172.21.15.3:0/2273665435 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5c781458b0 0x7f5c7815aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:30.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.365+0000 7f5c777fe640 1 -- 172.21.15.3:0/2273665435 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5c781458b0 msgr2=0x7f5c7815aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:30.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.365+0000 7f5c777fe640 1 --2- 172.21.15.3:0/2273665435 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5c781458b0 0x7f5c7815aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:30.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.365+0000 7f5c777fe640 1 -- 172.21.15.3:0/2273665435 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5c780cdec0 msgr2=0x7f5c7813ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:30.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.365+0000 7f5c777fe640 1 --2- 172.21.15.3:0/2273665435 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5c780cdec0 0x7f5c7813ffe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:30.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.365+0000 7f5c777fe640 1 -- 172.21.15.3:0/2273665435 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c7815b230 con 0x7f5c7815c690 2024-05-09T06:37:30.348 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.365+0000 7f5c777fe640 1 --2- 172.21.15.3:0/2273665435 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5c7815c690 0x7f5c78155600 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f5c70009d90 tx=0x7f5c70004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:30.348 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.365+0000 7f5c6ffff640 1 -- 172.21.15.3:0/2273665435 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c7003f070 con 0x7f5c7815c690 2024-05-09T06:37:30.348 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.365+0000 7f5c6ffff640 1 -- 172.21.15.3:0/2273665435 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c70031c40 con 0x7f5c7815c690 2024-05-09T06:37:30.348 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.365+0000 7f5c7d763640 1 -- 172.21.15.3:0/2273665435 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c7815b510 con 0x7f5c7815c690 2024-05-09T06:37:30.348 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.365+0000 7f5c6ffff640 1 -- 172.21.15.3:0/2273665435 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c70009450 con 0x7f5c7815c690 2024-05-09T06:37:30.348 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.365+0000 7f5c7d763640 1 -- 172.21.15.3:0/2273665435 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c7815bad0 con 0x7f5c7815c690 2024-05-09T06:37:30.348 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.366+0000 7f5c7d763640 1 -- 172.21.15.3:0/2273665435 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f5c7815afe0 con 0x7f5c7815c690 2024-05-09T06:37:30.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.367+0000 7f5c6ffff640 1 -- 172.21.15.3:0/2273665435 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 7) ==== 90651+0+0 (secure 0 0 0) 0x7f5c70003990 con 0x7f5c7815c690 2024-05-09T06:37:30.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.367+0000 7f5c6ffff640 1 --2- 172.21.15.3:0/2273665435 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f5c5806c010 0x7f5c5806e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:30.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.367+0000 7f5c6ffff640 1 -- 172.21.15.3:0/2273665435 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 4943+0+0 (secure 0 0 0) 0x7f5c700abf60 con 0x7f5c7815c690 2024-05-09T06:37:30.350 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.367+0000 7f5c6ffff640 1 --2- 172.21.15.3:0/2273665435 >> [v2:172.21.15.190:6812/1766923126,v1:172.21.15.190:6814/1766923126] conn(0x7f5c58071ed0 0x7f5c580742f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:30.350 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.367+0000 7f5c6ffff640 1 -- 172.21.15.3:0/2273665435 --> [v2:172.21.15.190:6812/1766923126,v1:172.21.15.190:6814/1766923126] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5c70032e50 con 0x7f5c58071ed0 2024-05-09T06:37:30.350 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.367+0000 7f5c6ffff640 1 -- 172.21.15.3:0/2273665435 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f5c700ae5e0 con 0x7f5c7815c690 2024-05-09T06:37:30.350 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.367+0000 7f5c76ffd640 1 --2- 172.21.15.3:0/2273665435 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f5c5806c010 0x7f5c5806e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:30.350 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.368+0000 7f5c6e7fc640 1 --2- 172.21.15.3:0/2273665435 >> [v2:172.21.15.190:6812/1766923126,v1:172.21.15.190:6814/1766923126] conn(0x7f5c58071ed0 0x7f5c580742f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:30.350 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.368+0000 7f5c76ffd640 1 --2- 172.21.15.3:0/2273665435 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f5c5806c010 0x7f5c5806e4d0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f5c68001780 tx=0x7f5c68009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:30.350 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.368+0000 7f5c6e7fc640 1 --2- 172.21.15.3:0/2273665435 >> [v2:172.21.15.190:6812/1766923126,v1:172.21.15.190:6814/1766923126] conn(0x7f5c58071ed0 0x7f5c580742f0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:30.350 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.369+0000 7f5c6ffff640 1 -- 172.21.15.3:0/2273665435 <== osd.5 v2:172.21.15.190:6812/1766923126 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5c70032e50 con 0x7f5c58071ed0 2024-05-09T06:37:30.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.382+0000 7f5c7d763640 1 -- 172.21.15.3:0/2273665435 --> [v2:172.21.15.190:6812/1766923126,v1:172.21.15.190:6814/1766923126] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f5c78069550 con 0x7f5c58071ed0 2024-05-09T06:37:30.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.382+0000 7f5c6ffff640 1 -- 172.21.15.3:0/2273665435 <== osd.5 v2:172.21.15.190:6812/1766923126 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f5c78069550 con 0x7f5c58071ed0 2024-05-09T06:37:30.363 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-05-09T06:37:30.363 INFO:teuthology.orchestra.run.smithi003.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-09T06:37:30.363 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-05-09T06:37:30.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.383+0000 7f5c7d763640 1 -- 172.21.15.3:0/2273665435 >> [v2:172.21.15.190:6812/1766923126,v1:172.21.15.190:6814/1766923126] conn(0x7f5c58071ed0 msgr2=0x7f5c580742f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:30.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.383+0000 7f5c7d763640 1 --2- 172.21.15.3:0/2273665435 >> [v2:172.21.15.190:6812/1766923126,v1:172.21.15.190:6814/1766923126] conn(0x7f5c58071ed0 0x7f5c580742f0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:30.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.383+0000 7f5c7d763640 1 -- 172.21.15.3:0/2273665435 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f5c5806c010 msgr2=0x7f5c5806e4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:30.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.383+0000 7f5c7d763640 1 --2- 172.21.15.3:0/2273665435 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f5c5806c010 0x7f5c5806e4d0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f5c68001780 tx=0x7f5c68009040 comp rx=0 tx=0).stop 2024-05-09T06:37:30.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.383+0000 7f5c7d763640 1 -- 172.21.15.3:0/2273665435 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5c7815c690 msgr2=0x7f5c78155600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:30.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.383+0000 7f5c7d763640 1 --2- 172.21.15.3:0/2273665435 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5c7815c690 0x7f5c78155600 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f5c70009d90 tx=0x7f5c70004a40 comp rx=0 tx=0).stop 2024-05-09T06:37:30.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.383+0000 7f5c777fe640 1 -- 172.21.15.3:0/2273665435 reap_dead start 2024-05-09T06:37:30.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.383+0000 7f5c7d763640 1 -- 172.21.15.3:0/2273665435 shutdown_connections 2024-05-09T06:37:30.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.383+0000 7f5c7d763640 1 -- 172.21.15.3:0/2273665435 >> 172.21.15.3:0/2273665435 conn(0x7f5c7806b640 msgr2=0x7f5c7806ba20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:30.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.383+0000 7f5c7d763640 1 -- 172.21.15.3:0/2273665435 shutdown_connections 2024-05-09T06:37:30.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.383+0000 7f5c7d763640 1 -- 172.21.15.3:0/2273665435 wait complete. 2024-05-09T06:37:30.374 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-05-09T06:37:30.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.498+0000 7fa9f5e38640 1 Processor -- start 2024-05-09T06:37:30.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.499+0000 7fa9f5e38640 1 -- start start 2024-05-09T06:37:30.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.499+0000 7fa9f5e38640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa9f00d1300 0x7fa9f00d2800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:30.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.499+0000 7fa9f5e38640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa9f00d4c40 0x7fa9f00b47e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:30.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.499+0000 7fa9f5e38640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa9f015e9a0 con 0x7fa9f015e5c0 2024-05-09T06:37:30.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.499+0000 7fa9f5e38640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fa9f00d50d0 con 0x7fa9f00d4c40 2024-05-09T06:37:30.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.499+0000 7fa9f5e38640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fa9f00d31c0 con 0x7fa9f00d1300 2024-05-09T06:37:30.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.499+0000 7fa9effff640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa9f00d4c40 0x7fa9f00b47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:30.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.500+0000 7fa9effff640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa9f00d4c40 0x7fa9f00b47e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40928/0 (socket says 172.21.15.3:40928) 2024-05-09T06:37:30.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.500+0000 7fa9effff640 1 -- 172.21.15.3:0/3241507322 learned_addr learned my addr 172.21.15.3:0/3241507322 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:37:30.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.500+0000 7fa9eeffd640 1 --2- 172.21.15.3:0/3241507322 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa9f00d1300 0x7fa9f00d2800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:30.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.500+0000 7fa9effff640 1 -- 172.21.15.3:0/3241507322 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa9f00d1300 msgr2=0x7fa9f00d2800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:37:30.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.500+0000 7fa9effff640 1 --2- 172.21.15.3:0/3241507322 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa9f00d1300 0x7fa9f00d2800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:30.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.500+0000 7fa9effff640 1 -- 172.21.15.3:0/3241507322 >> v1:172.21.15.190:6789/0 conn(0x7fa9f015e5c0 legacy=0x7fa9f00d0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:30.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.500+0000 7fa9effff640 1 -- 172.21.15.3:0/3241507322 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9f00d2ee0 con 0x7fa9f00d4c40 2024-05-09T06:37:30.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.501+0000 7fa9effff640 1 --2- 172.21.15.3:0/3241507322 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa9f00d4c40 0x7fa9f00b47e0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fa9e8009e50 tx=0x7fa9e802a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=118607f026ce6f00 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:30.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.501+0000 7fa9ee7fc640 1 -- 172.21.15.3:0/3241507322 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9e8036070 con 0x7fa9f00d4c40 2024-05-09T06:37:30.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.501+0000 7fa9ee7fc640 1 -- 172.21.15.3:0/3241507322 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa9e8031600 con 0x7fa9f00d4c40 2024-05-09T06:37:30.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.501+0000 7fa9ee7fc640 1 -- 172.21.15.3:0/3241507322 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9e80319d0 con 0x7fa9f00d4c40 2024-05-09T06:37:30.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.501+0000 7fa9f5e38640 1 -- 172.21.15.3:0/3241507322 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa9f00d4c40 msgr2=0x7fa9f00b47e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:30.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.501+0000 7fa9f5e38640 1 --2- 172.21.15.3:0/3241507322 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa9f00d4c40 0x7fa9f00b47e0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fa9e8009e50 tx=0x7fa9e802a710 comp rx=0 tx=0).stop 2024-05-09T06:37:30.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.501+0000 7fa9f5e38640 1 -- 172.21.15.3:0/3241507322 shutdown_connections 2024-05-09T06:37:30.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.501+0000 7fa9f5e38640 1 --2- 172.21.15.3:0/3241507322 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa9f00d4c40 0x7fa9f00b47e0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:30.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.501+0000 7fa9f5e38640 1 --2- 172.21.15.3:0/3241507322 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa9f00d1300 0x7fa9f00d2800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:30.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.501+0000 7fa9f5e38640 1 -- 172.21.15.3:0/3241507322 >> 172.21.15.3:0/3241507322 conn(0x7fa9f009d500 msgr2=0x7fa9f00d4220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:30.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.501+0000 7fa9f5e38640 1 -- 172.21.15.3:0/3241507322 shutdown_connections 2024-05-09T06:37:30.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.501+0000 7fa9f5e38640 1 -- 172.21.15.3:0/3241507322 wait complete. 2024-05-09T06:37:30.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.502+0000 7fa9f5e38640 1 Processor -- start 2024-05-09T06:37:30.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.503+0000 7fa9f5e38640 1 -- start start 2024-05-09T06:37:30.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.503+0000 7fa9f5e38640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa9f00d1300 0x7fa9f0157a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:30.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.503+0000 7fa9f5e38640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa9f00d4c40 0x7fa9f0160e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:30.486 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.503+0000 7fa9f5e38640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa9f015e5c0 0x7fa9f00cbae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:30.486 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.503+0000 7fa9f5e38640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fa9f00b42c0 con 0x7fa9f00d4c40 2024-05-09T06:37:30.486 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.503+0000 7fa9f5e38640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fa9f00b3fe0 con 0x7fa9f015e5c0 2024-05-09T06:37:30.486 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.503+0000 7fa9f5e38640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fa9f00b4d20 con 0x7fa9f00d1300 2024-05-09T06:37:30.486 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.504+0000 7fa9eeffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa9f00d4c40 0x7fa9f0160e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:30.486 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.504+0000 7fa9eeffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa9f00d4c40 0x7fa9f0160e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40934/0 (socket says 172.21.15.3:40934) 2024-05-09T06:37:30.486 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.504+0000 7fa9eeffd640 1 -- 172.21.15.3:0/794795415 learned_addr learned my addr 172.21.15.3:0/794795415 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:37:30.487 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.504+0000 7fa9effff640 1 --2- 172.21.15.3:0/794795415 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa9f015e5c0 0x7fa9f00cbae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:30.487 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.504+0000 7fa9ef7fe640 1 --2- 172.21.15.3:0/794795415 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa9f00d1300 0x7fa9f0157a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:30.487 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.504+0000 7fa9eeffd640 1 -- 172.21.15.3:0/794795415 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa9f00d1300 msgr2=0x7fa9f0157a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:30.487 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.504+0000 7fa9eeffd640 1 --2- 172.21.15.3:0/794795415 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa9f00d1300 0x7fa9f0157a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:30.487 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.504+0000 7fa9eeffd640 1 -- 172.21.15.3:0/794795415 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa9f015e5c0 msgr2=0x7fa9f00cbae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:30.487 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.504+0000 7fa9eeffd640 1 --2- 172.21.15.3:0/794795415 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa9f015e5c0 0x7fa9f00cbae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:30.487 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.504+0000 7fa9eeffd640 1 -- 172.21.15.3:0/794795415 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9f015fdc0 con 0x7fa9f00d4c40 2024-05-09T06:37:30.487 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.504+0000 7fa9ef7fe640 1 --2- 172.21.15.3:0/794795415 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa9f00d1300 0x7fa9f0157a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:37:30.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.504+0000 7fa9effff640 1 --2- 172.21.15.3:0/794795415 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa9f015e5c0 0x7fa9f00cbae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:37:30.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.504+0000 7fa9eeffd640 1 --2- 172.21.15.3:0/794795415 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa9f00d4c40 0x7fa9f0160e60 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fa9e400e9b0 tx=0x7fa9e400ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:30.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.504+0000 7fa9d7fff640 1 -- 172.21.15.3:0/794795415 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9e4017070 con 0x7fa9f00d4c40 2024-05-09T06:37:30.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.504+0000 7fa9d7fff640 1 -- 172.21.15.3:0/794795415 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa9e400c9a0 con 0x7fa9f00d4c40 2024-05-09T06:37:30.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.504+0000 7fa9f5e38640 1 -- 172.21.15.3:0/794795415 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9f01600a0 con 0x7fa9f00d4c40 2024-05-09T06:37:30.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.504+0000 7fa9d7fff640 1 -- 172.21.15.3:0/794795415 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9e400cd90 con 0x7fa9f00d4c40 2024-05-09T06:37:30.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.504+0000 7fa9f5e38640 1 -- 172.21.15.3:0/794795415 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9f0160660 con 0x7fa9f00d4c40 2024-05-09T06:37:30.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.505+0000 7fa9f5e38640 1 -- 172.21.15.3:0/794795415 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9f0069550 con 0x7fa9f00d4c40 2024-05-09T06:37:30.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.506+0000 7fa9d7fff640 1 -- 172.21.15.3:0/794795415 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 7) ==== 90651+0+0 (secure 0 0 0) 0x7fa9e401a070 con 0x7fa9f00d4c40 2024-05-09T06:37:30.490 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.506+0000 7fa9d7fff640 1 --2- 172.21.15.3:0/794795415 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fa9dc06bfe0 0x7fa9dc06e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:30.490 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.507+0000 7fa9d7fff640 1 -- 172.21.15.3:0/794795415 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 4943+0+0 (secure 0 0 0) 0x7fa9e408b310 con 0x7fa9f00d4c40 2024-05-09T06:37:30.490 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.507+0000 7fa9ef7fe640 1 --2- 172.21.15.3:0/794795415 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fa9dc06bfe0 0x7fa9dc06e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:30.490 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.507+0000 7fa9ef7fe640 1 --2- 172.21.15.3:0/794795415 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fa9dc06bfe0 0x7fa9dc06e4a0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fa9f01f86b0 tx=0x7fa9e0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:30.491 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.511+0000 7fa9d7fff640 1 -- 172.21.15.3:0/794795415 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fa9e4059ca0 con 0x7fa9f00d4c40 2024-05-09T06:37:30.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.694+0000 7fa9f5e38640 1 -- 172.21.15.3:0/794795415 --> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) -- 0x7fa9f0158020 con 0x7fa9dc06bfe0 2024-05-09T06:37:30.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.695+0000 7fa9d7fff640 1 -- 172.21.15.3:0/794795415 <== mgr.4105 v2:172.21.15.190:6832/975893321 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fa9f0158020 con 0x7fa9dc06bfe0 2024-05-09T06:37:30.675 INFO:teuthology.orchestra.run.smithi003.stderr:instructed osd(s) 5 to deep-scrub 2024-05-09T06:37:30.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.696+0000 7fa9f5e38640 1 -- 172.21.15.3:0/794795415 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fa9dc06bfe0 msgr2=0x7fa9dc06e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:30.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.696+0000 7fa9f5e38640 1 --2- 172.21.15.3:0/794795415 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fa9dc06bfe0 0x7fa9dc06e4a0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fa9f01f86b0 tx=0x7fa9e0009040 comp rx=0 tx=0).stop 2024-05-09T06:37:30.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.696+0000 7fa9f5e38640 1 -- 172.21.15.3:0/794795415 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa9f00d4c40 msgr2=0x7fa9f0160e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:30.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.696+0000 7fa9f5e38640 1 --2- 172.21.15.3:0/794795415 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa9f00d4c40 0x7fa9f0160e60 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fa9e400e9b0 tx=0x7fa9e400ee70 comp rx=0 tx=0).stop 2024-05-09T06:37:30.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.696+0000 7fa9f5e38640 1 -- 172.21.15.3:0/794795415 shutdown_connections 2024-05-09T06:37:30.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.696+0000 7fa9f5e38640 1 --2- 172.21.15.3:0/794795415 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fa9dc06bfe0 0x7fa9dc06e4a0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:30.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.696+0000 7fa9f5e38640 1 --2- 172.21.15.3:0/794795415 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa9f015e5c0 0x7fa9f00cbae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:30.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.696+0000 7fa9f5e38640 1 --2- 172.21.15.3:0/794795415 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa9f00d4c40 0x7fa9f0160e60 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:30.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.696+0000 7fa9f5e38640 1 --2- 172.21.15.3:0/794795415 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa9f00d1300 0x7fa9f0157a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:30.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.696+0000 7fa9f5e38640 1 -- 172.21.15.3:0/794795415 >> 172.21.15.3:0/794795415 conn(0x7fa9f009d500 msgr2=0x7fa9f015d890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:30.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.697+0000 7fa9f5e38640 1 -- 172.21.15.3:0/794795415 shutdown_connections 2024-05-09T06:37:30.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.697+0000 7fa9f5e38640 1 -- 172.21.15.3:0/794795415 wait complete. 2024-05-09T06:37:30.688 INFO:tasks.ceph:Scrubbing osd.6 2024-05-09T06:37:30.689 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-05-09T06:37:30.798 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.817+0000 7fb50e528640 1 Processor -- start 2024-05-09T06:37:30.798 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.817+0000 7fb50e528640 1 -- start start 2024-05-09T06:37:30.802 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.818+0000 7fb50e528640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fb5080cdec0 0x7fb50815c690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:30.802 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.818+0000 7fb50e528640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb508145c90 con 0x7fb5080b4410 2024-05-09T06:37:30.802 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.818+0000 7fb50e528640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb5080b4b60 con 0x7fb5081458b0 2024-05-09T06:37:30.802 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.818+0000 7fb50e528640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7fb5080b4e40 con 0x7fb5080cdec0 2024-05-09T06:37:30.802 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.818+0000 7fb50cac5640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fb5080b4410 0x7fb50806dbd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:50544/0 (socket says 172.21.15.3:50544) 2024-05-09T06:37:30.802 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.818+0000 7fb50cac5640 1 -- 172.21.15.3:0/3273956694 learned_addr learned my addr 172.21.15.3:0/3273956694 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-05-09T06:37:30.802 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.818+0000 7fb5077fe640 1 --2- 172.21.15.3:0/3273956694 >> v2:172.21.15.190:3300/0 conn(0x7fb5080cdec0 0x7fb50815c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:30.802 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.818+0000 7fb506ffd640 1 -- 172.21.15.3:0/3273956694 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2391368688 0 0) 0x7fb508145c90 con 0x7fb5080b4410 2024-05-09T06:37:30.802 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.818+0000 7fb506ffd640 1 -- 172.21.15.3:0/3273956694 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb4f0003520 con 0x7fb5080b4410 2024-05-09T06:37:30.802 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.819+0000 7fb506ffd640 1 -- 172.21.15.3:0/3273956694 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1998475284 0 0) 0x7fb4f0003520 con 0x7fb5080b4410 2024-05-09T06:37:30.802 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.819+0000 7fb506ffd640 1 -- 172.21.15.3:0/3273956694 >> v2:172.21.15.190:3300/0 conn(0x7fb5080cdec0 msgr2=0x7fb50815c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:37:30.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.819+0000 7fb506ffd640 1 --2- 172.21.15.3:0/3273956694 >> v2:172.21.15.190:3300/0 conn(0x7fb5080cdec0 0x7fb50815c690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:30.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.819+0000 7fb506ffd640 1 -- 172.21.15.3:0/3273956694 >> v1:172.21.15.190:6789/0 conn(0x7fb5081458b0 legacy=0x7fb5080cd7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:30.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.819+0000 7fb506ffd640 1 -- 172.21.15.3:0/3273956694 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5080b5120 con 0x7fb5080b4410 2024-05-09T06:37:30.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.819+0000 7fb506ffd640 1 -- 172.21.15.3:0/3273956694 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 563312031 0 0) 0x7fb5000019f0 con 0x7fb5080b4410 2024-05-09T06:37:30.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.819+0000 7fb506ffd640 1 -- 172.21.15.3:0/3273956694 >> v1:172.21.15.3:6789/0 conn(0x7fb5080b4410 legacy=0x7fb50806dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:30.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.819+0000 7fb506ffd640 1 --2- 172.21.15.3:0/3273956694 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb4f00039b0 0x7fb4f0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:30.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.819+0000 7fb506ffd640 1 --2- 172.21.15.3:0/3273956694 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb4f0004800 0x7fb4f0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:30.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.819+0000 7fb506ffd640 1 --2- 172.21.15.3:0/3273956694 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb4f00051c0 0x7fb4f000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:30.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.819+0000 7fb506ffd640 1 -- 172.21.15.3:0/3273956694 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fb5080b4e40 con 0x7fb4f0004800 2024-05-09T06:37:30.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.819+0000 7fb506ffd640 1 -- 172.21.15.3:0/3273956694 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fb4f000c160 con 0x7fb4f00039b0 2024-05-09T06:37:30.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.819+0000 7fb506ffd640 1 -- 172.21.15.3:0/3273956694 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fb4f000c360 con 0x7fb4f00051c0 2024-05-09T06:37:30.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.819+0000 7fb5077fe640 1 --2- 172.21.15.3:0/3273956694 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb4f0004800 0x7fb4f0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:30.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.819+0000 7fb507fff640 1 --2- 172.21.15.3:0/3273956694 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb4f00039b0 0x7fb4f0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:30.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.819+0000 7fb50cac5640 1 --2- 172.21.15.3:0/3273956694 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb4f00051c0 0x7fb4f000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:30.804 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.819+0000 7fb5077fe640 1 -- 172.21.15.3:0/3273956694 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb4f00051c0 msgr2=0x7fb4f000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:37:30.804 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.819+0000 7fb5077fe640 1 --2- 172.21.15.3:0/3273956694 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb4f00051c0 0x7fb4f000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:30.804 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.819+0000 7fb5077fe640 1 -- 172.21.15.3:0/3273956694 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb4f00039b0 msgr2=0x7fb4f0003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:37:30.804 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.819+0000 7fb5077fe640 1 --2- 172.21.15.3:0/3273956694 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb4f00039b0 0x7fb4f0003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:30.804 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.819+0000 7fb5077fe640 1 -- 172.21.15.3:0/3273956694 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4f0003520 con 0x7fb4f0004800 2024-05-09T06:37:30.804 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.819+0000 7fb50cac5640 1 --2- 172.21.15.3:0/3273956694 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb4f00051c0 0x7fb4f000b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:37:30.804 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.819+0000 7fb507fff640 1 --2- 172.21.15.3:0/3273956694 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb4f00039b0 0x7fb4f0003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T06:37:30.804 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.819+0000 7fb50cac5640 1 -- 172.21.15.3:0/3273956694 reap_dead start 2024-05-09T06:37:30.804 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.820+0000 7fb5077fe640 1 --2- 172.21.15.3:0/3273956694 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb4f0004800 0x7fb4f0004c80 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fb4fc009ac0 tx=0x7fb4f0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc0c5facb08bfd17 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:30.804 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.820+0000 7fb506ffd640 1 -- 172.21.15.3:0/3273956694 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4fc036070 con 0x7fb4f0004800 2024-05-09T06:37:30.804 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.820+0000 7fb506ffd640 1 -- 172.21.15.3:0/3273956694 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb4fc031400 con 0x7fb4f0004800 2024-05-09T06:37:30.804 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.820+0000 7fb506ffd640 1 -- 172.21.15.3:0/3273956694 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4fc009690 con 0x7fb4f0004800 2024-05-09T06:37:30.804 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.821+0000 7fb50e528640 1 -- 172.21.15.3:0/3273956694 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb4f0004800 msgr2=0x7fb4f0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:30.804 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.821+0000 7fb50e528640 1 --2- 172.21.15.3:0/3273956694 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb4f0004800 0x7fb4f0004c80 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fb4fc009ac0 tx=0x7fb4f0005830 comp rx=0 tx=0).stop 2024-05-09T06:37:30.805 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.821+0000 7fb50e528640 1 -- 172.21.15.3:0/3273956694 shutdown_connections 2024-05-09T06:37:30.805 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.821+0000 7fb50e528640 1 --2- 172.21.15.3:0/3273956694 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb4f0004800 0x7fb4f0004c80 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:30.805 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.821+0000 7fb50e528640 1 -- 172.21.15.3:0/3273956694 >> 172.21.15.3:0/3273956694 conn(0x7fb50806b640 msgr2=0x7fb50806ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:30.805 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.821+0000 7fb50e528640 1 -- 172.21.15.3:0/3273956694 shutdown_connections 2024-05-09T06:37:30.805 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.821+0000 7fb50e528640 1 -- 172.21.15.3:0/3273956694 wait complete. 2024-05-09T06:37:30.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.822+0000 7fb50e528640 1 Processor -- start 2024-05-09T06:37:30.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.822+0000 7fb50e528640 1 -- start start 2024-05-09T06:37:30.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.823+0000 7fb50e528640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb4f0004800 0x7fb5081581b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:30.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.823+0000 7fb50e528640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb5081441a0 0x7fb5080a9490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:30.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.823+0000 7fb50e528640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb508144620 0x7fb5080ae710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:30.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.823+0000 7fb50e528640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fb5080b4860 con 0x7fb4f0004800 2024-05-09T06:37:30.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.823+0000 7fb50e528640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fb50806d6b0 con 0x7fb508144620 2024-05-09T06:37:30.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.823+0000 7fb50e528640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fb50815c690 con 0x7fb5081441a0 2024-05-09T06:37:30.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.823+0000 7fb507fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb4f0004800 0x7fb5081581b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:30.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.823+0000 7fb507fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb4f0004800 0x7fb5081581b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:56320/0 (socket says 172.21.15.3:56320) 2024-05-09T06:37:30.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.823+0000 7fb507fff640 1 -- 172.21.15.3:0/2911214401 learned_addr learned my addr 172.21.15.3:0/2911214401 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:37:30.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.823+0000 7fb50cac5640 1 --2- 172.21.15.3:0/2911214401 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb508144620 0x7fb5080ae710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:30.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.823+0000 7fb5077fe640 1 --2- 172.21.15.3:0/2911214401 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb5081441a0 0x7fb5080a9490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:30.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.823+0000 7fb507fff640 1 -- 172.21.15.3:0/2911214401 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb5081441a0 msgr2=0x7fb5080a9490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:30.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.823+0000 7fb507fff640 1 --2- 172.21.15.3:0/2911214401 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb5081441a0 0x7fb5080a9490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:30.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.823+0000 7fb507fff640 1 -- 172.21.15.3:0/2911214401 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb508144620 msgr2=0x7fb5080ae710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:30.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.823+0000 7fb507fff640 1 --2- 172.21.15.3:0/2911214401 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb508144620 0x7fb5080ae710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:30.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.823+0000 7fb507fff640 1 -- 172.21.15.3:0/2911214401 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb508157110 con 0x7fb4f0004800 2024-05-09T06:37:30.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.823+0000 7fb507fff640 1 --2- 172.21.15.3:0/2911214401 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb4f0004800 0x7fb5081581b0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fb4f800f8b0 tx=0x7fb4f800fd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:30.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.824+0000 7fb504ff9640 1 -- 172.21.15.3:0/2911214401 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4f80052e0 con 0x7fb4f0004800 2024-05-09T06:37:30.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.824+0000 7fb504ff9640 1 -- 172.21.15.3:0/2911214401 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb4f8018070 con 0x7fb4f0004800 2024-05-09T06:37:30.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.824+0000 7fb50e528640 1 -- 172.21.15.3:0/2911214401 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5081573f0 con 0x7fb4f0004800 2024-05-09T06:37:30.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.824+0000 7fb504ff9640 1 -- 172.21.15.3:0/2911214401 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4f800d970 con 0x7fb4f0004800 2024-05-09T06:37:30.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.824+0000 7fb50e528640 1 -- 172.21.15.3:0/2911214401 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5081579b0 con 0x7fb4f0004800 2024-05-09T06:37:30.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.825+0000 7fb50e528640 1 -- 172.21.15.3:0/2911214401 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fb508157d90 con 0x7fb4f0004800 2024-05-09T06:37:30.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.825+0000 7fb504ff9640 1 -- 172.21.15.3:0/2911214401 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 7) ==== 90651+0+0 (secure 0 0 0) 0x7fb4f800db10 con 0x7fb4f0004800 2024-05-09T06:37:30.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.826+0000 7fb504ff9640 1 --2- 172.21.15.3:0/2911214401 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fb4f406bfe0 0x7fb4f406e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:30.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.826+0000 7fb504ff9640 1 -- 172.21.15.3:0/2911214401 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 4943+0+0 (secure 0 0 0) 0x7fb4f808aed0 con 0x7fb4f0004800 2024-05-09T06:37:30.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.826+0000 7fb504ff9640 1 --2- 172.21.15.3:0/2911214401 >> [v2:172.21.15.190:6824/342414248,v1:172.21.15.190:6825/342414248] conn(0x7fb4f4071ea0 0x7fb4f40742c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:30.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.826+0000 7fb504ff9640 1 -- 172.21.15.3:0/2911214401 --> [v2:172.21.15.190:6824/342414248,v1:172.21.15.190:6825/342414248] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb4f8009e50 con 0x7fb4f4071ea0 2024-05-09T06:37:30.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.826+0000 7fb504ff9640 1 -- 172.21.15.3:0/2911214401 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fb4f808d5b0 con 0x7fb4f0004800 2024-05-09T06:37:30.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.826+0000 7fb5077fe640 1 --2- 172.21.15.3:0/2911214401 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fb4f406bfe0 0x7fb4f406e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:30.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.826+0000 7fb50cac5640 1 --2- 172.21.15.3:0/2911214401 >> [v2:172.21.15.190:6824/342414248,v1:172.21.15.190:6825/342414248] conn(0x7fb4f4071ea0 0x7fb4f40742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:30.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.827+0000 7fb5077fe640 1 --2- 172.21.15.3:0/2911214401 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fb4f406bfe0 0x7fb4f406e4a0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fb4fc033000 tx=0x7fb4fc008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:30.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.827+0000 7fb50cac5640 1 --2- 172.21.15.3:0/2911214401 >> [v2:172.21.15.190:6824/342414248,v1:172.21.15.190:6825/342414248] conn(0x7fb4f4071ea0 0x7fb4f40742c0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:30.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.828+0000 7fb504ff9640 1 -- 172.21.15.3:0/2911214401 <== osd.6 v2:172.21.15.190:6824/342414248 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb4f8009e50 con 0x7fb4f4071ea0 2024-05-09T06:37:30.821 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.841+0000 7fb50e528640 1 -- 172.21.15.3:0/2911214401 --> [v2:172.21.15.190:6824/342414248,v1:172.21.15.190:6825/342414248] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fb508069550 con 0x7fb4f4071ea0 2024-05-09T06:37:30.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.841+0000 7fb504ff9640 1 -- 172.21.15.3:0/2911214401 <== osd.6 v2:172.21.15.190:6824/342414248 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fb508069550 con 0x7fb4f4071ea0 2024-05-09T06:37:30.823 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-05-09T06:37:30.823 INFO:teuthology.orchestra.run.smithi003.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-09T06:37:30.823 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-05-09T06:37:30.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.842+0000 7fb50e528640 1 -- 172.21.15.3:0/2911214401 >> [v2:172.21.15.190:6824/342414248,v1:172.21.15.190:6825/342414248] conn(0x7fb4f4071ea0 msgr2=0x7fb4f40742c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:30.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.842+0000 7fb50e528640 1 --2- 172.21.15.3:0/2911214401 >> [v2:172.21.15.190:6824/342414248,v1:172.21.15.190:6825/342414248] conn(0x7fb4f4071ea0 0x7fb4f40742c0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:30.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.842+0000 7fb50e528640 1 -- 172.21.15.3:0/2911214401 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fb4f406bfe0 msgr2=0x7fb4f406e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:30.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.842+0000 7fb50e528640 1 --2- 172.21.15.3:0/2911214401 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7fb4f406bfe0 0x7fb4f406e4a0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fb4fc033000 tx=0x7fb4fc008030 comp rx=0 tx=0).stop 2024-05-09T06:37:30.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.842+0000 7fb50e528640 1 -- 172.21.15.3:0/2911214401 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb4f0004800 msgr2=0x7fb5081581b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:30.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.842+0000 7fb50e528640 1 --2- 172.21.15.3:0/2911214401 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb4f0004800 0x7fb5081581b0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fb4f800f8b0 tx=0x7fb4f800fd70 comp rx=0 tx=0).stop 2024-05-09T06:37:30.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.842+0000 7fb50cac5640 1 -- 172.21.15.3:0/2911214401 reap_dead start 2024-05-09T06:37:30.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.842+0000 7fb50e528640 1 -- 172.21.15.3:0/2911214401 shutdown_connections 2024-05-09T06:37:30.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.842+0000 7fb50e528640 1 -- 172.21.15.3:0/2911214401 >> 172.21.15.3:0/2911214401 conn(0x7fb50806b640 msgr2=0x7fb50806ba20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:30.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.842+0000 7fb50e528640 1 -- 172.21.15.3:0/2911214401 shutdown_connections 2024-05-09T06:37:30.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.842+0000 7fb50e528640 1 -- 172.21.15.3:0/2911214401 wait complete. 2024-05-09T06:37:30.833 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-05-09T06:37:30.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.958+0000 7f7a0459e640 1 Processor -- start 2024-05-09T06:37:30.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.958+0000 7f7a0459e640 1 -- start start 2024-05-09T06:37:30.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.959+0000 7f7a0459e640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f79fc15c0f0 0x7f79fc160730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:30.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.959+0000 7f7a0459e640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f79fc0b4e30 0x7f79fc06e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:30.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.959+0000 7f7a0459e640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f79fc06e790 0x7f79fc0cd5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:30.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.959+0000 7f7a0459e640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f79fc06ec00 con 0x7f79fc0b4e30 2024-05-09T06:37:30.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.959+0000 7f7a0459e640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f79fc160c70 con 0x7f79fc15c0f0 2024-05-09T06:37:30.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.959+0000 7f7a0459e640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f79fc0b40f0 con 0x7f79fc06e790 2024-05-09T06:37:30.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.959+0000 7f7a01b39640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f79fc0b4e30 0x7f79fc06e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:30.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.959+0000 7f7a01b39640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f79fc0b4e30 0x7f79fc06e250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:56324/0 (socket says 172.21.15.3:56324) 2024-05-09T06:37:30.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.959+0000 7f7a01b39640 1 -- 172.21.15.3:0/1078601639 learned_addr learned my addr 172.21.15.3:0/1078601639 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:37:30.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.959+0000 7f7a0233a640 1 --2- 172.21.15.3:0/1078601639 >> v2:172.21.15.190:3300/0 conn(0x7f79fc15c0f0 0x7f79fc160730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:30.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.959+0000 7f7a02b3b640 1 --2- 172.21.15.3:0/1078601639 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f79fc06e790 0x7f79fc0cd5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:30.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.959+0000 7f7a01b39640 1 -- 172.21.15.3:0/1078601639 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f79fc06e790 msgr2=0x7f79fc0cd5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:37:30.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.959+0000 7f7a01b39640 1 --2- 172.21.15.3:0/1078601639 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f79fc06e790 0x7f79fc0cd5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:30.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.959+0000 7f7a01b39640 1 -- 172.21.15.3:0/1078601639 >> v2:172.21.15.190:3300/0 conn(0x7f79fc15c0f0 msgr2=0x7f79fc160730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:37:30.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.959+0000 7f7a01b39640 1 --2- 172.21.15.3:0/1078601639 >> v2:172.21.15.190:3300/0 conn(0x7f79fc15c0f0 0x7f79fc160730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:30.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.959+0000 7f7a01b39640 1 -- 172.21.15.3:0/1078601639 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79fc0b3050 con 0x7f79fc0b4e30 2024-05-09T06:37:30.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.959+0000 7f7a02b3b640 1 --2- 172.21.15.3:0/1078601639 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f79fc06e790 0x7f79fc0cd5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:37:30.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.960+0000 7f7a01b39640 1 --2- 172.21.15.3:0/1078601639 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f79fc0b4e30 0x7f79fc06e250 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f79fc0b2860 tx=0x7f79f002a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcd75e59cb8c22e9 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:30.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.960+0000 7f7a01338640 1 -- 172.21.15.3:0/1078601639 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79f0037070 con 0x7f79fc0b4e30 2024-05-09T06:37:30.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.960+0000 7f7a01338640 1 -- 172.21.15.3:0/1078601639 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79f0032440 con 0x7f79fc0b4e30 2024-05-09T06:37:30.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.960+0000 7f7a01338640 1 -- 172.21.15.3:0/1078601639 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79f0032810 con 0x7f79fc0b4e30 2024-05-09T06:37:30.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.961+0000 7f7a0459e640 1 -- 172.21.15.3:0/1078601639 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f79fc0b4e30 msgr2=0x7f79fc06e250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:30.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.961+0000 7f7a0459e640 1 --2- 172.21.15.3:0/1078601639 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f79fc0b4e30 0x7f79fc06e250 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f79fc0b2860 tx=0x7f79f002a140 comp rx=0 tx=0).stop 2024-05-09T06:37:30.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.961+0000 7f7a0459e640 1 -- 172.21.15.3:0/1078601639 shutdown_connections 2024-05-09T06:37:30.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.961+0000 7f7a0459e640 1 --2- 172.21.15.3:0/1078601639 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f79fc06e790 0x7f79fc0cd5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:30.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.961+0000 7f7a0459e640 1 --2- 172.21.15.3:0/1078601639 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f79fc0b4e30 0x7f79fc06e250 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:30.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.961+0000 7f7a0459e640 1 --2- 172.21.15.3:0/1078601639 >> v2:172.21.15.190:3300/0 conn(0x7f79fc15c0f0 0x7f79fc160730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:30.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.961+0000 7f7a0459e640 1 -- 172.21.15.3:0/1078601639 >> 172.21.15.3:0/1078601639 conn(0x7f79fc0b1c20 msgr2=0x7f79fc0b2030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:30.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.961+0000 7f7a0459e640 1 -- 172.21.15.3:0/1078601639 shutdown_connections 2024-05-09T06:37:30.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.961+0000 7f7a0459e640 1 -- 172.21.15.3:0/1078601639 wait complete. 2024-05-09T06:37:30.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.962+0000 7f7a0459e640 1 Processor -- start 2024-05-09T06:37:30.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.962+0000 7f7a0459e640 1 -- start start 2024-05-09T06:37:30.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.963+0000 7f7a0459e640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f79fc06e790 0x7f79fc1473f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:30.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.963+0000 7f7a0459e640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f79fc0b4e30 0x7f79fc15c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:30.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.963+0000 7f7a0459e640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f79fc15c0f0 0x7f79fc0afc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:30.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.963+0000 7f7a0459e640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f79fc0b38f0 con 0x7f79fc06e790 2024-05-09T06:37:30.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.963+0000 7f7a0459e640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f79fc0b3610 con 0x7f79fc15c0f0 2024-05-09T06:37:30.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.963+0000 7f7a0459e640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f79fc0b3bd0 con 0x7f79fc0b4e30 2024-05-09T06:37:30.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.963+0000 7f7a0233a640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f79fc06e790 0x7f79fc1473f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:30.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.963+0000 7f7a0233a640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f79fc06e790 0x7f79fc1473f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:56336/0 (socket says 172.21.15.3:56336) 2024-05-09T06:37:30.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.963+0000 7f7a0233a640 1 -- 172.21.15.3:0/287369244 learned_addr learned my addr 172.21.15.3:0/287369244 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:37:30.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.963+0000 7f7a01b39640 1 --2- 172.21.15.3:0/287369244 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f79fc0b4e30 0x7f79fc15c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:30.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.963+0000 7f7a0233a640 1 -- 172.21.15.3:0/287369244 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f79fc0b4e30 msgr2=0x7f79fc15c4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:30.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.963+0000 7f7a0233a640 1 --2- 172.21.15.3:0/287369244 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f79fc0b4e30 0x7f79fc15c4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:30.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.963+0000 7f7a0233a640 1 -- 172.21.15.3:0/287369244 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f79fc15c0f0 msgr2=0x7f79fc0afc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:30.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.963+0000 7f7a02b3b640 1 --2- 172.21.15.3:0/287369244 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f79fc15c0f0 0x7f79fc0afc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:30.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.963+0000 7f7a0233a640 1 --2- 172.21.15.3:0/287369244 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f79fc15c0f0 0x7f79fc0afc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:30.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.963+0000 7f7a0233a640 1 -- 172.21.15.3:0/287369244 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79fc147930 con 0x7f79fc06e790 2024-05-09T06:37:30.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.964+0000 7f7a0233a640 1 --2- 172.21.15.3:0/287369244 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f79fc06e790 0x7f79fc1473f0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f79f400e8a0 tx=0x7f79f400ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:30.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.964+0000 7f79faffd640 1 -- 172.21.15.3:0/287369244 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79f4017070 con 0x7f79fc06e790 2024-05-09T06:37:30.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.964+0000 7f79faffd640 1 -- 172.21.15.3:0/287369244 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79f400c940 con 0x7f79fc06e790 2024-05-09T06:37:30.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.964+0000 7f7a0459e640 1 -- 172.21.15.3:0/287369244 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79fc15ed60 con 0x7f79fc06e790 2024-05-09T06:37:30.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.964+0000 7f79faffd640 1 -- 172.21.15.3:0/287369244 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79f400cd30 con 0x7f79fc06e790 2024-05-09T06:37:30.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.964+0000 7f7a0459e640 1 -- 172.21.15.3:0/287369244 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79fc15f320 con 0x7f79fc06e790 2024-05-09T06:37:30.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.965+0000 7f7a0459e640 1 -- 172.21.15.3:0/287369244 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79fc069550 con 0x7f79fc06e790 2024-05-09T06:37:30.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.965+0000 7f79faffd640 1 -- 172.21.15.3:0/287369244 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 7) ==== 90651+0+0 (secure 0 0 0) 0x7f79f401a070 con 0x7f79fc06e790 2024-05-09T06:37:30.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.966+0000 7f79faffd640 1 --2- 172.21.15.3:0/287369244 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f79e806bfe0 0x7f79e806e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:30.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.966+0000 7f79faffd640 1 -- 172.21.15.3:0/287369244 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 4943+0+0 (secure 0 0 0) 0x7f79f408ac80 con 0x7f79fc06e790 2024-05-09T06:37:30.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.966+0000 7f7a01b39640 1 --2- 172.21.15.3:0/287369244 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f79e806bfe0 0x7f79e806e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:30.949 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.967+0000 7f7a01b39640 1 --2- 172.21.15.3:0/287369244 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f79e806bfe0 0x7f79e806e4a0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f79f0034000 tx=0x7f79f002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:30.951 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:30.970+0000 7f79faffd640 1 -- 172.21.15.3:0/287369244 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f79f40596b0 con 0x7f79fc06e790 2024-05-09T06:37:31.133 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.152+0000 7f7a0459e640 1 -- 172.21.15.3:0/287369244 --> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) -- 0x7f79fc0b42e0 con 0x7f79e806bfe0 2024-05-09T06:37:31.133 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.153+0000 7f79faffd640 1 -- 172.21.15.3:0/287369244 <== mgr.4105 v2:172.21.15.190:6832/975893321 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f79fc0b42e0 con 0x7f79e806bfe0 2024-05-09T06:37:31.133 INFO:teuthology.orchestra.run.smithi003.stderr:instructed osd(s) 6 to deep-scrub 2024-05-09T06:37:31.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.155+0000 7f7a0459e640 1 -- 172.21.15.3:0/287369244 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f79e806bfe0 msgr2=0x7f79e806e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:31.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.155+0000 7f7a0459e640 1 --2- 172.21.15.3:0/287369244 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f79e806bfe0 0x7f79e806e4a0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f79f0034000 tx=0x7f79f002e000 comp rx=0 tx=0).stop 2024-05-09T06:37:31.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.155+0000 7f7a0459e640 1 -- 172.21.15.3:0/287369244 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f79fc06e790 msgr2=0x7f79fc1473f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:31.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.155+0000 7f7a0459e640 1 --2- 172.21.15.3:0/287369244 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f79fc06e790 0x7f79fc1473f0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f79f400e8a0 tx=0x7f79f400ed60 comp rx=0 tx=0).stop 2024-05-09T06:37:31.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.155+0000 7f7a0459e640 1 -- 172.21.15.3:0/287369244 shutdown_connections 2024-05-09T06:37:31.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.155+0000 7f7a0459e640 1 --2- 172.21.15.3:0/287369244 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f79e806bfe0 0x7f79e806e4a0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:31.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.155+0000 7f7a0459e640 1 --2- 172.21.15.3:0/287369244 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f79fc15c0f0 0x7f79fc0afc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:31.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.155+0000 7f7a0459e640 1 --2- 172.21.15.3:0/287369244 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f79fc0b4e30 0x7f79fc15c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:31.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.155+0000 7f7a0459e640 1 --2- 172.21.15.3:0/287369244 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f79fc06e790 0x7f79fc1473f0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:31.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.155+0000 7f7a0459e640 1 -- 172.21.15.3:0/287369244 >> 172.21.15.3:0/287369244 conn(0x7f79fc0b1c20 msgr2=0x7f79fc156750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:31.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.155+0000 7f7a0459e640 1 -- 172.21.15.3:0/287369244 shutdown_connections 2024-05-09T06:37:31.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.155+0000 7f7a0459e640 1 -- 172.21.15.3:0/287369244 wait complete. 2024-05-09T06:37:31.146 INFO:tasks.ceph:Scrubbing osd.7 2024-05-09T06:37:31.147 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-05-09T06:37:31.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.274+0000 7f0e5eb14640 1 Processor -- start 2024-05-09T06:37:31.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.274+0000 7f0e5eb14640 1 -- start start 2024-05-09T06:37:31.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.275+0000 7f0e5eb14640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f0e581458b0 0x7f0e5815c690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:31.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.275+0000 7f0e5eb14640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0e580cdec0 0x7f0e5806dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:31.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.275+0000 7f0e5eb14640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0e580b48c0 con 0x7f0e580b4410 2024-05-09T06:37:31.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.275+0000 7f0e5eb14640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f0e58145c90 con 0x7f0e581458b0 2024-05-09T06:37:31.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.275+0000 7f0e5eb14640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f0e580b4b60 con 0x7f0e580cdec0 2024-05-09T06:37:31.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.275+0000 7f0e5d0b1640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0e580cdec0 0x7f0e5806dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:31.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.275+0000 7f0e5c8b0640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f0e581458b0 0x7f0e5815c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:31.257 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.275+0000 7f0e5d0b1640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0e580cdec0 0x7f0e5806dbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.3:43978/0 (socket says 172.21.15.3:43978) 2024-05-09T06:37:31.257 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.275+0000 7f0e5d0b1640 1 -- 172.21.15.3:0/33308868 learned_addr learned my addr 172.21.15.3:0/33308868 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:37:31.257 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.275+0000 7f0e4ffff640 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7f0e580b4410 0x7f0e580cd7b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.3:34422/0 (socket says 172.21.15.3:34422) 2024-05-09T06:37:31.257 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.275+0000 7f0e5c8b0640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f0e581458b0 0x7f0e5815c690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.3:34626/0 (socket says 172.21.15.3:34626) 2024-05-09T06:37:31.257 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.276+0000 7f0e5d0b1640 1 -- 172.21.15.3:0/33308868 >> v2:172.21.15.190:3300/0 conn(0x7f0e581458b0 msgr2=0x7f0e5815c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:37:31.257 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.276+0000 7f0e5d0b1640 1 --2- 172.21.15.3:0/33308868 >> v2:172.21.15.190:3300/0 conn(0x7f0e581458b0 0x7f0e5815c690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:31.257 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.276+0000 7f0e5d0b1640 1 -- 172.21.15.3:0/33308868 >> v1:172.21.15.190:6789/0 conn(0x7f0e580b4410 legacy=0x7f0e580cd7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:31.257 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.276+0000 7f0e5d0b1640 1 -- 172.21.15.3:0/33308868 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e580b4e40 con 0x7f0e580cdec0 2024-05-09T06:37:31.258 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.276+0000 7f0e5d0b1640 1 --2- 172.21.15.3:0/33308868 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0e580cdec0 0x7f0e5806dbd0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f0e50009e50 tx=0x7f0e5002a710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7d375df1871584fe server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:31.258 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.276+0000 7f0e4f7fe640 1 -- 172.21.15.3:0/33308868 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e50036070 con 0x7f0e580cdec0 2024-05-09T06:37:31.258 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.276+0000 7f0e4f7fe640 1 -- 172.21.15.3:0/33308868 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e50031600 con 0x7f0e580cdec0 2024-05-09T06:37:31.258 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.277+0000 7f0e4f7fe640 1 -- 172.21.15.3:0/33308868 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e500319d0 con 0x7f0e580cdec0 2024-05-09T06:37:31.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.277+0000 7f0e5eb14640 1 -- 172.21.15.3:0/33308868 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0e580cdec0 msgr2=0x7f0e5806dbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:31.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.277+0000 7f0e5eb14640 1 --2- 172.21.15.3:0/33308868 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0e580cdec0 0x7f0e5806dbd0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f0e50009e50 tx=0x7f0e5002a710 comp rx=0 tx=0).stop 2024-05-09T06:37:31.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.277+0000 7f0e5eb14640 1 -- 172.21.15.3:0/33308868 shutdown_connections 2024-05-09T06:37:31.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.277+0000 7f0e5eb14640 1 --2- 172.21.15.3:0/33308868 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0e580cdec0 0x7f0e5806dbd0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:31.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.277+0000 7f0e5eb14640 1 --2- 172.21.15.3:0/33308868 >> v2:172.21.15.190:3300/0 conn(0x7f0e581458b0 0x7f0e5815c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:31.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.277+0000 7f0e5eb14640 1 -- 172.21.15.3:0/33308868 >> 172.21.15.3:0/33308868 conn(0x7f0e5806b640 msgr2=0x7f0e5806ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:31.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.277+0000 7f0e5eb14640 1 -- 172.21.15.3:0/33308868 shutdown_connections 2024-05-09T06:37:31.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.277+0000 7f0e5eb14640 1 -- 172.21.15.3:0/33308868 wait complete. 2024-05-09T06:37:31.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.278+0000 7f0e5eb14640 1 Processor -- start 2024-05-09T06:37:31.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.278+0000 7f0e5eb14640 1 -- start start 2024-05-09T06:37:31.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.279+0000 7f0e5eb14640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0e580b4410 0x7f0e581e6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:31.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.279+0000 7f0e5eb14640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0e580cdec0 0x7f0e58155b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:31.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.279+0000 7f0e5eb14640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0e581458b0 0x7f0e581e81b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:31.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.279+0000 7f0e5eb14640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f0e5806d6b0 con 0x7f0e580b4410 2024-05-09T06:37:31.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.279+0000 7f0e5eb14640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f0e580b5400 con 0x7f0e580cdec0 2024-05-09T06:37:31.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.279+0000 7f0e5eb14640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f0e5806e110 con 0x7f0e581458b0 2024-05-09T06:37:31.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.279+0000 7f0e5c8b0640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0e580b4410 0x7f0e581e6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:31.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.279+0000 7f0e5c8b0640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0e580b4410 0x7f0e581e6210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:56342/0 (socket says 172.21.15.3:56342) 2024-05-09T06:37:31.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.279+0000 7f0e5c8b0640 1 -- 172.21.15.3:0/2537222975 learned_addr learned my addr 172.21.15.3:0/2537222975 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:37:31.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.280+0000 7f0e4ffff640 1 --2- 172.21.15.3:0/2537222975 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0e580cdec0 0x7f0e58155b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:31.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.280+0000 7f0e5d0b1640 1 --2- 172.21.15.3:0/2537222975 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0e581458b0 0x7f0e581e81b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:31.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.280+0000 7f0e5c8b0640 1 -- 172.21.15.3:0/2537222975 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0e581458b0 msgr2=0x7f0e581e81b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:31.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.280+0000 7f0e5c8b0640 1 --2- 172.21.15.3:0/2537222975 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0e581458b0 0x7f0e581e81b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:31.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.280+0000 7f0e5c8b0640 1 -- 172.21.15.3:0/2537222975 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0e580cdec0 msgr2=0x7f0e58155b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:31.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.280+0000 7f0e5c8b0640 1 --2- 172.21.15.3:0/2537222975 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0e580cdec0 0x7f0e58155b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:31.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.280+0000 7f0e5c8b0640 1 -- 172.21.15.3:0/2537222975 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e581e6880 con 0x7f0e580b4410 2024-05-09T06:37:31.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.280+0000 7f0e5d0b1640 1 --2- 172.21.15.3:0/2537222975 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0e581458b0 0x7f0e581e81b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:37:31.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.280+0000 7f0e4ffff640 1 --2- 172.21.15.3:0/2537222975 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0e580cdec0 0x7f0e58155b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:37:31.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.280+0000 7f0e5c8b0640 1 --2- 172.21.15.3:0/2537222975 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0e580b4410 0x7f0e581e6210 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f0e5400e870 tx=0x7f0e5400ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:31.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.280+0000 7f0e4d7fa640 1 -- 172.21.15.3:0/2537222975 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e54017070 con 0x7f0e580b4410 2024-05-09T06:37:31.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.280+0000 7f0e4d7fa640 1 -- 172.21.15.3:0/2537222975 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e5400c940 con 0x7f0e580b4410 2024-05-09T06:37:31.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.280+0000 7f0e5eb14640 1 -- 172.21.15.3:0/2537222975 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e581e6b60 con 0x7f0e580b4410 2024-05-09T06:37:31.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.280+0000 7f0e4d7fa640 1 -- 172.21.15.3:0/2537222975 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e5400cd30 con 0x7f0e580b4410 2024-05-09T06:37:31.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.280+0000 7f0e5eb14640 1 -- 172.21.15.3:0/2537222975 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e581e7120 con 0x7f0e580b4410 2024-05-09T06:37:31.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.281+0000 7f0e5eb14640 1 -- 172.21.15.3:0/2537222975 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f0e58069550 con 0x7f0e580b4410 2024-05-09T06:37:31.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.281+0000 7f0e4d7fa640 1 -- 172.21.15.3:0/2537222975 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 7) ==== 90651+0+0 (secure 0 0 0) 0x7f0e5401a070 con 0x7f0e580b4410 2024-05-09T06:37:31.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.282+0000 7f0e4d7fa640 1 --2- 172.21.15.3:0/2537222975 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f0e4406bfe0 0x7f0e4406e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:31.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.282+0000 7f0e4d7fa640 1 -- 172.21.15.3:0/2537222975 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 4943+0+0 (secure 0 0 0) 0x7f0e5408b040 con 0x7f0e580b4410 2024-05-09T06:37:31.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.282+0000 7f0e4d7fa640 1 --2- 172.21.15.3:0/2537222975 >> [v2:172.21.15.190:6808/537849891,v1:172.21.15.190:6809/537849891] conn(0x7f0e44071ea0 0x7f0e440742c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:31.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.282+0000 7f0e4d7fa640 1 -- 172.21.15.3:0/2537222975 --> [v2:172.21.15.190:6808/537849891,v1:172.21.15.190:6809/537849891] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0e44074970 con 0x7f0e44071ea0 2024-05-09T06:37:31.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.282+0000 7f0e4d7fa640 1 -- 172.21.15.3:0/2537222975 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f0e54092050 con 0x7f0e580b4410 2024-05-09T06:37:31.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.282+0000 7f0e4ffff640 1 --2- 172.21.15.3:0/2537222975 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f0e4406bfe0 0x7f0e4406e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:31.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.282+0000 7f0e5d0b1640 1 --2- 172.21.15.3:0/2537222975 >> [v2:172.21.15.190:6808/537849891,v1:172.21.15.190:6809/537849891] conn(0x7f0e44071ea0 0x7f0e440742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:31.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.283+0000 7f0e4ffff640 1 --2- 172.21.15.3:0/2537222975 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f0e4406bfe0 0x7f0e4406e4a0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f0e58142310 tx=0x7f0e48009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:31.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.283+0000 7f0e5d0b1640 1 --2- 172.21.15.3:0/2537222975 >> [v2:172.21.15.190:6808/537849891,v1:172.21.15.190:6809/537849891] conn(0x7f0e44071ea0 0x7f0e440742c0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:31.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.284+0000 7f0e4d7fa640 1 -- 172.21.15.3:0/2537222975 <== osd.7 v2:172.21.15.190:6808/537849891 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0e44074970 con 0x7f0e44071ea0 2024-05-09T06:37:31.278 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.297+0000 7f0e5eb14640 1 -- 172.21.15.3:0/2537222975 --> [v2:172.21.15.190:6808/537849891,v1:172.21.15.190:6809/537849891] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f0e580b3ce0 con 0x7f0e44071ea0 2024-05-09T06:37:31.278 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.297+0000 7f0e4d7fa640 1 -- 172.21.15.3:0/2537222975 <== osd.7 v2:172.21.15.190:6808/537849891 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f0e580b3ce0 con 0x7f0e44071ea0 2024-05-09T06:37:31.279 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-05-09T06:37:31.279 INFO:teuthology.orchestra.run.smithi003.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-09T06:37:31.279 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-05-09T06:37:31.281 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.297+0000 7f0e5eb14640 1 -- 172.21.15.3:0/2537222975 >> [v2:172.21.15.190:6808/537849891,v1:172.21.15.190:6809/537849891] conn(0x7f0e44071ea0 msgr2=0x7f0e440742c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:31.281 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.297+0000 7f0e5eb14640 1 --2- 172.21.15.3:0/2537222975 >> [v2:172.21.15.190:6808/537849891,v1:172.21.15.190:6809/537849891] conn(0x7f0e44071ea0 0x7f0e440742c0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:31.281 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.298+0000 7f0e5eb14640 1 -- 172.21.15.3:0/2537222975 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f0e4406bfe0 msgr2=0x7f0e4406e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:31.281 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.298+0000 7f0e5eb14640 1 --2- 172.21.15.3:0/2537222975 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f0e4406bfe0 0x7f0e4406e4a0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f0e58142310 tx=0x7f0e48009040 comp rx=0 tx=0).stop 2024-05-09T06:37:31.281 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.298+0000 7f0e5eb14640 1 -- 172.21.15.3:0/2537222975 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0e580b4410 msgr2=0x7f0e581e6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:31.281 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.298+0000 7f0e5eb14640 1 --2- 172.21.15.3:0/2537222975 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0e580b4410 0x7f0e581e6210 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f0e5400e870 tx=0x7f0e5400ed30 comp rx=0 tx=0).stop 2024-05-09T06:37:31.282 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.298+0000 7f0e5d0b1640 1 -- 172.21.15.3:0/2537222975 reap_dead start 2024-05-09T06:37:31.282 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.298+0000 7f0e5eb14640 1 -- 172.21.15.3:0/2537222975 shutdown_connections 2024-05-09T06:37:31.282 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.298+0000 7f0e5eb14640 1 -- 172.21.15.3:0/2537222975 >> 172.21.15.3:0/2537222975 conn(0x7f0e5806b640 msgr2=0x7f0e5806e4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:31.282 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.298+0000 7f0e5eb14640 1 -- 172.21.15.3:0/2537222975 shutdown_connections 2024-05-09T06:37:31.282 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.298+0000 7f0e5eb14640 1 -- 172.21.15.3:0/2537222975 wait complete. 2024-05-09T06:37:31.289 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-05-09T06:37:31.394 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.414+0000 7f3228d48640 1 Processor -- start 2024-05-09T06:37:31.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.414+0000 7f3228d48640 1 -- start start 2024-05-09T06:37:31.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.415+0000 7f3228d48640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f322406e960 0x7f322415d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:31.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.415+0000 7f3228d48640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f32240b40f0 con 0x7f322415c0f0 2024-05-09T06:37:31.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.415+0000 7f3228d48640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f32240b3050 con 0x7f32240b4e30 2024-05-09T06:37:31.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.415+0000 7f3228d48640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f32240b3330 con 0x7f322406e960 2024-05-09T06:37:31.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.415+0000 7f322259c640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f322415c0f0 0x7f3224160730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:50550/0 (socket says 172.21.15.3:50550) 2024-05-09T06:37:31.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.415+0000 7f322259c640 1 -- 172.21.15.3:0/246262895 learned_addr learned my addr 172.21.15.3:0/246262895 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-05-09T06:37:31.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.415+0000 7f3222d9d640 1 --2- 172.21.15.3:0/246262895 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f322406e960 0x7f322415d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:31.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.415+0000 7f3221d9b640 1 -- 172.21.15.3:0/246262895 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2172134262 0 0) 0x7f32240b40f0 con 0x7f322415c0f0 2024-05-09T06:37:31.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.415+0000 7f3221d9b640 1 -- 172.21.15.3:0/246262895 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3210003520 con 0x7f322415c0f0 2024-05-09T06:37:31.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.415+0000 7f3221d9b640 1 -- 172.21.15.3:0/246262895 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3214263591 0 0) 0x7f32240b3050 con 0x7f32240b4e30 2024-05-09T06:37:31.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.415+0000 7f3221d9b640 1 -- 172.21.15.3:0/246262895 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f32240b40f0 con 0x7f32240b4e30 2024-05-09T06:37:31.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.415+0000 7f3222d9d640 1 -- 172.21.15.3:0/246262895 >> v1:172.21.15.190:6789/0 conn(0x7f32240b4e30 legacy=0x7f322406e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:31.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.415+0000 7f3222d9d640 1 -- 172.21.15.3:0/246262895 >> v1:172.21.15.3:6789/0 conn(0x7f322415c0f0 legacy=0x7f3224160730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:31.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.415+0000 7f3222d9d640 1 -- 172.21.15.3:0/246262895 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32240b3610 con 0x7f322406e960 2024-05-09T06:37:31.398 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.416+0000 7f3222d9d640 1 --2- 172.21.15.3:0/246262895 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f322406e960 0x7f322415d4e0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f321c009dc0 tx=0x7f321c02a640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7f19d8ef777d0e59 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:31.398 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.416+0000 7f3221d9b640 1 -- 172.21.15.3:0/246262895 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f321c02ae60 con 0x7f322406e960 2024-05-09T06:37:31.398 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.416+0000 7f3221d9b640 1 -- 172.21.15.3:0/246262895 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f321c007740 con 0x7f322406e960 2024-05-09T06:37:31.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.416+0000 7f3221d9b640 1 -- 172.21.15.3:0/246262895 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f321c007b10 con 0x7f322406e960 2024-05-09T06:37:31.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.417+0000 7f3228d48640 1 -- 172.21.15.3:0/246262895 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f322406e960 msgr2=0x7f322415d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:31.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.417+0000 7f3228d48640 1 --2- 172.21.15.3:0/246262895 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f322406e960 0x7f322415d4e0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f321c009dc0 tx=0x7f321c02a640 comp rx=0 tx=0).stop 2024-05-09T06:37:31.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.417+0000 7f3228d48640 1 -- 172.21.15.3:0/246262895 shutdown_connections 2024-05-09T06:37:31.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.417+0000 7f3228d48640 1 --2- 172.21.15.3:0/246262895 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f322406e960 0x7f322415d4e0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:31.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.417+0000 7f3228d48640 1 -- 172.21.15.3:0/246262895 >> 172.21.15.3:0/246262895 conn(0x7f32240b1c20 msgr2=0x7f32240b2030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:31.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.417+0000 7f3228d48640 1 -- 172.21.15.3:0/246262895 shutdown_connections 2024-05-09T06:37:31.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.417+0000 7f3228d48640 1 -- 172.21.15.3:0/246262895 wait complete. 2024-05-09T06:37:31.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.418+0000 7f3228d48640 1 Processor -- start 2024-05-09T06:37:31.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.418+0000 7f3228d48640 1 -- start start 2024-05-09T06:37:31.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.419+0000 7f3228d48640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f322406e960 0x7f322415ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:31.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.419+0000 7f3228d48640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f32240b4e30 0x7f322414b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:31.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.419+0000 7f3228d48640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f322415c0f0 0x7f322415d700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:31.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.419+0000 7f3228d48640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f32241670b0 con 0x7f322415c0f0 2024-05-09T06:37:31.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.419+0000 7f3228d48640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f32240b3bd0 con 0x7f322406e960 2024-05-09T06:37:31.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.419+0000 7f3228d48640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f3224167230 con 0x7f32240b4e30 2024-05-09T06:37:31.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.419+0000 7f3222d9d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f322415c0f0 0x7f322415d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:31.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.419+0000 7f3222d9d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f322415c0f0 0x7f322415d700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:56356/0 (socket says 172.21.15.3:56356) 2024-05-09T06:37:31.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.419+0000 7f3222d9d640 1 -- 172.21.15.3:0/688716783 learned_addr learned my addr 172.21.15.3:0/688716783 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:37:31.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.420+0000 7f322259c640 1 --2- 172.21.15.3:0/688716783 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f322406e960 0x7f322415ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:31.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.420+0000 7f321bfff640 1 --2- 172.21.15.3:0/688716783 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f32240b4e30 0x7f322414b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:31.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.420+0000 7f3222d9d640 1 -- 172.21.15.3:0/688716783 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f32240b4e30 msgr2=0x7f322414b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:31.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.420+0000 7f3222d9d640 1 --2- 172.21.15.3:0/688716783 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f32240b4e30 0x7f322414b440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:31.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.420+0000 7f3222d9d640 1 -- 172.21.15.3:0/688716783 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f322406e960 msgr2=0x7f322415ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:31.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.420+0000 7f3222d9d640 1 --2- 172.21.15.3:0/688716783 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f322406e960 0x7f322415ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:31.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.420+0000 7f3222d9d640 1 -- 172.21.15.3:0/688716783 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f322415de70 con 0x7f322415c0f0 2024-05-09T06:37:31.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.420+0000 7f322259c640 1 --2- 172.21.15.3:0/688716783 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f322406e960 0x7f322415ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:37:31.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.420+0000 7f321bfff640 1 --2- 172.21.15.3:0/688716783 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f32240b4e30 0x7f322414b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:37:31.404 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.420+0000 7f3222d9d640 1 --2- 172.21.15.3:0/688716783 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f322415c0f0 0x7f322415d700 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f321c02ac30 tx=0x7f321c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:31.404 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.420+0000 7f321affd640 1 -- 172.21.15.3:0/688716783 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f321c040070 con 0x7f322415c0f0 2024-05-09T06:37:31.404 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.420+0000 7f321affd640 1 -- 172.21.15.3:0/688716783 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f321c02ae60 con 0x7f322415c0f0 2024-05-09T06:37:31.404 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.420+0000 7f3228d48640 1 -- 172.21.15.3:0/688716783 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f322415e150 con 0x7f322415c0f0 2024-05-09T06:37:31.404 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.420+0000 7f321affd640 1 -- 172.21.15.3:0/688716783 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f321c031be0 con 0x7f322415c0f0 2024-05-09T06:37:31.404 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.420+0000 7f3228d48640 1 -- 172.21.15.3:0/688716783 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f322415e710 con 0x7f322415c0f0 2024-05-09T06:37:31.404 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.421+0000 7f3228d48640 1 -- 172.21.15.3:0/688716783 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32240b1860 con 0x7f322415c0f0 2024-05-09T06:37:31.404 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.421+0000 7f321affd640 1 -- 172.21.15.3:0/688716783 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 7) ==== 90651+0+0 (secure 0 0 0) 0x7f321c039040 con 0x7f322415c0f0 2024-05-09T06:37:31.405 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.422+0000 7f321affd640 1 --2- 172.21.15.3:0/688716783 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f320406bfe0 0x7f320406e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:31.405 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.422+0000 7f321affd640 1 -- 172.21.15.3:0/688716783 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 4943+0+0 (secure 0 0 0) 0x7f321c0ac650 con 0x7f322415c0f0 2024-05-09T06:37:31.405 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.422+0000 7f322259c640 1 --2- 172.21.15.3:0/688716783 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f320406bfe0 0x7f320406e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:31.405 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.423+0000 7f322259c640 1 --2- 172.21.15.3:0/688716783 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f320406bfe0 0x7f320406e4a0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f3214000f60 tx=0x7f3214009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:31.407 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.426+0000 7f321affd640 1 -- 172.21.15.3:0/688716783 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f32240b1860 con 0x7f322415c0f0 2024-05-09T06:37:31.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.608+0000 7f3228d48640 1 -- 172.21.15.3:0/688716783 --> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) -- 0x7f3224069550 con 0x7f320406bfe0 2024-05-09T06:37:31.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.608+0000 7f321affd640 1 -- 172.21.15.3:0/688716783 <== mgr.4105 v2:172.21.15.190:6832/975893321 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f3224069550 con 0x7f320406bfe0 2024-05-09T06:37:31.589 INFO:teuthology.orchestra.run.smithi003.stderr:instructed osd(s) 7 to deep-scrub 2024-05-09T06:37:31.591 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.610+0000 7f3228d48640 1 -- 172.21.15.3:0/688716783 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f320406bfe0 msgr2=0x7f320406e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:31.591 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.610+0000 7f3228d48640 1 --2- 172.21.15.3:0/688716783 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f320406bfe0 0x7f320406e4a0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f3214000f60 tx=0x7f3214009040 comp rx=0 tx=0).stop 2024-05-09T06:37:31.591 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.610+0000 7f3228d48640 1 -- 172.21.15.3:0/688716783 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f322415c0f0 msgr2=0x7f322415d700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:31.591 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.610+0000 7f3228d48640 1 --2- 172.21.15.3:0/688716783 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f322415c0f0 0x7f322415d700 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f321c02ac30 tx=0x7f321c004a40 comp rx=0 tx=0).stop 2024-05-09T06:37:31.591 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.610+0000 7f3228d48640 1 -- 172.21.15.3:0/688716783 shutdown_connections 2024-05-09T06:37:31.591 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.610+0000 7f3228d48640 1 --2- 172.21.15.3:0/688716783 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f320406bfe0 0x7f320406e4a0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:31.591 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.610+0000 7f3228d48640 1 --2- 172.21.15.3:0/688716783 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f322415c0f0 0x7f322415d700 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:31.591 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.610+0000 7f3228d48640 1 --2- 172.21.15.3:0/688716783 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f32240b4e30 0x7f322414b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:31.591 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.610+0000 7f3228d48640 1 --2- 172.21.15.3:0/688716783 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f322406e960 0x7f322415ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:31.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.610+0000 7f3228d48640 1 -- 172.21.15.3:0/688716783 >> 172.21.15.3:0/688716783 conn(0x7f32240b1c20 msgr2=0x7f32241596e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:31.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.610+0000 7f3228d48640 1 -- 172.21.15.3:0/688716783 shutdown_connections 2024-05-09T06:37:31.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.610+0000 7f3228d48640 1 -- 172.21.15.3:0/688716783 wait complete. 2024-05-09T06:37:31.602 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-09T06:37:31.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.725+0000 7fedc899e640 1 Processor -- start 2024-05-09T06:37:31.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.725+0000 7fedc899e640 1 -- start start 2024-05-09T06:37:31.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.726+0000 7fedc899e640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fedc415e480 0x7fedc40d1dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:31.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.726+0000 7fedc899e640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fedc40d2ed0 0x7fedc41683a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:31.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.726+0000 7fedc899e640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fedc40d4e50 con 0x7fedc40d2310 2024-05-09T06:37:31.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.726+0000 7fedc899e640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fedc40d0270 con 0x7fedc40d2ed0 2024-05-09T06:37:31.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.726+0000 7fedc899e640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7fedc40cf1d0 con 0x7fedc415e480 2024-05-09T06:37:31.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.726+0000 7fedc2d9d640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fedc40d2ed0 0x7fedc41683a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:31.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.726+0000 7fedc2d9d640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fedc40d2ed0 0x7fedc41683a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:56368/0 (socket says 172.21.15.3:56368) 2024-05-09T06:37:31.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.726+0000 7fedc1d9b640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fedc40d2310 0x7fedc40d27c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:50564/0 (socket says 172.21.15.3:50564) 2024-05-09T06:37:31.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.726+0000 7fedc2d9d640 1 -- 172.21.15.3:0/410544749 learned_addr learned my addr 172.21.15.3:0/410544749 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:37:31.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.727+0000 7fedc2d9d640 1 -- 172.21.15.3:0/410544749 >> v2:172.21.15.190:3300/0 conn(0x7fedc415e480 msgr2=0x7fedc40d1dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:37:31.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.727+0000 7fedc259c640 1 --2- 172.21.15.3:0/410544749 >> v2:172.21.15.190:3300/0 conn(0x7fedc415e480 0x7fedc40d1dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:31.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.727+0000 7fedc2d9d640 1 --2- 172.21.15.3:0/410544749 >> v2:172.21.15.190:3300/0 conn(0x7fedc415e480 0x7fedc40d1dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:31.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.727+0000 7fedc2d9d640 1 -- 172.21.15.3:0/410544749 >> v1:172.21.15.3:6789/0 conn(0x7fedc40d2310 legacy=0x7fedc40d27c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:31.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.727+0000 7fedc159a640 1 -- 172.21.15.3:0/410544749 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4017668867 0 0) 0x7fedac001110 con 0x7fedc40d2310 2024-05-09T06:37:31.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.727+0000 7fedc2d9d640 1 -- 172.21.15.3:0/410544749 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fedc40cf4b0 con 0x7fedc40d2ed0 2024-05-09T06:37:31.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.727+0000 7fedc259c640 1 --2- 172.21.15.3:0/410544749 >> v2:172.21.15.190:3300/0 conn(0x7fedc415e480 0x7fedc40d1dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:37:31.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.727+0000 7fedc2d9d640 1 --2- 172.21.15.3:0/410544749 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fedc40d2ed0 0x7fedc41683a0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fedc409da00 tx=0x7fedb402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2893c95bdac2b3de server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:31.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.728+0000 7fedc159a640 1 -- 172.21.15.3:0/410544749 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedb4037070 con 0x7fedc40d2ed0 2024-05-09T06:37:31.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.728+0000 7fedc159a640 1 -- 172.21.15.3:0/410544749 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fedb4032440 con 0x7fedc40d2ed0 2024-05-09T06:37:31.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.728+0000 7fedc159a640 1 -- 172.21.15.3:0/410544749 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedb4032810 con 0x7fedc40d2ed0 2024-05-09T06:37:31.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.728+0000 7fedc899e640 1 -- 172.21.15.3:0/410544749 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fedc40d2ed0 msgr2=0x7fedc41683a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:31.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.728+0000 7fedc899e640 1 --2- 172.21.15.3:0/410544749 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fedc40d2ed0 0x7fedc41683a0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fedc409da00 tx=0x7fedb402a270 comp rx=0 tx=0).stop 2024-05-09T06:37:31.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.728+0000 7fedc899e640 1 -- 172.21.15.3:0/410544749 shutdown_connections 2024-05-09T06:37:31.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.728+0000 7fedc899e640 1 --2- 172.21.15.3:0/410544749 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fedc40d2ed0 0x7fedc41683a0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:31.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.728+0000 7fedc899e640 1 --2- 172.21.15.3:0/410544749 >> v2:172.21.15.190:3300/0 conn(0x7fedc415e480 0x7fedc40d1dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:31.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.728+0000 7fedc899e640 1 -- 172.21.15.3:0/410544749 >> 172.21.15.3:0/410544749 conn(0x7fedc409d4d0 msgr2=0x7fedc40d4200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:31.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.728+0000 7fedc899e640 1 -- 172.21.15.3:0/410544749 shutdown_connections 2024-05-09T06:37:31.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.728+0000 7fedc899e640 1 -- 172.21.15.3:0/410544749 wait complete. 2024-05-09T06:37:31.713 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.730+0000 7fedc899e640 1 Processor -- start 2024-05-09T06:37:31.713 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.730+0000 7fedc899e640 1 -- start start 2024-05-09T06:37:31.713 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.731+0000 7fedc899e640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fedc40d2310 0x7fedc41f7480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:31.713 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.731+0000 7fedc899e640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fedc40d2ed0 0x7fedc4156e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:31.713 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.731+0000 7fedc899e640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fedc415e480 0x7fedc4152930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:31.713 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.731+0000 7fedc899e640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fedc40cfd50 con 0x7fedc40d2ed0 2024-05-09T06:37:31.713 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.731+0000 7fedc899e640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fedc40cfa70 con 0x7fedc415e480 2024-05-09T06:37:31.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.731+0000 7fedc899e640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fedc40b5ab0 con 0x7fedc40d2310 2024-05-09T06:37:31.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.731+0000 7fedc1d9b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fedc40d2ed0 0x7fedc4156e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:31.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.731+0000 7fedc1d9b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fedc40d2ed0 0x7fedc4156e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:56380/0 (socket says 172.21.15.3:56380) 2024-05-09T06:37:31.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.731+0000 7fedc1d9b640 1 -- 172.21.15.3:0/1166461904 learned_addr learned my addr 172.21.15.3:0/1166461904 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:37:31.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.731+0000 7fedc259c640 1 --2- 172.21.15.3:0/1166461904 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fedc40d2310 0x7fedc41f7480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:31.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.731+0000 7fedc2d9d640 1 --2- 172.21.15.3:0/1166461904 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fedc415e480 0x7fedc4152930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:31.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.731+0000 7fedc1d9b640 1 -- 172.21.15.3:0/1166461904 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fedc40d2310 msgr2=0x7fedc41f7480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:31.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.731+0000 7fedc1d9b640 1 --2- 172.21.15.3:0/1166461904 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fedc40d2310 0x7fedc41f7480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:31.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.731+0000 7fedc1d9b640 1 -- 172.21.15.3:0/1166461904 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fedc415e480 msgr2=0x7fedc4152930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:31.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.731+0000 7fedc1d9b640 1 --2- 172.21.15.3:0/1166461904 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fedc415e480 0x7fedc4152930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:31.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.731+0000 7fedc1d9b640 1 -- 172.21.15.3:0/1166461904 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fedc41f7ad0 con 0x7fedc40d2ed0 2024-05-09T06:37:31.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.731+0000 7fedc1d9b640 1 --2- 172.21.15.3:0/1166461904 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fedc40d2ed0 0x7fedc4156e90 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fedac00c410 tx=0x7fedac00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:31.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.731+0000 7fedbaffd640 1 -- 172.21.15.3:0/1166461904 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedac0074e0 con 0x7fedc40d2ed0 2024-05-09T06:37:31.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.731+0000 7fedbaffd640 1 -- 172.21.15.3:0/1166461904 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fedac0047e0 con 0x7fedc40d2ed0 2024-05-09T06:37:31.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.731+0000 7fedbaffd640 1 -- 172.21.15.3:0/1166461904 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedac004bd0 con 0x7fedc40d2ed0 2024-05-09T06:37:31.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.731+0000 7fedc899e640 1 -- 172.21.15.3:0/1166461904 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fedc41f7db0 con 0x7fedc40d2ed0 2024-05-09T06:37:31.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.732+0000 7fedc899e640 1 -- 172.21.15.3:0/1166461904 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fedc41f8370 con 0x7fedc40d2ed0 2024-05-09T06:37:31.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.732+0000 7fedc899e640 1 -- 172.21.15.3:0/1166461904 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fedc404f660 con 0x7fedc40d2ed0 2024-05-09T06:37:31.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.733+0000 7fedbaffd640 1 -- 172.21.15.3:0/1166461904 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 7) ==== 90651+0+0 (secure 0 0 0) 0x7fedac011070 con 0x7fedc40d2ed0 2024-05-09T06:37:31.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.733+0000 7fedbaffd640 1 --2- 172.21.15.3:0/1166461904 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7feda406bfe0 0x7feda406e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:31.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.734+0000 7fedbaffd640 1 -- 172.21.15.3:0/1166461904 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 4943+0+0 (secure 0 0 0) 0x7fedac08aad0 con 0x7fedc40d2ed0 2024-05-09T06:37:31.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.734+0000 7fedc259c640 1 --2- 172.21.15.3:0/1166461904 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7feda406bfe0 0x7feda406e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:31.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.734+0000 7fedc259c640 1 --2- 172.21.15.3:0/1166461904 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7feda406bfe0 0x7feda406e4a0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fedbc0075b0 tx=0x7fedbc007540 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:31.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.738+0000 7fedbaffd640 1 -- 172.21.15.3:0/1166461904 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fedc40d4e50 con 0x7fedc40d2ed0 2024-05-09T06:37:31.899 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.918+0000 7fedc899e640 1 -- 172.21.15.3:0/1166461904 --> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fedc40d18f0 con 0x7feda406bfe0 2024-05-09T06:37:31.900 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.919+0000 7fedbaffd640 1 -- 172.21.15.3:0/1166461904 <== mgr.4105 v2:172.21.15.190:6832/975893321 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13352 (secure 0 0 0) 0x7fedc40d18f0 con 0x7feda406bfe0 2024-05-09T06:37:31.900 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-09T06:37:31.901 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2024-05-09T06:37:31.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.921+0000 7fedc899e640 1 -- 172.21.15.3:0/1166461904 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7feda406bfe0 msgr2=0x7feda406e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:31.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.921+0000 7fedc899e640 1 --2- 172.21.15.3:0/1166461904 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7feda406bfe0 0x7feda406e4a0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fedbc0075b0 tx=0x7fedbc007540 comp rx=0 tx=0).stop 2024-05-09T06:37:31.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.921+0000 7fedc899e640 1 -- 172.21.15.3:0/1166461904 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fedc40d2ed0 msgr2=0x7fedc4156e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:31.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.921+0000 7fedc899e640 1 --2- 172.21.15.3:0/1166461904 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fedc40d2ed0 0x7fedc4156e90 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fedac00c410 tx=0x7fedac00c8d0 comp rx=0 tx=0).stop 2024-05-09T06:37:31.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.921+0000 7fedc899e640 1 -- 172.21.15.3:0/1166461904 shutdown_connections 2024-05-09T06:37:31.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.921+0000 7fedc899e640 1 --2- 172.21.15.3:0/1166461904 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7feda406bfe0 0x7feda406e4a0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:31.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.921+0000 7fedc899e640 1 --2- 172.21.15.3:0/1166461904 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fedc415e480 0x7fedc4152930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:31.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.921+0000 7fedc899e640 1 --2- 172.21.15.3:0/1166461904 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fedc40d2ed0 0x7fedc4156e90 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:31.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.921+0000 7fedc899e640 1 --2- 172.21.15.3:0/1166461904 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fedc40d2310 0x7fedc41f7480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:31.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.921+0000 7fedc899e640 1 -- 172.21.15.3:0/1166461904 >> 172.21.15.3:0/1166461904 conn(0x7fedc409d4d0 msgr2=0x7fedc40b5330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:31.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.922+0000 7fedc899e640 1 -- 172.21.15.3:0/1166461904 shutdown_connections 2024-05-09T06:37:31.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:31.922+0000 7fedc899e640 1 -- 172.21.15.3:0/1166461904 wait complete. 2024-05-09T06:37:31.914 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":78,"stamp":"2024-05-09T06:37:30.794005+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":220596,"kb_used_data":4424,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754124,"statfs":{"total":773094113280,"available":772868222976,"internally_reserved":0,"allocated":4530176,"data_stored":2494856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-6,"num_write_kb":-3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.001584"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":198,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-09T06:37:22.578579+0000","last_change":"2024-05-09T06:35:55.895214+0000","last_active":"2024-05-09T06:37:22.578579+0000","last_peered":"2024-05-09T06:37:22.578579+0000","last_clean":"2024-05-09T06:37:22.578579+0000","last_became_active":"2024-05-09T06:35:55.894087+0000","last_became_peered":"2024-05-09T06:35:55.894087+0000","last_unstale":"2024-05-09T06:37:22.578579+0000","last_undegraded":"2024-05-09T06:37:22.578579+0000","last_fullsized":"2024-05-09T06:37:22.578579+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T06:35:54.820102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T06:35:54.820102+0000","last_clean_scrub_stamp":"2024-05-09T06:35:54.820102+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:29:20.408099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574871,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28012,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343828,"statfs":{"total":96636764160,"available":96608079872,"internally_reserved":0,"allocated":1011712,"data_stored":754633,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574871,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27432,"kb_used_data":408,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344408,"statfs":{"total":96636764160,"available":96608673792,"internally_reserved":0,"allocated":417792,"data_stored":164265,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574870,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27428,"kb_used_data":408,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344412,"statfs":{"total":96636764160,"available":96608677888,"internally_reserved":0,"allocated":417792,"data_stored":164265,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574870,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27428,"kb_used_data":408,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344412,"statfs":{"total":96636764160,"available":96608677888,"internally_reserved":0,"allocated":417792,"data_stored":164265,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574871,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27428,"kb_used_data":408,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344412,"statfs":{"total":96636764160,"available":96608677888,"internally_reserved":0,"allocated":417792,"data_stored":164265,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574870,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27428,"kb_used_data":408,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344412,"statfs":{"total":96636764160,"available":96608677888,"internally_reserved":0,"allocated":417792,"data_stored":164265,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574870,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27428,"kb_used_data":408,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344412,"statfs":{"total":96636764160,"available":96608677888,"internally_reserved":0,"allocated":417792,"data_stored":164265,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574871,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28012,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343828,"statfs":{"total":96636764160,"available":96608079872,"internally_reserved":0,"allocated":1011712,"data_stored":754633,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-09T06:37:31.919 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-05-09T06:35:54.820102+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=6, tm_min=35, tm_sec=54, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=6, tm_min=37, tm_sec=26, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T06:37:31.920 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-05-09T06:37:36.442 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-09T06:37:36.446 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-09T06:37:51.921 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-09T06:37:52.040 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:52.060+0000 7faa956ae640 1 Processor -- start 2024-05-09T06:37:52.040 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:52.060+0000 7faa956ae640 1 -- start start 2024-05-09T06:37:52.041 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:52.061+0000 7faa956ae640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7faa900cc6f0 0x7faa900b4250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:52.041 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:52.061+0000 7faa956ae640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faa900ccad0 0x7faa9015d160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:52.041 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:52.061+0000 7faa956ae640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faa900b48e0 con 0x7faa900cd740 2024-05-09T06:37:52.041 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:52.061+0000 7faa956ae640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7faa900b4bc0 con 0x7faa900cc6f0 2024-05-09T06:37:52.041 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:52.061+0000 7faa956ae640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7faa900b4ea0 con 0x7faa900ccad0 2024-05-09T06:37:52.041 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:52.061+0000 7faa8f7fe640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7faa900cd740 0x7faa9015fc60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:45802/0 (socket says 172.21.15.3:45802) 2024-05-09T06:37:52.041 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:52.061+0000 7faa8f7fe640 1 -- 172.21.15.3:0/3228185786 learned_addr learned my addr 172.21.15.3:0/3228185786 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-05-09T06:37:52.042 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:52.061+0000 7faa8e7fc640 1 --2- 172.21.15.3:0/3228185786 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faa900ccad0 0x7faa9015d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:52.042 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:52.061+0000 7faa8effd640 1 --2- 172.21.15.3:0/3228185786 >> v2:172.21.15.190:3300/0 conn(0x7faa900cc6f0 0x7faa900b4250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:52.042 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:52.061+0000 7faa8dffb640 1 -- 172.21.15.3:0/3228185786 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 425873493 0 0) 0x7faa900b48e0 con 0x7faa900cd740 2024-05-09T06:37:52.042 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:52.061+0000 7faa8dffb640 1 -- 172.21.15.3:0/3228185786 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7faa7c003520 con 0x7faa900cd740 2024-05-09T06:37:52.042 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:52.062+0000 7faa8dffb640 1 -- 172.21.15.3:0/3228185786 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1712437369 0 0) 0x7faa7c003520 con 0x7faa900cd740 2024-05-09T06:37:52.042 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:52.062+0000 7faa8effd640 1 -- 172.21.15.3:0/3228185786 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faa900ccad0 msgr2=0x7faa9015d160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:37:52.042 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:52.062+0000 7faa8effd640 1 --2- 172.21.15.3:0/3228185786 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faa900ccad0 0x7faa9015d160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:52.042 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:52.062+0000 7faa8effd640 1 -- 172.21.15.3:0/3228185786 >> v1:172.21.15.3:6789/0 conn(0x7faa900cd740 legacy=0x7faa9015fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:52.042 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:52.062+0000 7faa8effd640 1 -- 172.21.15.3:0/3228185786 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa900b5180 con 0x7faa900cc6f0 2024-05-09T06:37:52.043 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:52.062+0000 7faa8effd640 1 --2- 172.21.15.3:0/3228185786 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faa900cc6f0 0x7faa900b4250 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7faa80009d90 tx=0x7faa8002a5f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b3ce9cef340ab15c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:52.043 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:52.063+0000 7faa8dffb640 1 -- 172.21.15.3:0/3228185786 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa88001670 con 0x7faa900cc6f0 2024-05-09T06:37:52.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:52.063+0000 7faa8dffb640 1 -- 172.21.15.3:0/3228185786 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa80007540 con 0x7faa900cc6f0 2024-05-09T06:37:52.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:52.063+0000 7faa8dffb640 1 -- 172.21.15.3:0/3228185786 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa80007910 con 0x7faa900cc6f0 2024-05-09T06:37:52.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:52.063+0000 7faa956ae640 1 -- 172.21.15.3:0/3228185786 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faa900cc6f0 msgr2=0x7faa900b4250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:52.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:52.063+0000 7faa956ae640 1 --2- 172.21.15.3:0/3228185786 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faa900cc6f0 0x7faa900b4250 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7faa80009d90 tx=0x7faa8002a5f0 comp rx=0 tx=0).stop 2024-05-09T06:37:52.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:52.063+0000 7faa956ae640 1 -- 172.21.15.3:0/3228185786 shutdown_connections 2024-05-09T06:37:52.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:52.063+0000 7faa956ae640 1 --2- 172.21.15.3:0/3228185786 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faa900ccad0 0x7faa9015d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:52.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:52.063+0000 7faa956ae640 1 --2- 172.21.15.3:0/3228185786 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faa900cc6f0 0x7faa900b4250 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:52.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:52.063+0000 7faa956ae640 1 -- 172.21.15.3:0/3228185786 >> 172.21.15.3:0/3228185786 conn(0x7faa9006b430 msgr2=0x7faa9006b810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:52.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:52.063+0000 7faa956ae640 1 -- 172.21.15.3:0/3228185786 shutdown_connections 2024-05-09T06:37:52.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:52.063+0000 7faa956ae640 1 -- 172.21.15.3:0/3228185786 wait complete. 2024-05-09T06:37:52.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:52.065+0000 7faa956ae640 1 Processor -- start 2024-05-09T06:37:52.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:52.065+0000 7faa956ae640 1 -- start start 2024-05-09T06:37:52.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:52.066+0000 7faa956ae640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faa900cc6f0 0x7faa901f2720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:52.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:52.066+0000 7faa956ae640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faa900ccad0 0x7faa90142c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:52.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:52.066+0000 7faa956ae640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faa900cd740 0x7faa901517f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:52.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:52.066+0000 7faa956ae640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7faa9006ebb0 con 0x7faa900cc6f0 2024-05-09T06:37:52.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:52.066+0000 7faa956ae640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7faa9006ea30 con 0x7faa900ccad0 2024-05-09T06:37:52.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:52.066+0000 7faa956ae640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7faa9006ed30 con 0x7faa900cd740 2024-05-09T06:37:52.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:52.066+0000 7faa8effd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faa900cc6f0 0x7faa901f2720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:52.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:52.066+0000 7faa8effd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faa900cc6f0 0x7faa901f2720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37048/0 (socket says 172.21.15.3:37048) 2024-05-09T06:37:52.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:52.066+0000 7faa8effd640 1 -- 172.21.15.3:0/3011469258 learned_addr learned my addr 172.21.15.3:0/3011469258 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:37:52.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:52.066+0000 7faa8e7fc640 1 --2- 172.21.15.3:0/3011469258 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faa900ccad0 0x7faa90142c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:52.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:52.066+0000 7faa8f7fe640 1 --2- 172.21.15.3:0/3011469258 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faa900cd740 0x7faa901517f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:52.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:52.066+0000 7faa8effd640 1 -- 172.21.15.3:0/3011469258 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faa900cd740 msgr2=0x7faa901517f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:52.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:52.066+0000 7faa8effd640 1 --2- 172.21.15.3:0/3011469258 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faa900cd740 0x7faa901517f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:52.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:52.066+0000 7faa8effd640 1 -- 172.21.15.3:0/3011469258 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faa900ccad0 msgr2=0x7faa90142c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:52.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:52.066+0000 7faa8effd640 1 --2- 172.21.15.3:0/3011469258 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faa900ccad0 0x7faa90142c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:52.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:52.066+0000 7faa8effd640 1 -- 172.21.15.3:0/3011469258 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa901434b0 con 0x7faa900cc6f0 2024-05-09T06:37:52.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:52.066+0000 7faa8effd640 1 --2- 172.21.15.3:0/3011469258 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faa900cc6f0 0x7faa901f2720 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7faa80009d90 tx=0x7faa80004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:52.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:52.066+0000 7faa877fe640 1 -- 172.21.15.3:0/3011469258 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa80040070 con 0x7faa900cc6f0 2024-05-09T06:37:52.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:52.066+0000 7faa877fe640 1 -- 172.21.15.3:0/3011469258 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa80031c40 con 0x7faa900cc6f0 2024-05-09T06:37:52.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:52.066+0000 7faa956ae640 1 -- 172.21.15.3:0/3011469258 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa90143790 con 0x7faa900cc6f0 2024-05-09T06:37:52.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:52.067+0000 7faa877fe640 1 -- 172.21.15.3:0/3011469258 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa80009450 con 0x7faa900cc6f0 2024-05-09T06:37:52.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:52.067+0000 7faa956ae640 1 -- 172.21.15.3:0/3011469258 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa90143d50 con 0x7faa900cc6f0 2024-05-09T06:37:52.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:52.067+0000 7faa956ae640 1 -- 172.21.15.3:0/3011469258 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa9015df20 con 0x7faa900cc6f0 2024-05-09T06:37:52.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:52.068+0000 7faa877fe640 1 -- 172.21.15.3:0/3011469258 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 7) ==== 90651+0+0 (secure 0 0 0) 0x7faa80039040 con 0x7faa900cc6f0 2024-05-09T06:37:52.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:52.069+0000 7faa877fe640 1 --2- 172.21.15.3:0/3011469258 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7faa7006bfe0 0x7faa7006e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:52.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:52.069+0000 7faa877fe640 1 -- 172.21.15.3:0/3011469258 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 4943+0+0 (secure 0 0 0) 0x7faa800ac570 con 0x7faa900cc6f0 2024-05-09T06:37:52.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:52.069+0000 7faa8e7fc640 1 --2- 172.21.15.3:0/3011469258 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7faa7006bfe0 0x7faa7006e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:52.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:52.070+0000 7faa8e7fc640 1 --2- 172.21.15.3:0/3011469258 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7faa7006bfe0 0x7faa7006e4a0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7faa9014efa0 tx=0x7faa78009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:52.053 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:52.073+0000 7faa877fe640 1 -- 172.21.15.3:0/3011469258 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7faa9015df20 con 0x7faa900cc6f0 2024-05-09T06:37:52.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:52.253+0000 7faa956ae640 1 -- 172.21.15.3:0/3011469258 --> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7faa900b4530 con 0x7faa7006bfe0 2024-05-09T06:37:52.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:52.254+0000 7faa877fe640 1 -- 172.21.15.3:0/3011469258 <== mgr.4105 v2:172.21.15.190:6832/975893321 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13356 (secure 0 0 0) 0x7faa900b4530 con 0x7faa7006bfe0 2024-05-09T06:37:52.234 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-09T06:37:52.234 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2024-05-09T06:37:52.236 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:52.257+0000 7faa956ae640 1 -- 172.21.15.3:0/3011469258 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7faa7006bfe0 msgr2=0x7faa7006e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:52.237 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:52.257+0000 7faa956ae640 1 --2- 172.21.15.3:0/3011469258 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7faa7006bfe0 0x7faa7006e4a0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7faa9014efa0 tx=0x7faa78009190 comp rx=0 tx=0).stop 2024-05-09T06:37:52.237 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:52.257+0000 7faa956ae640 1 -- 172.21.15.3:0/3011469258 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faa900cc6f0 msgr2=0x7faa901f2720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:52.237 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:52.257+0000 7faa956ae640 1 --2- 172.21.15.3:0/3011469258 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faa900cc6f0 0x7faa901f2720 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7faa80009d90 tx=0x7faa80004a40 comp rx=0 tx=0).stop 2024-05-09T06:37:52.237 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:52.257+0000 7faa956ae640 1 -- 172.21.15.3:0/3011469258 shutdown_connections 2024-05-09T06:37:52.237 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:52.257+0000 7faa956ae640 1 --2- 172.21.15.3:0/3011469258 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7faa7006bfe0 0x7faa7006e4a0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:52.237 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:52.257+0000 7faa956ae640 1 --2- 172.21.15.3:0/3011469258 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faa900cd740 0x7faa901517f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:52.237 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:52.257+0000 7faa956ae640 1 --2- 172.21.15.3:0/3011469258 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faa900ccad0 0x7faa90142c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:52.237 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:52.257+0000 7faa956ae640 1 --2- 172.21.15.3:0/3011469258 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faa900cc6f0 0x7faa901f2720 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:52.237 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:52.257+0000 7faa956ae640 1 -- 172.21.15.3:0/3011469258 >> 172.21.15.3:0/3011469258 conn(0x7faa9006b430 msgr2=0x7faa9015d810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:52.237 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:52.257+0000 7faa956ae640 1 -- 172.21.15.3:0/3011469258 shutdown_connections 2024-05-09T06:37:52.237 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:52.257+0000 7faa956ae640 1 -- 172.21.15.3:0/3011469258 wait complete. 2024-05-09T06:37:52.247 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":88,"stamp":"2024-05-09T06:37:50.798377+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":220580,"kb_used_data":4424,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754140,"statfs":{"total":773094113280,"available":772868239360,"internally_reserved":0,"allocated":4530176,"data_stored":2494856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002737"},"pg_stats":[{"pgid":"1.0","version":"36'119","reported_seq":209,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-09T06:37:31.935225+0000","last_change":"2024-05-09T06:37:31.935264+0000","last_active":"2024-05-09T06:37:31.935225+0000","last_peered":"2024-05-09T06:37:31.935225+0000","last_clean":"2024-05-09T06:37:31.935225+0000","last_became_active":"2024-05-09T06:35:55.894087+0000","last_became_peered":"2024-05-09T06:35:55.894087+0000","last_unstale":"2024-05-09T06:37:31.935225+0000","last_undegraded":"2024-05-09T06:37:31.935225+0000","last_fullsized":"2024-05-09T06:37:31.935225+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'119","last_scrub_stamp":"2024-05-09T06:37:31.935126+0000","last_deep_scrub":"36'119","last_deep_scrub_stamp":"2024-05-09T06:37:31.935126+0000","last_clean_scrub_stamp":"2024-05-09T06:37:31.935126+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:21:29.461107+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574875,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28004,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343836,"statfs":{"total":96636764160,"available":96608088064,"internally_reserved":0,"allocated":1011712,"data_stored":754633,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574875,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27432,"kb_used_data":408,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344408,"statfs":{"total":96636764160,"available":96608673792,"internally_reserved":0,"allocated":417792,"data_stored":164265,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574874,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27428,"kb_used_data":408,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344412,"statfs":{"total":96636764160,"available":96608677888,"internally_reserved":0,"allocated":417792,"data_stored":164265,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574874,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27428,"kb_used_data":408,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344412,"statfs":{"total":96636764160,"available":96608677888,"internally_reserved":0,"allocated":417792,"data_stored":164265,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574875,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27428,"kb_used_data":408,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344412,"statfs":{"total":96636764160,"available":96608677888,"internally_reserved":0,"allocated":417792,"data_stored":164265,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574874,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27428,"kb_used_data":408,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344412,"statfs":{"total":96636764160,"available":96608677888,"internally_reserved":0,"allocated":417792,"data_stored":164265,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574874,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27428,"kb_used_data":408,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344412,"statfs":{"total":96636764160,"available":96608677888,"internally_reserved":0,"allocated":417792,"data_stored":164265,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574875,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28004,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343836,"statfs":{"total":96636764160,"available":96608088064,"internally_reserved":0,"allocated":1011712,"data_stored":754633,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-09T06:37:52.248 DEBUG:teuthology.orchestra.run.smithi003:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-05-09T06:37:52.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:52.372+0000 7f5da0a4c640 1 Processor -- start 2024-05-09T06:37:52.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:52.372+0000 7f5da0a4c640 1 -- start start 2024-05-09T06:37:52.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:52.373+0000 7f5da0a4c640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f5d9c15e350 0x7f5d9c15e800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:52.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:52.373+0000 7f5da0a4c640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5d9c0b2f20 0x7f5d9c0b3370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:52.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:52.373+0000 7f5da0a4c640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5d9c071220 con 0x7f5d9c0cf990 2024-05-09T06:37:52.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:52.373+0000 7f5da0a4c640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f5d9c071500 con 0x7f5d9c15e350 2024-05-09T06:37:52.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:52.373+0000 7f5da0a4c640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f5d9c0717e0 con 0x7f5d9c0b2f20 2024-05-09T06:37:52.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:52.373+0000 7f5d99d9b640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f5d9c15e350 0x7f5d9c15e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:52.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:52.373+0000 7f5d9ad9d640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5d9c0b2f20 0x7f5d9c0b3370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:52.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:52.373+0000 7f5d9ad9d640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5d9c0b2f20 0x7f5d9c0b3370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.3:42460/0 (socket says 172.21.15.3:42460) 2024-05-09T06:37:52.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:52.373+0000 7f5d9ad9d640 1 -- 172.21.15.3:0/696145647 learned_addr learned my addr 172.21.15.3:0/696145647 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:37:52.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:52.373+0000 7f5d9a59c640 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7f5d9c0cf990 0x7f5d9c0cc1a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.3:34520/0 (socket says 172.21.15.3:34520) 2024-05-09T06:37:52.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:52.374+0000 7f5d9ad9d640 1 -- 172.21.15.3:0/696145647 >> v2:172.21.15.190:3300/0 conn(0x7f5d9c15e350 msgr2=0x7f5d9c15e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:37:52.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:52.374+0000 7f5d9ad9d640 1 --2- 172.21.15.3:0/696145647 >> v2:172.21.15.190:3300/0 conn(0x7f5d9c15e350 0x7f5d9c15e800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:52.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:52.374+0000 7f5d9ad9d640 1 -- 172.21.15.3:0/696145647 >> v1:172.21.15.190:6789/0 conn(0x7f5d9c0cf990 legacy=0x7f5d9c0cc1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:52.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:52.374+0000 7f5d9ad9d640 1 -- 172.21.15.3:0/696145647 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d9c071ac0 con 0x7f5d9c0b2f20 2024-05-09T06:37:52.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:52.374+0000 7f5d99d9b640 1 --2- 172.21.15.3:0/696145647 >> v2:172.21.15.190:3300/0 conn(0x7f5d9c15e350 0x7f5d9c15e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T06:37:52.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:52.374+0000 7f5d9ad9d640 1 --2- 172.21.15.3:0/696145647 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5d9c0b2f20 0x7f5d9c0b3370 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f5d94009e50 tx=0x7f5d9402a710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eefac7fee8d913fb server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:52.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:52.375+0000 7f5d9959a640 1 -- 172.21.15.3:0/696145647 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d94036070 con 0x7f5d9c0b2f20 2024-05-09T06:37:52.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:52.375+0000 7f5d9959a640 1 -- 172.21.15.3:0/696145647 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d94031600 con 0x7f5d9c0b2f20 2024-05-09T06:37:52.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:52.375+0000 7f5d9959a640 1 -- 172.21.15.3:0/696145647 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d940319d0 con 0x7f5d9c0b2f20 2024-05-09T06:37:52.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:52.375+0000 7f5da0a4c640 1 -- 172.21.15.3:0/696145647 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5d9c0b2f20 msgr2=0x7f5d9c0b3370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:52.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:52.375+0000 7f5da0a4c640 1 --2- 172.21.15.3:0/696145647 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5d9c0b2f20 0x7f5d9c0b3370 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f5d94009e50 tx=0x7f5d9402a710 comp rx=0 tx=0).stop 2024-05-09T06:37:52.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:52.375+0000 7f5da0a4c640 1 -- 172.21.15.3:0/696145647 shutdown_connections 2024-05-09T06:37:52.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:52.375+0000 7f5da0a4c640 1 --2- 172.21.15.3:0/696145647 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5d9c0b2f20 0x7f5d9c0b3370 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:52.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:52.375+0000 7f5da0a4c640 1 --2- 172.21.15.3:0/696145647 >> v2:172.21.15.190:3300/0 conn(0x7f5d9c15e350 0x7f5d9c15e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:52.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:52.375+0000 7f5da0a4c640 1 -- 172.21.15.3:0/696145647 >> 172.21.15.3:0/696145647 conn(0x7f5d9c0b1c20 msgr2=0x7f5d9c0b2030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:52.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:52.375+0000 7f5da0a4c640 1 -- 172.21.15.3:0/696145647 shutdown_connections 2024-05-09T06:37:52.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:52.375+0000 7f5da0a4c640 1 -- 172.21.15.3:0/696145647 wait complete. 2024-05-09T06:37:52.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:52.377+0000 7f5da0a4c640 1 Processor -- start 2024-05-09T06:37:52.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:52.377+0000 7f5da0a4c640 1 -- start start 2024-05-09T06:37:52.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:52.377+0000 7f5da0a4c640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5d9c0b2f20 0x7f5d9c13c840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:52.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:52.377+0000 7f5da0a4c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5d9c0cf990 0x7f5d9c1520f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:52.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:52.377+0000 7f5da0a4c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5d9c15e350 0x7f5d9c1553e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:52.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:52.377+0000 7f5da0a4c640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f5d9c0cca30 con 0x7f5d9c15e350 2024-05-09T06:37:52.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:52.377+0000 7f5da0a4c640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f5d9c0cc8b0 con 0x7f5d9c0cf990 2024-05-09T06:37:52.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:52.377+0000 7f5da0a4c640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f5d9c06e250 con 0x7f5d9c0b2f20 2024-05-09T06:37:52.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:52.377+0000 7f5d9ad9d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5d9c15e350 0x7f5d9c1553e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:52.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:52.378+0000 7f5d9ad9d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5d9c15e350 0x7f5d9c1553e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37054/0 (socket says 172.21.15.3:37054) 2024-05-09T06:37:52.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:52.378+0000 7f5d9ad9d640 1 -- 172.21.15.3:0/301720478 learned_addr learned my addr 172.21.15.3:0/301720478 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:37:52.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:52.378+0000 7f5d9a59c640 1 --2- 172.21.15.3:0/301720478 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5d9c0b2f20 0x7f5d9c13c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:52.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:52.378+0000 7f5d99d9b640 1 --2- 172.21.15.3:0/301720478 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5d9c0cf990 0x7f5d9c1520f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:52.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:52.378+0000 7f5d9ad9d640 1 -- 172.21.15.3:0/301720478 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5d9c0b2f20 msgr2=0x7f5d9c13c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:52.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:52.378+0000 7f5d9ad9d640 1 --2- 172.21.15.3:0/301720478 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5d9c0b2f20 0x7f5d9c13c840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:52.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:52.378+0000 7f5d9ad9d640 1 -- 172.21.15.3:0/301720478 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5d9c0cf990 msgr2=0x7f5d9c1520f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:52.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:52.378+0000 7f5d9ad9d640 1 --2- 172.21.15.3:0/301720478 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5d9c0cf990 0x7f5d9c1520f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:52.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:52.378+0000 7f5d9ad9d640 1 -- 172.21.15.3:0/301720478 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d9c13cd80 con 0x7f5d9c15e350 2024-05-09T06:37:52.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:52.378+0000 7f5d9ad9d640 1 --2- 172.21.15.3:0/301720478 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5d9c15e350 0x7f5d9c1553e0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f5d94034040 tx=0x7f5d94004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:52.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:52.378+0000 7f5d8affd640 1 -- 172.21.15.3:0/301720478 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d94036050 con 0x7f5d9c15e350 2024-05-09T06:37:52.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:52.378+0000 7f5d8affd640 1 -- 172.21.15.3:0/301720478 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d94042070 con 0x7f5d9c15e350 2024-05-09T06:37:52.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:52.378+0000 7f5da0a4c640 1 -- 172.21.15.3:0/301720478 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d9c151050 con 0x7f5d9c15e350 2024-05-09T06:37:52.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:52.378+0000 7f5d8affd640 1 -- 172.21.15.3:0/301720478 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d94009510 con 0x7f5d9c15e350 2024-05-09T06:37:52.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:52.378+0000 7f5da0a4c640 1 -- 172.21.15.3:0/301720478 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d9c151610 con 0x7f5d9c15e350 2024-05-09T06:37:52.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:52.379+0000 7f5da0a4c640 1 -- 172.21.15.3:0/301720478 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d9c069550 con 0x7f5d9c15e350 2024-05-09T06:37:52.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:52.380+0000 7f5d8affd640 1 -- 172.21.15.3:0/301720478 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 7) ==== 90651+0+0 (secure 0 0 0) 0x7f5d94007bf0 con 0x7f5d9c15e350 2024-05-09T06:37:52.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:52.380+0000 7f5d8affd640 1 --2- 172.21.15.3:0/301720478 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f5d7c06c010 0x7f5d7c06e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:52.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:52.380+0000 7f5d8affd640 1 -- 172.21.15.3:0/301720478 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 4943+0+0 (secure 0 0 0) 0x7f5d940ab720 con 0x7f5d9c15e350 2024-05-09T06:37:52.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:52.380+0000 7f5d9a59c640 1 --2- 172.21.15.3:0/301720478 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f5d7c06c010 0x7f5d7c06e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:52.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:52.381+0000 7f5d9a59c640 1 --2- 172.21.15.3:0/301720478 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f5d7c06c010 0x7f5d7c06e4d0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f5d9c152c60 tx=0x7f5d8c007ce0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:52.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:52.385+0000 7f5d8affd640 1 -- 172.21.15.3:0/301720478 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f5d9404b020 con 0x7f5d9c15e350 2024-05-09T06:37:52.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:52.566+0000 7f5da0a4c640 1 -- 172.21.15.3:0/301720478 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) -- 0x7f5d9c0cc670 con 0x7f5d9c15e350 2024-05-09T06:37:52.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:52.569+0000 7f5d8affd640 1 -- 172.21.15.3:0/301720478 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v12)=0 v12) ==== 123+0+0 (secure 0 0 0) 0x7f5d9c13cd80 con 0x7f5d9c15e350 2024-05-09T06:37:52.550 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:52.570+0000 7f5d8affd640 1 -- 172.21.15.3:0/301720478 <== mon.0 v2:172.21.15.3:3300/0 8 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f5d94037040 con 0x7f5d9c15e350 2024-05-09T06:37:52.551 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:52.571+0000 7f5da0a4c640 1 -- 172.21.15.3:0/301720478 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f5d7c06c010 msgr2=0x7f5d7c06e4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:52.551 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:52.571+0000 7f5da0a4c640 1 --2- 172.21.15.3:0/301720478 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f5d7c06c010 0x7f5d7c06e4d0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f5d9c152c60 tx=0x7f5d8c007ce0 comp rx=0 tx=0).stop 2024-05-09T06:37:52.551 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:52.571+0000 7f5da0a4c640 1 -- 172.21.15.3:0/301720478 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5d9c15e350 msgr2=0x7f5d9c1553e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:52.551 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:52.571+0000 7f5da0a4c640 1 --2- 172.21.15.3:0/301720478 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5d9c15e350 0x7f5d9c1553e0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f5d94034040 tx=0x7f5d94004a40 comp rx=0 tx=0).stop 2024-05-09T06:37:52.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:52.572+0000 7f5da0a4c640 1 -- 172.21.15.3:0/301720478 shutdown_connections 2024-05-09T06:37:52.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:52.572+0000 7f5da0a4c640 1 --2- 172.21.15.3:0/301720478 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f5d7c06c010 0x7f5d7c06e4d0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:52.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:52.572+0000 7f5da0a4c640 1 --2- 172.21.15.3:0/301720478 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5d9c15e350 0x7f5d9c1553e0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:52.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:52.572+0000 7f5da0a4c640 1 --2- 172.21.15.3:0/301720478 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5d9c0cf990 0x7f5d9c1520f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:52.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:52.572+0000 7f5da0a4c640 1 --2- 172.21.15.3:0/301720478 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5d9c0b2f20 0x7f5d9c13c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:52.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:52.572+0000 7f5da0a4c640 1 -- 172.21.15.3:0/301720478 >> 172.21.15.3:0/301720478 conn(0x7f5d9c0b1c20 msgr2=0x7f5d9c074240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:52.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:52.572+0000 7f5da0a4c640 1 -- 172.21.15.3:0/301720478 shutdown_connections 2024-05-09T06:37:52.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:52.572+0000 7f5da0a4c640 1 -- 172.21.15.3:0/301720478 wait complete. 2024-05-09T06:37:52.562 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-05-09T06:37:52.562 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T06:37:52.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:52.686+0000 7f1030815640 1 Processor -- start 2024-05-09T06:37:52.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:52.686+0000 7f1030815640 1 -- start start 2024-05-09T06:37:52.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:52.687+0000 7f1030815640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f102c15e480 0x7f102c0d1dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:52.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:52.687+0000 7f1030815640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f102c0d2310 0x7f102c0d27c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:52.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:52.687+0000 7f1030815640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f102c0d2d00 0x7f102c162070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:52.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:52.687+0000 7f1030815640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f102c0d4e50 con 0x7f102c15e480 2024-05-09T06:37:52.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:52.687+0000 7f1030815640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f102c0d0270 con 0x7f102c0d2d00 2024-05-09T06:37:52.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:52.687+0000 7f1030815640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f102c0cf1d0 con 0x7f102c0d2310 2024-05-09T06:37:52.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:52.687+0000 7f102a59c640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f102c15e480 0x7f102c0d1dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:52.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:52.688+0000 7f102a59c640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f102c15e480 0x7f102c0d1dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37056/0 (socket says 172.21.15.3:37056) 2024-05-09T06:37:52.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:52.688+0000 7f102a59c640 1 -- 172.21.15.3:0/1838491261 learned_addr learned my addr 172.21.15.3:0/1838491261 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:37:52.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:52.688+0000 7f102ad9d640 1 --2- 172.21.15.3:0/1838491261 >> v2:172.21.15.190:3300/0 conn(0x7f102c0d2d00 0x7f102c162070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:52.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:52.688+0000 7f102a59c640 1 -- 172.21.15.3:0/1838491261 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f102c0d2310 msgr2=0x7f102c0d27c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:37:52.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:52.688+0000 7f1029d9b640 1 --2- 172.21.15.3:0/1838491261 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f102c0d2310 0x7f102c0d27c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:52.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:52.688+0000 7f102a59c640 1 --2- 172.21.15.3:0/1838491261 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f102c0d2310 0x7f102c0d27c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:52.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:52.688+0000 7f102a59c640 1 -- 172.21.15.3:0/1838491261 >> v2:172.21.15.190:3300/0 conn(0x7f102c0d2d00 msgr2=0x7f102c162070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:37:52.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:52.688+0000 7f102a59c640 1 --2- 172.21.15.3:0/1838491261 >> v2:172.21.15.190:3300/0 conn(0x7f102c0d2d00 0x7f102c162070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:52.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:52.688+0000 7f102a59c640 1 -- 172.21.15.3:0/1838491261 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f102c0cf4b0 con 0x7f102c15e480 2024-05-09T06:37:52.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:52.688+0000 7f102ad9d640 1 --2- 172.21.15.3:0/1838491261 >> v2:172.21.15.190:3300/0 conn(0x7f102c0d2d00 0x7f102c162070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T06:37:52.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:52.689+0000 7f102a59c640 1 --2- 172.21.15.3:0/1838491261 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f102c15e480 0x7f102c0d1dd0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f102c0d3f80 tx=0x7f102402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=966bd30aa82c0805 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:52.670 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:52.689+0000 7f102959a640 1 -- 172.21.15.3:0/1838491261 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1024036070 con 0x7f102c15e480 2024-05-09T06:37:52.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:52.689+0000 7f102959a640 1 -- 172.21.15.3:0/1838491261 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f1024031600 con 0x7f102c15e480 2024-05-09T06:37:52.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:52.689+0000 7f102959a640 1 -- 172.21.15.3:0/1838491261 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10240319f0 con 0x7f102c15e480 2024-05-09T06:37:52.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:52.689+0000 7f1030815640 1 -- 172.21.15.3:0/1838491261 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f102c15e480 msgr2=0x7f102c0d1dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:52.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:52.689+0000 7f1030815640 1 --2- 172.21.15.3:0/1838491261 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f102c15e480 0x7f102c0d1dd0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f102c0d3f80 tx=0x7f102402a6a0 comp rx=0 tx=0).stop 2024-05-09T06:37:52.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:52.689+0000 7f1030815640 1 -- 172.21.15.3:0/1838491261 shutdown_connections 2024-05-09T06:37:52.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:52.689+0000 7f1030815640 1 --2- 172.21.15.3:0/1838491261 >> v2:172.21.15.190:3300/0 conn(0x7f102c0d2d00 0x7f102c162070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:52.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:52.689+0000 7f1030815640 1 --2- 172.21.15.3:0/1838491261 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f102c0d2310 0x7f102c0d27c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:52.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:52.689+0000 7f1030815640 1 --2- 172.21.15.3:0/1838491261 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f102c15e480 0x7f102c0d1dd0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:52.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:52.689+0000 7f1030815640 1 -- 172.21.15.3:0/1838491261 >> 172.21.15.3:0/1838491261 conn(0x7f102c09d4d0 msgr2=0x7f102c0d4200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:52.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:52.689+0000 7f1030815640 1 -- 172.21.15.3:0/1838491261 shutdown_connections 2024-05-09T06:37:52.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:52.690+0000 7f1030815640 1 -- 172.21.15.3:0/1838491261 wait complete. 2024-05-09T06:37:52.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:52.691+0000 7f1030815640 1 Processor -- start 2024-05-09T06:37:52.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:52.691+0000 7f1030815640 1 -- start start 2024-05-09T06:37:52.673 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:52.692+0000 7f1030815640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f102c0d2310 0x7f102c14d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:52.673 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:52.692+0000 7f1030815640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f102c0d2d00 0x7f102c0b2d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:52.673 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:52.692+0000 7f1030815640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f102c15e480 0x7f102c0cc5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:52.673 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:52.692+0000 7f1030815640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f102c0cfd50 con 0x7f102c0d2310 2024-05-09T06:37:52.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:52.692+0000 7f1030815640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f102c0cfa70 con 0x7f102c0d2d00 2024-05-09T06:37:52.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:52.692+0000 7f1030815640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f102c0b5ab0 con 0x7f102c15e480 2024-05-09T06:37:52.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:52.692+0000 7f102a59c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f102c0d2310 0x7f102c14d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:52.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:52.692+0000 7f102a59c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f102c0d2310 0x7f102c14d730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37064/0 (socket says 172.21.15.3:37064) 2024-05-09T06:37:52.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:52.692+0000 7f102a59c640 1 -- 172.21.15.3:0/3287977113 learned_addr learned my addr 172.21.15.3:0/3287977113 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-09T06:37:52.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:52.692+0000 7f1029d9b640 1 --2- 172.21.15.3:0/3287977113 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f102c0d2d00 0x7f102c0b2d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:52.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:52.692+0000 7f102ad9d640 1 --2- 172.21.15.3:0/3287977113 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f102c15e480 0x7f102c0cc5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:52.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:52.692+0000 7f102a59c640 1 -- 172.21.15.3:0/3287977113 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f102c15e480 msgr2=0x7f102c0cc5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:52.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:52.692+0000 7f102a59c640 1 --2- 172.21.15.3:0/3287977113 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f102c15e480 0x7f102c0cc5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:52.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:52.692+0000 7f102a59c640 1 -- 172.21.15.3:0/3287977113 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f102c0d2d00 msgr2=0x7f102c0b2d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:52.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:52.692+0000 7f102a59c640 1 --2- 172.21.15.3:0/3287977113 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f102c0d2d00 0x7f102c0b2d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:52.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:52.692+0000 7f102a59c640 1 -- 172.21.15.3:0/3287977113 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f102c0cb520 con 0x7f102c0d2310 2024-05-09T06:37:52.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:52.692+0000 7f102ad9d640 1 --2- 172.21.15.3:0/3287977113 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f102c15e480 0x7f102c0cc5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:37:52.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:52.692+0000 7f1029d9b640 1 --2- 172.21.15.3:0/3287977113 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f102c0d2d00 0x7f102c0b2d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:37:52.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:52.692+0000 7f102a59c640 1 --2- 172.21.15.3:0/3287977113 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f102c0d2310 0x7f102c14d730 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f1024034040 tx=0x7f1024004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:52.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:52.692+0000 7f1022ffd640 1 -- 172.21.15.3:0/3287977113 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f102403f030 con 0x7f102c0d2310 2024-05-09T06:37:52.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:52.692+0000 7f1022ffd640 1 -- 172.21.15.3:0/3287977113 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f1024037040 con 0x7f102c0d2310 2024-05-09T06:37:52.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:52.692+0000 7f1030815640 1 -- 172.21.15.3:0/3287977113 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f102c0cb800 con 0x7f102c0d2310 2024-05-09T06:37:52.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:52.692+0000 7f1022ffd640 1 -- 172.21.15.3:0/3287977113 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1024036070 con 0x7f102c0d2310 2024-05-09T06:37:52.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:52.693+0000 7f1030815640 1 -- 172.21.15.3:0/3287977113 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f102c0cbdc0 con 0x7f102c0d2310 2024-05-09T06:37:52.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:52.693+0000 7f1030815640 1 -- 172.21.15.3:0/3287977113 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f102c0cb2b0 con 0x7f102c0d2310 2024-05-09T06:37:52.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:52.694+0000 7f1022ffd640 1 -- 172.21.15.3:0/3287977113 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 7) ==== 90651+0+0 (secure 0 0 0) 0x7f1024042070 con 0x7f102c0d2310 2024-05-09T06:37:52.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:52.694+0000 7f1022ffd640 1 --2- 172.21.15.3:0/3287977113 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f100c06bfe0 0x7f100c06e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:37:52.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:52.694+0000 7f1022ffd640 1 -- 172.21.15.3:0/3287977113 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 4943+0+0 (secure 0 0 0) 0x7f10240ab330 con 0x7f102c0d2310 2024-05-09T06:37:52.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:52.695+0000 7f1029d9b640 1 --2- 172.21.15.3:0/3287977113 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f100c06bfe0 0x7f100c06e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:37:52.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:52.695+0000 7f1029d9b640 1 --2- 172.21.15.3:0/3287977113 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f100c06bfe0 0x7f100c06e4a0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f1014006fd0 tx=0x7f1014008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:37:52.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:52.699+0000 7f1022ffd640 1 -- 172.21.15.3:0/3287977113 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f1024049070 con 0x7f102c0d2310 2024-05-09T06:37:52.899 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:52.920+0000 7f1030815640 1 -- 172.21.15.3:0/3287977113 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f102c0d18f0 con 0x7f102c0d2310 2024-05-09T06:37:52.900 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:52.920+0000 7f1022ffd640 1 -- 172.21.15.3:0/3287977113 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+3869 (secure 0 0 0) 0x7f1024079c60 con 0x7f102c0d2310 2024-05-09T06:37:52.900 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-09T06:37:52.900 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":20,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4912,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6836/2400645940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":2400645940},{"type":"v1","addr":"172.21.15.190:6836","nonce":2400645940}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4915,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6837/3849729576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6835","nonce":3849729576},{"type":"v1","addr":"172.21.15.190:6837","nonce":3849729576}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4918,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6834/2647782276","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":2647782276},{"type":"v1","addr":"172.21.15.3:6834","nonce":2647782276}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":5499,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/2832434846","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6833","nonce":2832434846},{"type":"v1","addr":"172.21.15.3:6835","nonce":2832434846}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20}],"filesystems":[]} 2024-05-09T06:37:52.900 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 20 2024-05-09T06:37:52.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:52.922+0000 7f1030815640 1 -- 172.21.15.3:0/3287977113 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f100c06bfe0 msgr2=0x7f100c06e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:52.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:52.922+0000 7f1030815640 1 --2- 172.21.15.3:0/3287977113 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f100c06bfe0 0x7f100c06e4a0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f1014006fd0 tx=0x7f1014008040 comp rx=0 tx=0).stop 2024-05-09T06:37:52.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:52.922+0000 7f1030815640 1 -- 172.21.15.3:0/3287977113 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f102c0d2310 msgr2=0x7f102c14d730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:37:52.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:52.922+0000 7f1030815640 1 --2- 172.21.15.3:0/3287977113 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f102c0d2310 0x7f102c14d730 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f1024034040 tx=0x7f1024004a40 comp rx=0 tx=0).stop 2024-05-09T06:37:52.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:52.922+0000 7f1030815640 1 -- 172.21.15.3:0/3287977113 shutdown_connections 2024-05-09T06:37:52.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:52.922+0000 7f1030815640 1 --2- 172.21.15.3:0/3287977113 >> [v2:172.21.15.190:6832/975893321,v1:172.21.15.190:6833/975893321] conn(0x7f100c06bfe0 0x7f100c06e4a0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:52.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:52.922+0000 7f1030815640 1 --2- 172.21.15.3:0/3287977113 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f102c15e480 0x7f102c0cc5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:52.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:52.922+0000 7f1030815640 1 --2- 172.21.15.3:0/3287977113 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f102c0d2d00 0x7f102c0b2d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:52.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:52.922+0000 7f1030815640 1 --2- 172.21.15.3:0/3287977113 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f102c0d2310 0x7f102c14d730 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:37:52.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:52.922+0000 7f1030815640 1 -- 172.21.15.3:0/3287977113 >> 172.21.15.3:0/3287977113 conn(0x7f102c09d4d0 msgr2=0x7f102c0b5290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:37:52.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:52.922+0000 7f1030815640 1 -- 172.21.15.3:0/3287977113 shutdown_connections 2024-05-09T06:37:52.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-09T06:37:52.922+0000 7f1030815640 1 -- 172.21.15.3:0/3287977113 wait complete. 2024-05-09T06:37:52.913 DEBUG:tasks.cephfs.filesystem:already dead... 2024-05-09T06:37:52.913 INFO:teuthology.misc:Shutting down mds daemons... 2024-05-09T06:37:52.913 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-05-09T06:37:52.913 INFO:teuthology.orchestra.run:waiting for 300 2024-05-09T06:37:52.922 INFO:tasks.ceph.mds.a:Stopped 2024-05-09T06:37:52.922 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-05-09T06:37:52.922 INFO:teuthology.orchestra.run:waiting for 300 2024-05-09T06:37:52.973 INFO:tasks.ceph.mds.c:Stopped 2024-05-09T06:37:52.973 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-05-09T06:37:52.973 INFO:teuthology.orchestra.run:waiting for 300 2024-05-09T06:37:52.985 INFO:tasks.ceph.mds.b:Stopped 2024-05-09T06:37:52.985 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-05-09T06:37:52.985 INFO:teuthology.orchestra.run:waiting for 300 2024-05-09T06:37:53.037 INFO:tasks.ceph.mds.d:Stopped 2024-05-09T06:37:53.037 INFO:teuthology.misc:Shutting down osd daemons... 2024-05-09T06:37:53.038 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-05-09T06:37:53.038 INFO:teuthology.orchestra.run:waiting for 300 2024-05-09T06:37:53.065 INFO:tasks.ceph.osd.0:Stopped 2024-05-09T06:37:53.066 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-05-09T06:37:53.066 INFO:teuthology.orchestra.run:waiting for 300 2024-05-09T06:37:53.136 INFO:tasks.ceph.osd.1:Stopped 2024-05-09T06:37:53.136 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-05-09T06:37:53.136 INFO:teuthology.orchestra.run:waiting for 300 2024-05-09T06:37:53.201 INFO:tasks.ceph.osd.2:Stopped 2024-05-09T06:37:53.201 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-05-09T06:37:53.201 INFO:teuthology.orchestra.run:waiting for 300 2024-05-09T06:37:53.269 INFO:tasks.ceph.osd.3:Stopped 2024-05-09T06:37:53.269 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-05-09T06:37:53.269 INFO:teuthology.orchestra.run:waiting for 300 2024-05-09T06:37:53.304 INFO:tasks.ceph.osd.4:Stopped 2024-05-09T06:37:53.304 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-05-09T06:37:53.304 INFO:teuthology.orchestra.run:waiting for 300 2024-05-09T06:37:53.375 INFO:tasks.ceph.osd.5:Stopped 2024-05-09T06:37:53.375 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-05-09T06:37:53.375 INFO:teuthology.orchestra.run:waiting for 300 2024-05-09T06:37:53.457 INFO:tasks.ceph.osd.6:Stopped 2024-05-09T06:37:53.457 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-05-09T06:37:53.457 INFO:teuthology.orchestra.run:waiting for 300 2024-05-09T06:37:53.498 INFO:tasks.ceph.osd.7:Stopped 2024-05-09T06:37:53.498 INFO:teuthology.misc:Shutting down mgr daemons... 2024-05-09T06:37:53.498 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-05-09T06:37:53.499 INFO:teuthology.orchestra.run:waiting for 300 2024-05-09T06:37:53.524 INFO:tasks.ceph.mgr.y:Stopped 2024-05-09T06:37:53.524 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-05-09T06:37:53.524 INFO:teuthology.orchestra.run:waiting for 300 2024-05-09T06:37:53.556 INFO:tasks.ceph.mgr.x:Stopped 2024-05-09T06:37:53.556 INFO:teuthology.misc:Shutting down mon daemons... 2024-05-09T06:37:53.556 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-05-09T06:37:53.556 INFO:teuthology.orchestra.run:waiting for 300 2024-05-09T06:37:53.576 INFO:tasks.ceph.mon.a:Stopped 2024-05-09T06:37:53.576 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-05-09T06:37:53.576 INFO:teuthology.orchestra.run:waiting for 300 2024-05-09T06:37:53.609 INFO:tasks.ceph.mon.b:Stopped 2024-05-09T06:37:53.610 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-05-09T06:37:53.610 INFO:teuthology.orchestra.run:waiting for 300 2024-05-09T06:37:53.663 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~0s 2024-05-09T06:37:53.663 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~0s 2024-05-09T06:37:53.663 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~0s 2024-05-09T06:37:53.663 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~0s 2024-05-09T06:37:53.663 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~0s 2024-05-09T06:37:53.663 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~0s 2024-05-09T06:37:53.663 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~0s 2024-05-09T06:37:53.663 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~0s 2024-05-09T06:37:53.663 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~0s 2024-05-09T06:37:53.663 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~0s 2024-05-09T06:37:53.664 INFO:tasks.ceph.mon.c:Stopped 2024-05-09T06:37:53.664 INFO:tasks.ceph:Checking cluster log for badness... 2024-05-09T06:37:53.664 DEBUG:teuthology.orchestra.run.smithi003:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'OSD full dropping all updates' | egrep -v 'OSD near full' | egrep -v 'pausewr flag' | egrep -v 'failsafe engaged, dropping updates' | egrep -v 'failsafe disengaged, no longer dropping' | egrep -v 'is full \(reached quota' | egrep -v POOL_FULL | egrep -v POOL_BACKFILLFULL | head -n 1 2024-05-09T06:37:53.700 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi003.front.sepia.ceph.com 2024-05-09T06:37:53.700 DEBUG:teuthology.orchestra.run.smithi003:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-05-09T06:37:53.916 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi003.front.sepia.ceph.com 2024-05-09T06:37:53.916 DEBUG:teuthology.orchestra.run.smithi003:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-05-09T06:37:53.998 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi003.front.sepia.ceph.com 2024-05-09T06:37:53.998 DEBUG:teuthology.orchestra.run.smithi003:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-05-09T06:37:54.056 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi003.front.sepia.ceph.com 2024-05-09T06:37:54.056 DEBUG:teuthology.orchestra.run.smithi003:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-05-09T06:37:54.106 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi190.front.sepia.ceph.com 2024-05-09T06:37:54.107 DEBUG:teuthology.orchestra.run.smithi190:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-05-09T06:37:54.290 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi190.front.sepia.ceph.com 2024-05-09T06:37:54.290 DEBUG:teuthology.orchestra.run.smithi190:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-05-09T06:37:54.363 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi190.front.sepia.ceph.com 2024-05-09T06:37:54.364 DEBUG:teuthology.orchestra.run.smithi190:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-05-09T06:37:54.419 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi190.front.sepia.ceph.com 2024-05-09T06:37:54.420 DEBUG:teuthology.orchestra.run.smithi190:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-05-09T06:37:54.477 INFO:tasks.ceph:Archiving mon data... 2024-05-09T06:37:54.478 DEBUG:teuthology.misc:Transferring archived files from smithi003:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/teuthology-2024-05-08_21:24:04-fs-squid-distro-default-smithi/7698541/data/mon.a.tgz 2024-05-09T06:37:54.478 DEBUG:teuthology.orchestra.run.smithi003:> mktemp 2024-05-09T06:37:54.495 INFO:teuthology.orchestra.run.smithi003.stdout:/tmp/tmp.gEloYwafPA 2024-05-09T06:37:54.495 DEBUG:teuthology.orchestra.run.smithi003:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.gEloYwafPA 2024-05-09T06:37:54.643 DEBUG:teuthology.orchestra.run.smithi003:> sudo chmod 0666 /tmp/tmp.gEloYwafPA 2024-05-09T06:37:54.728 DEBUG:teuthology.orchestra.remote:smithi003:/tmp/tmp.gEloYwafPA is 353KB 2024-05-09T06:37:54.825 DEBUG:teuthology.orchestra.run.smithi003:> rm -fr /tmp/tmp.gEloYwafPA 2024-05-09T06:37:54.841 DEBUG:teuthology.misc:Transferring archived files from smithi190:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/teuthology-2024-05-08_21:24:04-fs-squid-distro-default-smithi/7698541/data/mon.b.tgz 2024-05-09T06:37:54.841 DEBUG:teuthology.orchestra.run.smithi190:> mktemp 2024-05-09T06:37:54.860 INFO:teuthology.orchestra.run.smithi190.stdout:/tmp/tmp.A4EZ2rVf4I 2024-05-09T06:37:54.860 DEBUG:teuthology.orchestra.run.smithi190:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.A4EZ2rVf4I 2024-05-09T06:37:55.024 DEBUG:teuthology.orchestra.run.smithi190:> sudo chmod 0666 /tmp/tmp.A4EZ2rVf4I 2024-05-09T06:37:55.108 DEBUG:teuthology.orchestra.remote:smithi190:/tmp/tmp.A4EZ2rVf4I is 486KB 2024-05-09T06:37:55.206 DEBUG:teuthology.orchestra.run.smithi190:> rm -fr /tmp/tmp.A4EZ2rVf4I 2024-05-09T06:37:55.221 DEBUG:teuthology.misc:Transferring archived files from smithi190:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/teuthology-2024-05-08_21:24:04-fs-squid-distro-default-smithi/7698541/data/mon.c.tgz 2024-05-09T06:37:55.221 DEBUG:teuthology.orchestra.run.smithi190:> mktemp 2024-05-09T06:37:55.276 INFO:teuthology.orchestra.run.smithi190.stdout:/tmp/tmp.ixGncvp3yO 2024-05-09T06:37:55.276 DEBUG:teuthology.orchestra.run.smithi190:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.ixGncvp3yO 2024-05-09T06:37:55.422 DEBUG:teuthology.orchestra.run.smithi190:> sudo chmod 0666 /tmp/tmp.ixGncvp3yO 2024-05-09T06:37:55.505 DEBUG:teuthology.orchestra.remote:smithi190:/tmp/tmp.ixGncvp3yO is 486KB 2024-05-09T06:37:55.574 DEBUG:teuthology.orchestra.run.smithi190:> rm -fr /tmp/tmp.ixGncvp3yO 2024-05-09T06:37:55.589 INFO:tasks.ceph:Cleaning ceph cluster... 2024-05-09T06:37:55.589 DEBUG:teuthology.orchestra.run.smithi003:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-05-09T06:37:55.591 DEBUG:teuthology.orchestra.run.smithi190:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-05-09T06:37:55.674 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-05-09T06:37:55.707 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-05-09T06:37:55.707 INFO:tasks.ceph:Archiving crash dumps... 2024-05-09T06:37:55.709 DEBUG:teuthology.misc:Transferring archived files from smithi003:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-05-08_21:24:04-fs-squid-distro-default-smithi/7698541/remote/smithi003/crash 2024-05-09T06:37:55.710 DEBUG:teuthology.orchestra.run.smithi003:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-05-09T06:37:55.741 DEBUG:teuthology.misc:Transferring archived files from smithi190:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-05-08_21:24:04-fs-squid-distro-default-smithi/7698541/remote/smithi190/crash 2024-05-09T06:37:55.742 DEBUG:teuthology.orchestra.run.smithi190:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-05-09T06:37:55.771 INFO:tasks.ceph:Shutting down logrotate 2024-05-09T06:37:55.771 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-09T06:37:55.786 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-09T06:37:55.866 DEBUG:teuthology.orchestra.run.smithi003:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-05-09T06:37:55.893 DEBUG:teuthology.orchestra.run.smithi190:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-05-09T06:37:55.917 INFO:tasks.ceph:Compressing logs... 2024-05-09T06:37:55.918 DEBUG:teuthology.orchestra.run.smithi003:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-09T06:37:55.936 DEBUG:teuthology.orchestra.run.smithi190:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-09T06:37:55.960 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51492.log 2024-05-09T06:37:55.960 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41916.log 2024-05-09T06:37:56.005 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.41757.log 2024-05-09T06:37:56.005 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-05-09T06:37:56.005 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.1.41757.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35752.log 2024-05-09T06:37:56.005 INFO:teuthology.orchestra.run.smithi190.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.1.41757.log.gz 2024-05-09T06:37:56.005 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-05-09T06:37:56.005 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.35752.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35712.log 2024-05-09T06:37:56.006 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-05-09T06:37:56.006 INFO:teuthology.orchestra.run.smithi190.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.35752.log.gz 2024-05-09T06:37:56.006 INFO:teuthology.orchestra.run.smithi190.stderr: 91.6% -- replaced with /var/log/ceph/ceph.log.gz 2024-05-09T06:37:56.006 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.35712.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35672.log 2024-05-09T06:37:56.006 INFO:teuthology.orchestra.run.smithi190.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.35712.log.gz 2024-05-09T06:37:56.006 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2024-05-09T06:37:56.006 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.35672.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-05-09T06:37:56.006 INFO:teuthology.orchestra.run.smithi190.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.35672.log.gz 2024-05-09T06:37:56.006 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-05-09T06:37:56.006 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-05-09T06:37:56.006 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-osd.5.log: 93.6% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-05-09T06:37:56.006 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-05-09T06:37:56.006 INFO:teuthology.orchestra.run.smithi190.stderr: 91.7% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2024-05-09T06:37:56.006 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-05-09T06:37:56.007 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35541.log 2024-05-09T06:37:56.007 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.42740.log 2024-05-09T06:37:56.007 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.35541.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35632.log 2024-05-09T06:37:56.007 INFO:teuthology.orchestra.run.smithi190.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.35541.log.gz 2024-05-09T06:37:56.007 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.1.42740.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35592.log 2024-05-09T06:37:56.007 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.35632.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.42740.log.gz 2024-05-09T06:37:56.007 INFO:teuthology.orchestra.run.smithi190.stderr:gzip 87.0% -- replaced with /var/log/ceph/ceph-client.admin.35632.log.gz 2024-05-09T06:37:56.007 INFO:teuthology.orchestra.run.smithi190.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.35461.log 2024-05-09T06:37:56.007 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.3.42155.log 2024-05-09T06:37:56.007 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.35461.log: /var/log/ceph/ceph-client.admin.35592.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.35592.log.gz 2024-05-09T06:37:56.007 INFO:teuthology.orchestra.run.smithi190.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.35461.log.gz 2024-05-09T06:37:56.007 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35501.log 2024-05-09T06:37:56.007 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.3.42155.log: 0.0% -- replaced with /var/log/ceph/ceph-client.3.42155.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2024-05-09T06:37:56.007 INFO:teuthology.orchestra.run.smithi190.stderr: 2024-05-09T06:37:56.007 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-05-09T06:37:56.008 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.35501.log: /var/log/ceph/ceph-mds.d.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.35501.log.gz 2024-05-09T06:37:56.043 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-mon.b.log: 94.3% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2024-05-09T06:37:56.062 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.51492.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43148.log 2024-05-09T06:37:56.062 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.41916.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43147.log 2024-05-09T06:37:56.062 INFO:teuthology.orchestra.run.smithi003.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.51492.log.gz 2024-05-09T06:37:56.062 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.43148.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48793.log 2024-05-09T06:37:56.062 INFO:teuthology.orchestra.run.smithi003.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.41916.log.gz 2024-05-09T06:37:56.062 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.43147.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.43148.log.gz 2024-05-09T06:37:56.062 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42811.log 2024-05-09T06:37:56.062 INFO:teuthology.orchestra.run.smithi003.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.43147.log.gz 2024-05-09T06:37:56.062 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.48793.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41296.log 2024-05-09T06:37:56.062 INFO:teuthology.orchestra.run.smithi003.stderr:gzip/var/log/ceph/ceph-client.admin.42811.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.41642.log 2024-05-09T06:37:56.063 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.41296.log: gzip 88.4% -5 -- replaced with /var/log/ceph/ceph-client.admin.42811.log.gz --verbose 2024-05-09T06:37:56.063 INFO:teuthology.orchestra.run.smithi003.stderr: -- /var/log/ceph/ceph-client.admin.48625.log 2024-05-09T06:37:56.063 INFO:teuthology.orchestra.run.smithi003.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.48793.log.gz 2024-05-09T06:37:56.063 INFO:teuthology.orchestra.run.smithi003.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.41296.log.gz 2024-05-09T06:37:56.063 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.41642.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41600.log 2024-05-09T06:37:56.063 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.48625.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.41642.log.gz 2024-05-09T06:37:56.063 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42252.log 2024-05-09T06:37:56.063 INFO:teuthology.orchestra.run.smithi003.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.48625.log.gz 2024-05-09T06:37:56.063 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.41600.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50133.log 2024-05-09T06:37:56.063 INFO:teuthology.orchestra.run.smithi003.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.41600.log.gz 2024-05-09T06:37:56.063 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.42252.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51870.log 2024-05-09T06:37:56.063 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.50133.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44112.log 2024-05-09T06:37:56.063 INFO:teuthology.orchestra.run.smithi003.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.42252.log.gz 2024-05-09T06:37:56.063 INFO:teuthology.orchestra.run.smithi003.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.50133.log.gz 2024-05-09T06:37:56.063 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.51870.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45161.log 2024-05-09T06:37:56.064 INFO:teuthology.orchestra.run.smithi003.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.51870.log.gz 2024-05-09T06:37:56.064 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.44112.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35843.log 2024-05-09T06:37:56.064 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.45161.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.44112.log.gz 2024-05-09T06:37:56.064 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43344.log 2024-05-09T06:37:56.064 INFO:teuthology.orchestra.run.smithi003.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.45161.log.gz 2024-05-09T06:37:56.064 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.35843.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43470.log 2024-05-09T06:37:56.064 INFO:teuthology.orchestra.run.smithi003.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.35843.log.gz 2024-05-09T06:37:56.064 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.43344.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48369.log 2024-05-09T06:37:56.064 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.43470.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.43344.log.gz 2024-05-09T06:37:56.064 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51996.log 2024-05-09T06:37:56.064 INFO:teuthology.orchestra.run.smithi003.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.43470.log.gz 2024-05-09T06:37:56.064 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.48369.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45539.log 2024-05-09T06:37:56.064 INFO:teuthology.orchestra.run.smithi003.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.48369.log.gz 2024-05-09T06:37:56.064 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.51996.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43848.log 2024-05-09T06:37:56.065 INFO:teuthology.orchestra.run.smithi003.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.51996.log.gz 2024-05-09T06:37:56.065 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.45539.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42803.log 2024-05-09T06:37:56.065 INFO:teuthology.orchestra.run.smithi003.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.45539.log.gz 2024-05-09T06:37:56.065 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.43848.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50714.log 2024-05-09T06:37:56.065 INFO:teuthology.orchestra.run.smithi003.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.43848.log.gz 2024-05-09T06:37:56.065 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.42803.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45371.log 2024-05-09T06:37:56.065 INFO:teuthology.orchestra.run.smithi003.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.42803.log.gz 2024-05-09T06:37:56.065 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.50714.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47564.log 2024-05-09T06:37:56.065 INFO:teuthology.orchestra.run.smithi003.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.50714.log.gz 2024-05-09T06:37:56.065 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.45371.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47438.log 2024-05-09T06:37:56.065 INFO:teuthology.orchestra.run.smithi003.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.45371.log.gz 2024-05-09T06:37:56.065 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.47564.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41684.log 2024-05-09T06:37:56.065 INFO:teuthology.orchestra.run.smithi003.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.47564.log.gz 2024-05-09T06:37:56.065 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.47438.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49228.log 2024-05-09T06:37:56.065 INFO:teuthology.orchestra.run.smithi003.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.47438.log.gz 2024-05-09T06:37:56.066 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.41684.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46688.log 2024-05-09T06:37:56.066 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.49228.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.41684.log.gz 2024-05-09T06:37:56.066 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43085.log 2024-05-09T06:37:56.066 INFO:teuthology.orchestra.run.smithi003.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.49228.log.gz/var/log/ceph/ceph-client.admin.46688.log: 2024-05-09T06:37:56.066 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49270.log 2024-05-09T06:37:56.066 INFO:teuthology.orchestra.run.smithi003.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.46688.log.gz 2024-05-09T06:37:56.066 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.43085.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-05-09T06:37:56.066 INFO:teuthology.orchestra.run.smithi003.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.43085.log.gz 2024-05-09T06:37:56.066 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.49270.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42810.log 2024-05-09T06:37:56.066 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.49270.log.gz 2024-05-09T06:37:56.066 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43512.log 2024-05-09T06:37:56.066 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.42810.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47606.log 2024-05-09T06:37:56.066 INFO:teuthology.orchestra.run.smithi003.stderr: 85.0% -- replaced with /var/log/ceph/ceph.log.gz 2024-05-09T06:37:56.066 INFO:teuthology.orchestra.run.smithi003.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.42810.log.gz 2024-05-09T06:37:56.067 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.43512.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51786.log 2024-05-09T06:37:56.067 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.47606.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.43512.log.gz 2024-05-09T06:37:56.067 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42000.log 2024-05-09T06:37:56.067 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.51786.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.47606.log.gz 2024-05-09T06:37:56.067 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42812.log 2024-05-09T06:37:56.067 INFO:teuthology.orchestra.run.smithi003.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.51786.log.gz 2024-05-09T06:37:56.067 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.42000.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.44461.log 2024-05-09T06:37:56.067 INFO:teuthology.orchestra.run.smithi003.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.42000.log.gz 2024-05-09T06:37:56.067 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.42812.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43974.log 2024-05-09T06:37:56.067 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.0.44461.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42042.log 2024-05-09T06:37:56.067 INFO:teuthology.orchestra.run.smithi003.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.42812.log.gz 2024-05-09T06:37:56.067 INFO:teuthology.orchestra.run.smithi003.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.44461.log.gz 2024-05-09T06:37:56.067 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.43974.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47060.log 2024-05-09T06:37:56.067 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.42042.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46184.log 2024-05-09T06:37:56.067 INFO:teuthology.orchestra.run.smithi003.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.43974.log.gz 2024-05-09T06:37:56.068 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.47060.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.42042.log.gz 2024-05-09T06:37:56.068 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-05-09T06:37:56.068 INFO:teuthology.orchestra.run.smithi003.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.47060.log.gz 2024-05-09T06:37:56.068 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.46184.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45623.log 2024-05-09T06:37:56.068 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-osd.2.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.46184.log.gz 2024-05-09T06:37:56.068 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43428.log 2024-05-09T06:37:56.068 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.45623.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46310.log 2024-05-09T06:37:56.068 INFO:teuthology.orchestra.run.smithi003.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.45623.log.gz 2024-05-09T06:37:56.068 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.43428.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46976.log 2024-05-09T06:37:56.068 INFO:teuthology.orchestra.run.smithi003.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.43428.log.gz 2024-05-09T06:37:56.068 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.46310.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43008.log 2024-05-09T06:37:56.068 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.46976.log: gzip -5 88.2% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.46310.log.gz /var/log/ceph/ceph-client.admin.42468.log 2024-05-09T06:37:56.068 INFO:teuthology.orchestra.run.smithi003.stderr: 2024-05-09T06:37:56.069 INFO:teuthology.orchestra.run.smithi003.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.46976.log.gz 2024-05-09T06:37:56.069 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.43008.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43554.log 2024-05-09T06:37:56.069 INFO:teuthology.orchestra.run.smithi003.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.43008.log.gz 2024-05-09T06:37:56.069 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.42468.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-05-09T06:37:56.069 INFO:teuthology.orchestra.run.smithi003.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.42468.log.gz 2024-05-09T06:37:56.069 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.43554.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47270.log 2024-05-09T06:37:56.069 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49312.log 2024-05-09T06:37:56.069 INFO:teuthology.orchestra.run.smithi003.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.43554.log.gz 2024-05-09T06:37:56.069 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.47270.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42471.log 2024-05-09T06:37:56.069 INFO:teuthology.orchestra.run.smithi003.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.47270.log.gz 2024-05-09T06:37:56.069 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.49312.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49845.log 2024-05-09T06:37:56.069 INFO:teuthology.orchestra.run.smithi003.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.49312.log.gz 2024-05-09T06:37:56.069 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43722.log 2024-05-09T06:37:56.069 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.42471.log: 88.9% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-05-09T06:37:56.069 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.49845.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50049.log 2024-05-09T06:37:56.069 INFO:teuthology.orchestra.run.smithi003.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.42471.log.gz 2024-05-09T06:37:56.070 INFO:teuthology.orchestra.run.smithi003.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.49845.log.gz 2024-05-09T06:37:56.070 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.43722.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51660.log 2024-05-09T06:37:56.070 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.50049.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.43722.log.gz 2024-05-09T06:37:56.070 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51828.log 2024-05-09T06:37:56.070 INFO:teuthology.orchestra.run.smithi003.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.50049.log.gz 2024-05-09T06:37:56.070 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.51660.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50091.log 2024-05-09T06:37:56.070 INFO:teuthology.orchestra.run.smithi003.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.51660.log.gz 2024-05-09T06:37:56.070 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.51828.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49887.log 2024-05-09T06:37:56.070 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.50091.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51198.log 2024-05-09T06:37:56.070 INFO:teuthology.orchestra.run.smithi003.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.51828.log.gz 2024-05-09T06:37:56.070 INFO:teuthology.orchestra.run.smithi003.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.50091.log.gz 2024-05-09T06:37:56.070 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.49887.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51912.log 2024-05-09T06:37:56.070 INFO:teuthology.orchestra.run.smithi003.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.49887.log.gz 2024-05-09T06:37:56.070 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.51198.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46772.log 2024-05-09T06:37:56.071 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.51912.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.51198.log.gz 2024-05-09T06:37:56.071 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50882.log 2024-05-09T06:37:56.071 INFO:teuthology.orchestra.run.smithi003.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.51912.log.gz 2024-05-09T06:37:56.071 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.46772.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-05-09T06:37:56.071 INFO:teuthology.orchestra.run.smithi003.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.46772.log.gz 2024-05-09T06:37:56.071 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.50882.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46646.log 2024-05-09T06:37:56.071 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-osd.3.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.50882.log.gz 2024-05-09T06:37:56.071 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50671.log 2024-05-09T06:37:56.071 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.46646.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49354.log 2024-05-09T06:37:56.071 INFO:teuthology.orchestra.run.smithi003.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.46646.log.gz 2024-05-09T06:37:56.071 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.50671.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51702.log 2024-05-09T06:37:56.071 INFO:teuthology.orchestra.run.smithi003.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.50671.log.gz 2024-05-09T06:37:56.071 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49606.log 2024-05-09T06:37:56.071 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.49354.log: /var/log/ceph/ceph-client.admin.51702.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41515.log 2024-05-09T06:37:56.072 INFO:teuthology.orchestra.run.smithi003.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.49354.log.gz 2024-05-09T06:37:56.072 INFO:teuthology.orchestra.run.smithi003.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.51702.log.gz 2024-05-09T06:37:56.072 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.49606.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48499.log 2024-05-09T06:37:56.072 INFO:teuthology.orchestra.run.smithi003.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.49606.log.gz 2024-05-09T06:37:56.072 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50275.log 2024-05-09T06:37:56.072 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.41515.log: /var/log/ceph/ceph-client.admin.48499.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43596.log 2024-05-09T06:37:56.072 INFO:teuthology.orchestra.run.smithi003.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.41515.log.gz 2024-05-09T06:37:56.072 INFO:teuthology.orchestra.run.smithi003.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.48499.log.gz 2024-05-09T06:37:56.072 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.50275.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50840.log 2024-05-09T06:37:56.072 INFO:teuthology.orchestra.run.smithi003.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.50275.log.gz 2024-05-09T06:37:56.072 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48020.log 2024-05-09T06:37:56.072 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.43596.log: /var/log/ceph/ceph-client.admin.50840.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41770.log 2024-05-09T06:37:56.072 INFO:teuthology.orchestra.run.smithi003.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.43596.log.gz 2024-05-09T06:37:56.072 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.48020.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.50840.log.gz 2024-05-09T06:37:56.072 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2024-05-09T06:37:56.073 INFO:teuthology.orchestra.run.smithi003.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.48020.log.gz 2024-05-09T06:37:56.073 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49396.log 2024-05-09T06:37:56.073 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.41770.log: /var/log/ceph/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49522.log 2024-05-09T06:37:56.073 INFO:teuthology.orchestra.run.smithi003.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.41770.log.gz 2024-05-09T06:37:56.073 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.49396.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48411.log 2024-05-09T06:37:56.073 INFO:teuthology.orchestra.run.smithi003.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.49396.log.gz 2024-05-09T06:37:56.073 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.49522.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43087.log 2024-05-09T06:37:56.073 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41726.log 2024-05-09T06:37:56.073 INFO:teuthology.orchestra.run.smithi003.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.49522.log.gz 2024-05-09T06:37:56.073 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.48411.log: /var/log/ceph/ceph-client.admin.43087.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42475.log 2024-05-09T06:37:56.073 INFO:teuthology.orchestra.run.smithi003.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.48411.log.gz 2024-05-09T06:37:56.073 INFO:teuthology.orchestra.run.smithi003.stderr: 87.8%/var/log/ceph/ceph-client.admin.41726.log: -- replaced with /var/log/ceph/ceph-client.admin.43087.log.gz 2024-05-09T06:37:56.073 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35724.log 2024-05-09T06:37:56.073 INFO:teuthology.orchestra.run.smithi003.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.41726.log.gz 2024-05-09T06:37:56.073 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.42475.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46934.log 2024-05-09T06:37:56.074 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-05-09T06:37:56.074 INFO:teuthology.orchestra.run.smithi003.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.42475.log.gz 2024-05-09T06:37:56.074 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.35724.log: /var/log/ceph/ceph-client.admin.46934.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.35724.log.gz 2024-05-09T06:37:56.074 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42808.log 2024-05-09T06:37:56.074 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-osd.1.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.46934.log.gz 2024-05-09T06:37:56.074 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-05-09T06:37:56.074 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.42808.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50007.log 2024-05-09T06:37:56.074 INFO:teuthology.orchestra.run.smithi003.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.42808.log.gz 2024-05-09T06:37:56.074 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47892.log 2024-05-09T06:37:56.074 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.50007.log: gzip -5 --verbose -- 88.7% /var/log/ceph/ceph-client.admin.46472.log 2024-05-09T06:37:56.074 INFO:teuthology.orchestra.run.smithi003.stderr: -- replaced with /var/log/ceph/ceph-client.admin.50007.log.gz 2024-05-09T06:37:56.074 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.47892.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47186.log 2024-05-09T06:37:56.074 INFO:teuthology.orchestra.run.smithi003.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.47892.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42084.log 2024-05-09T06:37:56.074 INFO:teuthology.orchestra.run.smithi003.stderr: 2024-05-09T06:37:56.075 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.46472.log: /var/log/ceph/ceph-client.admin.47186.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.46472.log.gz 2024-05-09T06:37:56.075 INFO:teuthology.orchestra.run.smithi003.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.47186.log.gz 2024-05-09T06:37:56.075 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52150.log 2024-05-09T06:37:56.075 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.42084.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45497.log 2024-05-09T06:37:56.075 INFO:teuthology.orchestra.run.smithi003.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.42084.log.gz 2024-05-09T06:37:56.075 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.52150.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-05-09T06:37:56.075 INFO:teuthology.orchestra.run.smithi003.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.52150.log.gz 2024-05-09T06:37:56.075 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.45497.log: 92.5% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-05-09T06:37:56.075 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43932.log 2024-05-09T06:37:56.075 INFO:teuthology.orchestra.run.smithi003.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.45497.log.gz 2024-05-09T06:37:56.075 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47728.log 2024-05-09T06:37:56.075 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-osd.0.log: /var/log/ceph/ceph-client.admin.43932.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46892.log 2024-05-09T06:37:56.075 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45287.log 2024-05-09T06:37:56.075 INFO:teuthology.orchestra.run.smithi003.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.43932.log.gz 2024-05-09T06:37:56.075 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.47728.log: gzip -5/var/log/ceph/ceph-client.admin.46892.log: --verbose -- /var/log/ceph/ceph-client.admin.42474.log 2024-05-09T06:37:56.076 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.45287.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.46892.log.gz 2024-05-09T06:37:56.076 INFO:teuthology.orchestra.run.smithi003.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.47728.log.gz 2024-05-09T06:37:56.076 INFO:teuthology.orchestra.run.smithi003.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.45287.log.gz 2024-05-09T06:37:56.076 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48453.log 2024-05-09T06:37:56.076 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.42474.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51324.log 2024-05-09T06:37:56.076 INFO:teuthology.orchestra.run.smithi003.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.42474.log.gz 2024-05-09T06:37:56.076 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.48453.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.2.44901.log 2024-05-09T06:37:56.076 INFO:teuthology.orchestra.run.smithi003.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.48453.log.gz 2024-05-09T06:37:56.076 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.51324.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48104.log 2024-05-09T06:37:56.076 INFO:teuthology.orchestra.run.smithi003.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.51324.log.gz 2024-05-09T06:37:56.076 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50611.log 2024-05-09T06:37:56.076 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.2.44901.log: /var/log/ceph/ceph-client.admin.48104.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47018.log 2024-05-09T06:37:56.076 INFO:teuthology.orchestra.run.smithi003.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.2.44901.log.gz 2024-05-09T06:37:56.076 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.50611.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41958.log 2024-05-09T06:37:56.077 INFO:teuthology.orchestra.run.smithi003.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.48104.log.gz 2024-05-09T06:37:56.077 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45791.log 2024-05-09T06:37:56.077 INFO:teuthology.orchestra.run.smithi003.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.50611.log.gz 2024-05-09T06:37:56.077 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.47018.log: /var/log/ceph/ceph-client.admin.41958.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.47018.log.gz 2024-05-09T06:37:56.077 INFO:teuthology.orchestra.run.smithi003.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.41958.log.gz 2024-05-09T06:37:56.077 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45749.log 2024-05-09T06:37:56.077 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.45791.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49126.log 2024-05-09T06:37:56.077 INFO:teuthology.orchestra.run.smithi003.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.45791.log.gz 2024-05-09T06:37:56.077 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.45749.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51576.log 2024-05-09T06:37:56.077 INFO:teuthology.orchestra.run.smithi003.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.45749.log.gz 2024-05-09T06:37:56.077 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.49126.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42809.log 2024-05-09T06:37:56.077 INFO:teuthology.orchestra.run.smithi003.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.49126.log.gz 2024-05-09T06:37:56.077 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41392.log 2024-05-09T06:37:56.077 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.51576.log: /var/log/ceph/ceph-client.admin.42809.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.51576.log.gz 2024-05-09T06:37:56.078 INFO:teuthology.orchestra.run.smithi003.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.42809.log.gz 2024-05-09T06:37:56.078 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43086.log 2024-05-09T06:37:56.078 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.41392.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49803.log 2024-05-09T06:37:56.078 INFO:teuthology.orchestra.run.smithi003.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.41392.log.gz 2024-05-09T06:37:56.078 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.43086.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41473.log 2024-05-09T06:37:56.078 INFO:teuthology.orchestra.run.smithi003.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.43086.log.gz 2024-05-09T06:37:56.078 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.49803.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41215.log 2024-05-09T06:37:56.078 INFO:teuthology.orchestra.run.smithi003.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.49803.log.gz 2024-05-09T06:37:56.078 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.41473.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-05-09T06:37:56.078 INFO:teuthology.orchestra.run.smithi003.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.41473.log.gz 2024-05-09T06:37:56.078 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.41215.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48318.log 2024-05-09T06:37:56.078 INFO:teuthology.orchestra.run.smithi003.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.41215.log.gz 2024-05-09T06:37:56.078 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41832.log 2024-05-09T06:37:56.078 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-mon.a.log: /var/log/ceph/ceph-client.admin.48318.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42470.log 2024-05-09T06:37:56.079 INFO:teuthology.orchestra.run.smithi003.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.48318.log.gz/var/log/ceph/ceph-client.admin.41832.log: 2024-05-09T06:37:56.079 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46030.log 2024-05-09T06:37:56.079 INFO:teuthology.orchestra.run.smithi003.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.41832.log.gz 2024-05-09T06:37:56.079 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.42470.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51282.log 2024-05-09T06:37:56.079 INFO:teuthology.orchestra.run.smithi003.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.42470.log.gz/var/log/ceph/ceph-client.admin.46030.log: 2024-05-09T06:37:56.079 INFO:teuthology.orchestra.run.smithi003.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.46030.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42798.log 2024-05-09T06:37:56.079 INFO:teuthology.orchestra.run.smithi003.stderr: 2024-05-09T06:37:56.079 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.51282.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49186.log 2024-05-09T06:37:56.079 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.42798.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.51282.log.gz 2024-05-09T06:37:56.079 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42469.log 2024-05-09T06:37:56.079 INFO:teuthology.orchestra.run.smithi003.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.42798.log.gz 2024-05-09T06:37:56.079 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.49186.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45917.log 2024-05-09T06:37:56.079 INFO:teuthology.orchestra.run.smithi003.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.49186.log.gz/var/log/ceph/ceph-client.admin.42469.log: 2024-05-09T06:37:56.079 INFO:teuthology.orchestra.run.smithi003.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.42469.log.gz 2024-05-09T06:37:56.079 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45707.log 2024-05-09T06:37:56.080 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.45917.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51408.log 2024-05-09T06:37:56.080 INFO:teuthology.orchestra.run.smithi003.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.45917.log.gz 2024-05-09T06:37:56.080 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.45707.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49719.log 2024-05-09T06:37:56.080 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.51408.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.45707.log.gz 2024-05-09T06:37:56.080 INFO:teuthology.orchestra.run.smithi003.stderr: 88.3%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45960.log 2024-05-09T06:37:56.080 INFO:teuthology.orchestra.run.smithi003.stderr: -- replaced with /var/log/ceph/ceph-client.admin.51408.log.gz 2024-05-09T06:37:56.080 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.49719.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.49719.log.gz 2024-05-09T06:37:56.080 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49564.log 2024-05-09T06:37:56.080 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.29387.log 2024-05-09T06:37:56.080 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.45960.log: /var/log/ceph/ceph-client.admin.49564.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.45960.log.gz 2024-05-09T06:37:56.080 INFO:teuthology.orchestra.run.smithi003.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.49564.log.gzgzip 2024-05-09T06:37:56.080 INFO:teuthology.orchestra.run.smithi003.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.45665.log 2024-05-09T06:37:56.080 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph.tmp-client.admin.29387.log: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.29387.log.gz 2024-05-09T06:37:56.081 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48541.log 2024-05-09T06:37:56.081 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52108.log 2024-05-09T06:37:56.081 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.45665.log: /var/log/ceph/ceph-client.admin.48541.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.45665.log.gz 2024-05-09T06:37:56.081 INFO:teuthology.orchestra.run.smithi003.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.48541.log.gz 2024-05-09T06:37:56.081 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50443.log 2024-05-09T06:37:56.081 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.52108.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.52108.log.gz 2024-05-09T06:37:56.081 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47312.log 2024-05-09T06:37:56.081 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48188.log 2024-05-09T06:37:56.081 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.50443.log: /var/log/ceph/ceph-client.admin.47312.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.50443.log.gz 2024-05-09T06:37:56.081 INFO:teuthology.orchestra.run.smithi003.stderr: 88.8%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46142.log 2024-05-09T06:37:56.081 INFO:teuthology.orchestra.run.smithi003.stderr: -- replaced with /var/log/ceph/ceph-client.admin.47312.log.gz 2024-05-09T06:37:56.081 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.48188.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.48188.log.gz 2024-05-09T06:37:56.081 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46226.log 2024-05-09T06:37:56.081 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49480.log 2024-05-09T06:37:56.081 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.46142.log: /var/log/ceph/ceph-client.admin.46226.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.46142.log.gz 2024-05-09T06:37:56.082 INFO:teuthology.orchestra.run.smithi003.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.46226.log.gz 2024-05-09T06:37:56.082 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43146.log 2024-05-09T06:37:56.082 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.49480.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.49480.log.gz 2024-05-09T06:37:56.082 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46268.log 2024-05-09T06:37:56.082 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50485.log 2024-05-09T06:37:56.082 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.43146.log: /var/log/ceph/ceph-client.admin.46268.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.43146.log.gz 2024-05-09T06:37:56.082 INFO:teuthology.orchestra.run.smithi003.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.46268.log.gz 2024-05-09T06:37:56.082 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50798.log 2024-05-09T06:37:56.082 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.50485.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.50485.log.gz 2024-05-09T06:37:56.082 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45119.log 2024-05-09T06:37:56.082 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42168.log 2024-05-09T06:37:56.082 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.50798.log: /var/log/ceph/ceph-client.admin.45119.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.50798.log.gz 2024-05-09T06:37:56.082 INFO:teuthology.orchestra.run.smithi003.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.45119.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45035.log 2024-05-09T06:37:56.083 INFO:teuthology.orchestra.run.smithi003.stderr: 2024-05-09T06:37:56.083 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.42168.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.42168.log.gz 2024-05-09T06:37:56.083 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50966.log 2024-05-09T06:37:56.083 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46430.log 2024-05-09T06:37:56.083 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.45035.log: /var/log/ceph/ceph-client.admin.50966.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.45035.log.gz 2024-05-09T06:37:56.083 INFO:teuthology.orchestra.run.smithi003.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.50966.log.gz 2024-05-09T06:37:56.083 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46514.log 2024-05-09T06:37:56.083 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.46430.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.46430.log.gz 2024-05-09T06:37:56.083 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48751.log 2024-05-09T06:37:56.083 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42466.log 2024-05-09T06:37:56.083 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.46514.log: /var/log/ceph/ceph-client.admin.48751.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.46514.log.gz 2024-05-09T06:37:56.083 INFO:teuthology.orchestra.run.smithi003.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.48751.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45875.log 2024-05-09T06:37:56.083 INFO:teuthology.orchestra.run.smithi003.stderr: 2024-05-09T06:37:56.083 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.42466.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.42466.log.gz 2024-05-09T06:37:56.083 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50569.log 2024-05-09T06:37:56.083 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47850.log 2024-05-09T06:37:56.084 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.45875.log: /var/log/ceph/ceph-client.admin.50569.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.45875.log.gz 2024-05-09T06:37:56.084 INFO:teuthology.orchestra.run.smithi003.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.50569.log.gz 2024-05-09T06:37:56.084 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42126.log 2024-05-09T06:37:56.084 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.47850.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.47850.log.gz 2024-05-09T06:37:56.084 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45413.log 2024-05-09T06:37:56.084 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43680.log 2024-05-09T06:37:56.084 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.42126.log: /var/log/ceph/ceph-client.admin.45413.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.42126.log.gz 2024-05-09T06:37:56.084 INFO:teuthology.orchestra.run.smithi003.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.45413.log.gz 2024-05-09T06:37:56.084 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43386.log 2024-05-09T06:37:56.084 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.43680.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.43680.log.gz 2024-05-09T06:37:56.084 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51618.log 2024-05-09T06:37:56.084 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51450.log 2024-05-09T06:37:56.084 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.43386.log: /var/log/ceph/ceph-client.admin.51618.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.43386.log.gz 2024-05-09T06:37:56.084 INFO:teuthology.orchestra.run.smithi003.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.51618.log.gz 2024-05-09T06:37:56.085 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45245.log 2024-05-09T06:37:56.085 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.51450.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.51450.log.gz 2024-05-09T06:37:56.085 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42294.log 2024-05-09T06:37:56.085 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48274.log 2024-05-09T06:37:56.085 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.45245.log: /var/log/ceph/ceph-client.admin.42294.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.45245.log.gz 2024-05-09T06:37:56.085 INFO:teuthology.orchestra.run.smithi003.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.42294.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47102.log 2024-05-09T06:37:56.085 INFO:teuthology.orchestra.run.smithi003.stderr: 2024-05-09T06:37:56.085 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.48274.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.48274.log.gz 2024-05-09T06:37:56.085 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43764.log 2024-05-09T06:37:56.085 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50359.log 2024-05-09T06:37:56.085 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.47102.log: /var/log/ceph/ceph-client.admin.43764.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.47102.log.gz 2024-05-09T06:37:56.085 INFO:teuthology.orchestra.run.smithi003.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.43764.log.gz 2024-05-09T06:37:56.085 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47228.log 2024-05-09T06:37:56.085 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.50359.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.50359.log.gz 2024-05-09T06:37:56.085 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46730.log 2024-05-09T06:37:56.086 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46814.log 2024-05-09T06:37:56.086 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.47228.log: /var/log/ceph/ceph-client.admin.46730.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.47228.log.gz 2024-05-09T06:37:56.086 INFO:teuthology.orchestra.run.smithi003.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.46730.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51744.log 2024-05-09T06:37:56.086 INFO:teuthology.orchestra.run.smithi003.stderr: 2024-05-09T06:37:56.086 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.46814.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.46814.log.gz 2024-05-09T06:37:56.086 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45203.log 2024-05-09T06:37:56.086 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42796.log 2024-05-09T06:37:56.086 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.51744.log: /var/log/ceph/ceph-client.admin.45203.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.51744.log.gz 2024-05-09T06:37:56.086 INFO:teuthology.orchestra.run.smithi003.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.45203.log.gz 2024-05-09T06:37:56.086 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49438.log 2024-05-09T06:37:56.086 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.42796.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.42796.log.gz 2024-05-09T06:37:56.086 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50401.log 2024-05-09T06:37:56.086 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45329.log 2024-05-09T06:37:56.086 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.49438.log: /var/log/ceph/ceph-client.admin.50401.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.49438.log.gz 2024-05-09T06:37:56.086 INFO:teuthology.orchestra.run.smithi003.stderr: 89.3%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42476.log 2024-05-09T06:37:56.087 INFO:teuthology.orchestra.run.smithi003.stderr: -- replaced with /var/log/ceph/ceph-client.admin.50401.log.gz 2024-05-09T06:37:56.087 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.45329.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.45329.log.gz 2024-05-09T06:37:56.087 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45455.log 2024-05-09T06:37:56.087 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48859.log 2024-05-09T06:37:56.087 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.42476.log: /var/log/ceph/ceph-client.admin.45455.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.42476.log.gz 2024-05-09T06:37:56.087 INFO:teuthology.orchestra.run.smithi003.stderr: 89.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42210.log 2024-05-09T06:37:56.087 INFO:teuthology.orchestra.run.smithi003.stderr: -- replaced with /var/log/ceph/ceph-client.admin.45455.log.gz 2024-05-09T06:37:56.087 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.48859.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.48859.log.gz 2024-05-09T06:37:56.087 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43638.log 2024-05-09T06:37:56.087 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45833.log 2024-05-09T06:37:56.087 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.42210.log: /var/log/ceph/ceph-client.admin.43638.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.42210.log.gz 2024-05-09T06:37:56.087 INFO:teuthology.orchestra.run.smithi003.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.43638.log.gz 2024-05-09T06:37:56.087 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48146.log 2024-05-09T06:37:56.087 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49761.log 2024-05-09T06:37:56.087 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.45833.log: /var/log/ceph/ceph-client.admin.48146.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.45833.log.gz 2024-05-09T06:37:56.088 INFO:teuthology.orchestra.run.smithi003.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.48146.log.gz 2024-05-09T06:37:56.088 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43890.log 2024-05-09T06:37:56.088 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.49761.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.49761.log.gz 2024-05-09T06:37:56.088 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48583.log 2024-05-09T06:37:56.088 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51156.log 2024-05-09T06:37:56.088 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.43890.log: /var/log/ceph/ceph-client.admin.48583.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.43890.log.gz 2024-05-09T06:37:56.088 INFO:teuthology.orchestra.run.smithi003.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.48583.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51366.log 2024-05-09T06:37:56.088 INFO:teuthology.orchestra.run.smithi003.stderr: 2024-05-09T06:37:56.088 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.51156.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.51156.log.gz 2024-05-09T06:37:56.088 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47978.log 2024-05-09T06:37:56.088 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47396.log 2024-05-09T06:37:56.088 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.51366.log: /var/log/ceph/ceph-client.admin.47978.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.51366.log.gz 2024-05-09T06:37:56.088 INFO:teuthology.orchestra.run.smithi003.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.47978.log.gz 2024-05-09T06:37:56.088 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35460.log 2024-05-09T06:37:56.089 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.47396.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.47396.log.gz 2024-05-09T06:37:56.089 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50924.log 2024-05-09T06:37:56.089 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45581.log 2024-05-09T06:37:56.089 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.35460.log: /var/log/ceph/ceph-client.admin.50924.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.35460.log.gz 2024-05-09T06:37:56.090 INFO:teuthology.orchestra.run.smithi190.stderr: 91.7% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-05-09T06:37:56.090 INFO:teuthology.orchestra.run.smithi190.stderr: 92.5% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-05-09T06:37:56.102 INFO:teuthology.orchestra.run.smithi190.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-05-09T06:37:56.102 INFO:teuthology.orchestra.run.smithi190.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-05-09T06:37:56.105 INFO:teuthology.orchestra.run.smithi003.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.50924.log.gz 2024-05-09T06:37:56.105 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47522.log 2024-05-09T06:37:56.106 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.45581.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.45581.log.gz 2024-05-09T06:37:56.106 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50527.log 2024-05-09T06:37:56.106 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2024-05-09T06:37:56.106 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.47522.log: /var/log/ceph/ceph-client.admin.50527.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.47522.log.gz 2024-05-09T06:37:56.106 INFO:teuthology.orchestra.run.smithi003.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.50527.log.gz 2024-05-09T06:37:56.106 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46388.log 2024-05-09T06:37:56.106 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-mds.a.log: 91.7% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2024-05-09T06:37:56.106 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48230.log 2024-05-09T06:37:56.106 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50233.log 2024-05-09T06:37:56.106 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.46388.log: /var/log/ceph/ceph-client.admin.48230.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.46388.log.gz 2024-05-09T06:37:56.106 INFO:teuthology.orchestra.run.smithi003.stderr: 89.2%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50191.log 2024-05-09T06:37:56.106 INFO:teuthology.orchestra.run.smithi003.stderr: -- replaced with /var/log/ceph/ceph-client.admin.48230.log.gz 2024-05-09T06:37:56.106 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.50233.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.50233.log.gz 2024-05-09T06:37:56.106 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47648.log 2024-05-09T06:37:56.107 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50317.log 2024-05-09T06:37:56.107 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.50191.log: /var/log/ceph/ceph-client.admin.47648.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.50191.log.gz 2024-05-09T06:37:56.107 INFO:teuthology.orchestra.run.smithi003.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.47648.log.gz 2024-05-09T06:37:56.107 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52068.log 2024-05-09T06:37:56.107 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47354.log 2024-05-09T06:37:56.107 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.50317.log: /var/log/ceph/ceph-client.admin.52068.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.50317.log.gz 2024-05-09T06:37:56.107 INFO:teuthology.orchestra.run.smithi003.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.52068.log.gz 2024-05-09T06:37:56.107 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43806.log 2024-05-09T06:37:56.107 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47144.log 2024-05-09T06:37:56.107 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.47354.log: /var/log/ceph/ceph-client.admin.43806.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.47354.log.gz 2024-05-09T06:37:56.107 INFO:teuthology.orchestra.run.smithi003.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.43806.log.gz 2024-05-09T06:37:56.107 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51534.log 2024-05-09T06:37:56.107 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47936.log 2024-05-09T06:37:56.107 INFO:teuthology.orchestra.run.smithi003.stderr: 93.4% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-05-09T06:37:56.108 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.47144.log: /var/log/ceph/ceph-client.admin.51534.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.47144.log.gz 2024-05-09T06:37:56.108 INFO:teuthology.orchestra.run.smithi003.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.51534.log.gz 2024-05-09T06:37:56.108 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41874.log 2024-05-09T06:37:56.108 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.47936.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44597.log 2024-05-09T06:37:56.108 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.41874.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.47936.log.gz 2024-05-09T06:37:56.108 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49965.log 2024-05-09T06:37:56.108 INFO:teuthology.orchestra.run.smithi003.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.41874.log.gz 2024-05-09T06:37:56.108 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51954.log 2024-05-09T06:37:56.108 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.44597.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41558.log 2024-05-09T06:37:56.108 INFO:teuthology.orchestra.run.smithi003.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.44597.log.gz/var/log/ceph/ceph-client.admin.49965.log: 2024-05-09T06:37:56.108 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.51954.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.49047.log 2024-05-09T06:37:56.108 INFO:teuthology.orchestra.run.smithi003.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.49965.log.gz 2024-05-09T06:37:56.108 INFO:teuthology.orchestra.run.smithi003.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.51954.log.gz 2024-05-09T06:37:56.108 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.41558.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47480.log 2024-05-09T06:37:56.108 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.0.49047.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45077.log 2024-05-09T06:37:56.109 INFO:teuthology.orchestra.run.smithi003.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.49047.log.gz 2024-05-09T06:37:56.109 INFO:teuthology.orchestra.run.smithi003.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.41558.log.gz 2024-05-09T06:37:56.109 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.47480.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48062.log 2024-05-09T06:37:56.109 INFO:teuthology.orchestra.run.smithi003.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.47480.log.gz 2024-05-09T06:37:56.109 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51240.log 2024-05-09T06:37:56.109 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.45077.log: /var/log/ceph/ceph-client.admin.48062.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43088.log 2024-05-09T06:37:56.109 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.51240.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.45077.log.gz 2024-05-09T06:37:56.109 INFO:teuthology.orchestra.run.smithi003.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.48062.log.gz 2024-05-09T06:37:56.109 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46556.log 2024-05-09T06:37:56.109 INFO:teuthology.orchestra.run.smithi003.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.51240.log.gz 2024-05-09T06:37:56.109 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50756.log/var/log/ceph/ceph-client.admin.43088.log: 2024-05-09T06:37:56.109 INFO:teuthology.orchestra.run.smithi003.stderr: 89.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48709.log 2024-05-09T06:37:56.109 INFO:teuthology.orchestra.run.smithi003.stderr: -- replaced with /var/log/ceph/ceph-client.admin.43088.log.gz 2024-05-09T06:37:56.109 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.46556.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48667.log 2024-05-09T06:37:56.110 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.50756.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.46556.log.gz 2024-05-09T06:37:56.110 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.48709.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47808.log 2024-05-09T06:37:56.110 INFO:teuthology.orchestra.run.smithi003.stderr: 88.7% 88.4% -- replaced with /var/log/ceph/ceph-client.admin.48709.log.gz 2024-05-09T06:37:56.110 INFO:teuthology.orchestra.run.smithi003.stderr: -- replaced with /var/log/ceph/ceph-client.admin.50756.log.gz 2024-05-09T06:37:56.110 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.48667.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.48667.log.gz 2024-05-09T06:37:56.110 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.47808.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.47808.log.gz 2024-05-09T06:37:56.110 INFO:teuthology.orchestra.run.smithi003.stderr: 93.5% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-05-09T06:37:56.110 INFO:teuthology.orchestra.run.smithi190.stderr: 91.6% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-05-09T06:37:56.111 INFO:teuthology.orchestra.run.smithi190.stderr: 93.4% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-05-09T06:37:56.123 INFO:teuthology.orchestra.run.smithi003.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-05-09T06:37:56.137 INFO:teuthology.orchestra.run.smithi003.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-05-09T06:37:56.139 INFO:teuthology.orchestra.run.smithi003.stderr: 93.4% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2024-05-09T06:37:56.147 INFO:teuthology.orchestra.run.smithi190.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-05-09T06:37:56.150 INFO:teuthology.orchestra.run.smithi190.stderr: 2024-05-09T06:37:56.150 INFO:teuthology.orchestra.run.smithi190.stderr:real 0m0.177s 2024-05-09T06:37:56.150 INFO:teuthology.orchestra.run.smithi190.stderr:user 0m0.816s 2024-05-09T06:37:56.150 INFO:teuthology.orchestra.run.smithi190.stderr:sys 0m0.063s 2024-05-09T06:37:56.191 INFO:teuthology.orchestra.run.smithi003.stderr: 90.9% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-05-09T06:37:56.194 INFO:teuthology.orchestra.run.smithi003.stderr: 2024-05-09T06:37:56.194 INFO:teuthology.orchestra.run.smithi003.stderr:real 0m0.245s 2024-05-09T06:37:56.194 INFO:teuthology.orchestra.run.smithi003.stderr:user 0m0.949s 2024-05-09T06:37:56.194 INFO:teuthology.orchestra.run.smithi003.stderr:sys 0m0.174s 2024-05-09T06:37:56.195 INFO:tasks.ceph:Archiving logs... 2024-05-09T06:37:56.195 DEBUG:teuthology.misc:Transferring archived files from smithi003:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-05-08_21:24:04-fs-squid-distro-default-smithi/7698541/remote/smithi003/log 2024-05-09T06:37:56.196 DEBUG:teuthology.orchestra.run.smithi003:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-09T06:37:56.529 DEBUG:teuthology.misc:Transferring archived files from smithi190:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-05-08_21:24:04-fs-squid-distro-default-smithi/7698541/remote/smithi190/log 2024-05-09T06:37:56.530 DEBUG:teuthology.orchestra.run.smithi190:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-09T06:37:56.667 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-05-09T06:37:56.693 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-05-09T06:37:56.693 DEBUG:teuthology.orchestra.run.smithi003:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-09T06:37:56.696 DEBUG:teuthology.orchestra.run.smithi190:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-09T06:37:56.776 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-05-09T06:37:56.786 INFO:teuthology.task.clock:Checking final clock skew... 2024-05-09T06:37:56.786 DEBUG:teuthology.orchestra.run.smithi003:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-09T06:37:56.789 DEBUG:teuthology.orchestra.run.smithi190:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-09T06:37:56.802 INFO:teuthology.orchestra.run.smithi003.stderr:bash: line 1: ntpq: command not found 2024-05-09T06:37:56.803 INFO:teuthology.orchestra.run.smithi190.stderr:bash: line 1: ntpq: command not found 2024-05-09T06:37:56.806 INFO:teuthology.orchestra.run.smithi003.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-09T06:37:56.806 INFO:teuthology.orchestra.run.smithi003.stdout:=============================================================================== 2024-05-09T06:37:56.806 INFO:teuthology.orchestra.run.smithi003.stdout:^+ hv01.front.sepia.ceph.com 3 6 377 18 +1678us[+1678us] +/- 63ms 2024-05-09T06:37:56.806 INFO:teuthology.orchestra.run.smithi003.stdout:^* hv02.front.sepia.ceph.com 3 6 377 19 -853us[ -865us] +/- 55ms 2024-05-09T06:37:56.806 INFO:teuthology.orchestra.run.smithi003.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 19 -786us[ -798us] +/- 73ms 2024-05-09T06:37:56.807 INFO:teuthology.orchestra.run.smithi003.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-05-09T06:37:56.808 INFO:teuthology.orchestra.run.smithi190.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-09T06:37:56.808 INFO:teuthology.orchestra.run.smithi190.stdout:=============================================================================== 2024-05-09T06:37:56.808 INFO:teuthology.orchestra.run.smithi190.stdout:^+ hv01.front.sepia.ceph.com 3 6 377 19 +1675us[+1673us] +/- 63ms 2024-05-09T06:37:56.808 INFO:teuthology.orchestra.run.smithi190.stdout:^* hv02.front.sepia.ceph.com 3 6 377 18 -855us[ -858us] +/- 55ms 2024-05-09T06:37:56.808 INFO:teuthology.orchestra.run.smithi190.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 18 -782us[ -785us] +/- 73ms 2024-05-09T06:37:56.808 INFO:teuthology.orchestra.run.smithi190.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-05-09T06:37:56.809 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-05-09T06:37:56.819 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-05-09T06:37:56.819 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-05-09T06:37:56.831 DEBUG:teuthology.orchestra.run.smithi003:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-09T06:37:56.918 DEBUG:teuthology.orchestra.run.smithi190:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-09T06:37:56.979 DEBUG:teuthology.orchestra.run.smithi003:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-09T06:37:57.008 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T06:37:57.009 DEBUG:teuthology.orchestra.run.smithi190:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-09T06:37:57.036 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T06:37:57.036 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-05-09T06:37:57.046 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-05-09T06:37:57.059 INFO:teuthology.task.internal:Duration was 527.652556 seconds 2024-05-09T06:37:57.060 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-05-09T06:37:57.070 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-05-09T06:37:57.070 DEBUG:teuthology.orchestra.run.smithi003:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-09T06:37:57.073 DEBUG:teuthology.orchestra.run.smithi190:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-09T06:37:57.110 INFO:teuthology.orchestra.run.smithi003.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-09T06:37:57.114 INFO:teuthology.orchestra.run.smithi190.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-09T06:37:57.394 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-05-09T06:37:57.394 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi003.front.sepia.ceph.com 2024-05-09T06:37:57.394 DEBUG:teuthology.orchestra.run.smithi003:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-09T06:37:57.426 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi190.front.sepia.ceph.com 2024-05-09T06:37:57.426 DEBUG:teuthology.orchestra.run.smithi190:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-09T06:37:57.458 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-05-09T06:37:57.458 DEBUG:teuthology.orchestra.run.smithi003:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-09T06:37:57.469 DEBUG:teuthology.orchestra.run.smithi190:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-09T06:37:57.525 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-05-09T06:37:57.525 DEBUG:teuthology.orchestra.run.smithi003:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-09T06:37:57.537 DEBUG:teuthology.orchestra.run.smithi190:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-09T06:37:57.663 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-05-09T06:37:57.674 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-05-09T06:37:57.674 DEBUG:teuthology.orchestra.run.smithi003:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-09T06:37:57.700 DEBUG:teuthology.orchestra.run.smithi190:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-09T06:37:57.725 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-05-09T06:37:57.737 DEBUG:teuthology.orchestra.run.smithi003:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-09T06:37:57.743 DEBUG:teuthology.orchestra.run.smithi190:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-09T06:37:57.764 INFO:teuthology.orchestra.run.smithi003.stdout:kernel.core_pattern = core 2024-05-09T06:37:57.788 INFO:teuthology.orchestra.run.smithi190.stdout:kernel.core_pattern = core 2024-05-09T06:37:57.801 DEBUG:teuthology.orchestra.run.smithi003:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-09T06:37:57.833 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T06:37:57.834 DEBUG:teuthology.orchestra.run.smithi190:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-09T06:37:57.855 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T06:37:57.856 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-05-09T06:37:57.867 INFO:teuthology.task.internal:Transferring archived files... 2024-05-09T06:37:57.868 DEBUG:teuthology.misc:Transferring archived files from smithi003:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-05-08_21:24:04-fs-squid-distro-default-smithi/7698541/remote/smithi003 2024-05-09T06:37:57.868 DEBUG:teuthology.orchestra.run.smithi003:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-09T06:37:57.909 DEBUG:teuthology.misc:Transferring archived files from smithi190:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-05-08_21:24:04-fs-squid-distro-default-smithi/7698541/remote/smithi190 2024-05-09T06:37:57.909 DEBUG:teuthology.orchestra.run.smithi190:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-09T06:37:57.942 INFO:teuthology.task.internal:Removing archive directory... 2024-05-09T06:37:57.942 DEBUG:teuthology.orchestra.run.smithi003:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-09T06:37:57.946 DEBUG:teuthology.orchestra.run.smithi190:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-09T06:37:57.993 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-05-09T06:37:58.005 INFO:teuthology.task.internal:Not uploading archives. 2024-05-09T06:37:58.005 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-05-09T06:37:58.017 INFO:teuthology.task.internal:Tidying up after the test... 2024-05-09T06:37:58.017 DEBUG:teuthology.orchestra.run.smithi003:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-09T06:37:58.019 DEBUG:teuthology.orchestra.run.smithi190:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-09T06:37:58.033 INFO:teuthology.orchestra.run.smithi003.stdout: 262165 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 9 06:37 /home/ubuntu/cephtest 2024-05-09T06:37:58.048 INFO:teuthology.orchestra.run.smithi190.stdout: 262165 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 9 06:37 /home/ubuntu/cephtest 2024-05-09T06:37:58.049 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-05-09T06:37:58.061 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-05-09T06:37:58.109 INFO:teuthology.run:Summary data: description: fs/volumes/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/volumes/{overrides test/finisher_per_module}} duration: 527.6525564193726 flavor: default owner: scheduled_teuthology@teuthology success: true 2024-05-09T06:37:58.109 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-05-09T06:37:58.187 INFO:teuthology.run:pass